Media information processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a media information processing method and device, electronic equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing and deep learning. The specific implementation scheme is: obtaining a target object structure graph according to a phrase structure graph corresponding to to-be-processed media information, wherein the phrase structure graph includes at least one additional node corresponding to additional information, and the additional information includes information related to the to-be-processed media information; obtaining a target object representation vector set according to the target object structure graph; and obtaining a processing result for the to-be-processed media information according to the target object representation vector set.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of natural language processing and deep learning. Specifically, it relates to a media information processing method and device, an electronic device and a storage medium. BACKGROUND

[0002] Syntactic analysis plays an important role in the field of artificial intelligence, and more and more research and application in the field of artificial intelligence rely on syntactic analysis. For example, in the information extraction task, syntactic analysis can be used to analyze the predicate and argument in the sentence. SUMMARY

[0003] The present disclosure provides a media information processing method and device, an electronic device and a storage medium.

[0004] According to an aspect of the present disclosure, a media information processing method is provided, comprising: obtaining a target object structure graph according to a phrase structure graph corresponding to to-be-processed media information, wherein the phrase structure graph comprises at least one additional node corresponding to additional information, and the additional information comprises information related to the to-be-processed media information; obtaining a target object representation vector set according to the target object structure graph; and obtaining a processing result for the to-be-processed media information according to the target object representation vector set.

[0005] According to another aspect of the present disclosure, a media information processing device is provided, comprising: a first obtaining module configured to obtain a target object structure graph according to a phrase structure graph corresponding to to-be-processed media information, wherein the phrase structure graph comprises at least one additional node corresponding to additional information, and the additional information comprises information related to the to-be-processed media information; a second obtaining module configured to obtain a target object representation vector set according to the target object structure graph; and a third obtaining module configured to obtain a processing result for the to-be-processed media information according to the target object representation vector set.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to the present disclosure.

[0009] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0011] Figure 1 An exemplary system architecture to which the media information processing method and device according to embodiments of the present disclosure can be applied is schematically shown;

[0012] Figure 2 A flowchart of a media information processing method according to embodiments of the present disclosure is schematically shown;

[0013] Figure 3 An exemplary diagram of obtaining a first intermediate object structure graph according to a phrase structure graph corresponding to media information to be processed according to embodiments of the present disclosure is schematically shown;

[0014] Figure 4 An exemplary diagram of converting a first target object node in the first intermediate object structure graph into an edge to obtain a second intermediate object structure graph according to embodiments of the present disclosure is schematically shown;

[0015] Figure 5 An exemplary diagram of obtaining a target object structure graph according to the second intermediate object structure graph according to embodiments of the present disclosure is schematically shown;

[0016] Figure 6 A block diagram of a media information processing device according to embodiments of the present disclosure is schematically shown; and

[0017] Figure 7 A block diagram of an electronic device suitable for implementing a media information processing method according to embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, descriptions of well-known functions and structures are omitted in the following description.

[0019] The basic task of syntax analysis is to determine the syntax structure of the to-be-processed sentence or the dependency relationship between the words of the to-be-processed sentence. Syntax analysis is not the ultimate goal of the natural language processing task, but a key link to achieve the ultimate goal. The analysis granularity of syntax analysis can be "word". The granularity of "word" is small, and therefore, the difficulty of applying the result obtained by analyzing the to-be-processed sentence by using syntax analysis to the natural language processing task is increased.

[0020] To this end, it is found that the phrase structure graph can reduce the difficulty of applying the result obtained by analyzing the to-be-processed sentence by using syntax analysis to the natural language processing task. This is because the nodes in the phrase structure graph can be phrase nodes corresponding to "phrases", and because the granularity of "phrases" is large, and the phrase structure graph can include additional nodes corresponding to additional information related to the to-be-processed sentence, the phrase structure graph can provide additional information in addition to the to-be-processed sentence.

[0021] Because the representation vector of a word can be obtained based on word embedding, it is difficult to obtain the representation vector of a phrase, and therefore, it is difficult to obtain a processing result for the to-be-processed sentence by using the phrase structure graph, thereby affecting the quality of the processing result and increasing the difficulty of obtaining the processing result.

[0022] To this end, the embodiment of the present disclosure provides a media information processing scheme. For example, converting the phrase structure graph corresponding to the to-be-processed media information into a target object structure graph, obtaining a target object representation vector set according to the target object structure graph, and obtaining a processing result of the to-be-processed media information, which makes it possible to implement the process that is difficult to implement by using word embedding, provides the possibility of implementing syntax analysis by using a neural network, and improves the quality of the processing result of the to-be-processed media information and reduces the difficulty of obtaining the processing result.

[0023] Figure 1 An exemplary system architecture to which the media information processing method and device according to the embodiment of the present disclosure can be applied is schematically shown.

[0024] It should be noted that Figure 1 The shown is only an example of a system architecture to which the embodiment of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiment of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, the exemplary system architecture to which the media information processing method and device can be applied can include a terminal device, but the terminal device can not need to interact with the server to implement the media information processing method and device provided by the embodiment of the present disclosure.

[0025] As Figure 1As shown, the system architecture 100 according to this embodiment can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired and / or wireless communication links, etc.

[0026] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0027] The terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.

[0028] The server 105 can be various types of servers providing various services. For example, the server 105 can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server, virtual private server). The server 105 can also be a server of a distributed system, or a server combined with a blockchain.

[0029] It should be noted that the media information processing method provided by the embodiments of the present disclosure can generally be executed by the terminal devices 101, 102, or 103. Correspondingly, the media information processing apparatus provided by the embodiments of the present disclosure can also be arranged in the terminal devices 101, 102, or 103.

[0030] Alternatively, the media information processing method provided by the embodiments of the present disclosure can also be generally executed by the server 105. Correspondingly, the media information processing apparatus provided by the embodiments of the present disclosure can generally be arranged in the server 105. The media information processing method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the media information processing apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0031] It should be understood that, Figure 1The number of terminal devices, networks and servers in the figure is merely illustrative. According to implementation needs, there can be any number of terminal devices, networks and servers.

[0032] It should be noted that the serial numbers of the various operations in the following method are merely used as a representation of the operation to describe, and should not be regarded as representing the execution sequence of the various operations. Unless explicitly indicated, the method does not need to be executed in the order shown.

[0033] Figure 2 A flowchart of a media information processing method according to an embodiment of the disclosure is schematically shown.

[0034] As shown in Figure 2 The method 200 includes operations S210-S230.

[0035] In operation S210, a target object structure graph is obtained according to a phrase structure graph corresponding to media information to be processed. The phrase structure graph includes at least one additional node corresponding to additional information. The additional information includes information related to the media information to be processed.

[0036] In operation S220, a target object representation vector set is obtained according to the target object structure graph.

[0037] In operation S230, a processing result for the media information to be processed is obtained according to the target object representation vector set.

[0038] According to an embodiment of the disclosure, the media information to be processed can refer to media information that needs to be processed. The media information can include at least one sentence. The media information can include one of the following: text information and audio information. The media information can be information related to a natural language processing task. The natural language processing task can include at least one of the following: a natural language understanding task and a natural language generation task.

[0039] According to an embodiment of the disclosure, the media information to be processed can include a plurality of objects. The object can include a word or a word. For example, if the media information to be processed includes media information related to Chinese, the object can include a word. If the media information to be processed is media information related to English, the object can include a word.

[0040] According to an embodiment of the present disclosure, the media information to be processed can have additional information related to the media information to be processed. The additional information can refer to information that plays an auxiliary role in obtaining a processing result of the media information to be processed. The additional information can include at least one of relationship information and limitation information. The relationship information can refer to information related to indicating a relationship between objects included in the media information to be processed. The relationship information can include at least one of juxtaposition relationship information, modification relationship information, continuation relationship information, progression relationship information, selection relationship information, transition relationship information, assumption relationship information, causation relationship information, condition relationship information, explanation relationship information, and purpose relationship information. The relationship information can be indicated by a relationship word. The limitation information can refer to information for limiting predetermined information in the media information to be processed. For example, the limitation information can include function information that characterizes a limiting role. The additional information can include at least one object. The object can include a word or a term.

[0041] According to an embodiment of the present disclosure, the structure graph can include at least two nodes and at least one edge. The edge can be used to connect two nodes. Node information of the node can include at least one node tag corresponding to the node and at least one node attribute corresponding to the node tag. Edge information of the edge can include an edge tag of the edge and at least one edge attribute corresponding to the edge tag. The edge tag is used to characterize a relationship between two nodes connected by the edge. The structure graph can include at least two structure subgraphs. Each structure subgraph can have a structure graph relationship with each other. The structure graph relationship can include an isomorphic structure graph relationship or a non-isomorphic structure graph relationship. The isomorphic structure graph relationship can refer to that there is no dependency relationship between two structure subgraphs. The non-isomorphic structure graph relationship can refer to that there is a dependency relationship between two structure subgraphs. The structure graph can be generated according to media information and additional information related to the media information. The structure graph can include a phrase structure graph or an object structure graph.

[0042] According to an embodiment of the present disclosure, the phrase structure graph can refer to a structure graph in which at least one node is a phrase node. The phrase node can be a node generated according to a phrase. The phrase can include at least two objects. The phrase structure graph can be a structure graph in which all nodes are phrase nodes. The phrase structure graph can also be a structure graph including at least one phrase node and at least one non-phrase node. For example, the non-phrase node can include an object node.

[0043] For example, the media information to be processed is "I will be there, I am responsible for stage supervision?". The additional information corresponding to the media information to be processed can include relationship information and limitation information. The relationship information can be parataxis relationship information. The limitation information can be function information. The phrase structure graph corresponding to the media information to be processed can include non-phrase node "I", phrase node "will be there", non-phrase node "I", phrase node "am responsible for", phrase node "stage supervision", non-phrase node "parataxis" and non-phrase node "whether". The non-phrase node "parataxis" can be an additional node corresponding to the parataxis relationship information. The non-phrase node "whether" can be an additional node corresponding to the function information.

[0044] For example, the media information to be processed is "My bicycle is blue". The additional information corresponding to the media information to be processed can include relationship information. The relationship information can be modification relationship information. The phrase structure graph corresponding to the media information to be processed can include phrase node "my", phrase node "bicycle", non-phrase node "is", phrase node "blue" and non-phrase node "decorate". The non-phrase node "decorate" can be an additional node corresponding to the modification relationship information.

[0045] According to an embodiment of the present disclosure, the object structure graph can refer to a structure graph in which all nodes are object nodes. The object structure graph can include at least two object nodes and at least one edge for connecting the two object nodes. For example, the media information to be processed is "I will be there, I am responsible for stage supervision?". The target object structure graph corresponding to the media information to be processed can include object node "I", object node "will", object node "be", object node "there", object node "I", object node "am", object node "responsible", object node "parataxis" and object node "whether".

[0046] According to an embodiment of the present disclosure, the target object representation vector set can include a target object representation vector corresponding to an object node in the target object structure graph. The target object representation vector can be used to indicate the characteristics of the object itself corresponding to the object node and the relationship characteristics between the object and other objects.

[0047] According to an embodiment of the present disclosure, in response to receiving a media information processing request for the to-be-processed media information, a phrase structure graph corresponding to the to-be-processed media information can be obtained. For example, a media information obtaining interface can be invoked, and the to-be-processed media information can be obtained from a database by using the media information obtaining interface. According to the to-be-processed media information, additional information related to the to-be-processed media information can be obtained. For example, the to-be-processed media information can be analyzed to obtain a relationship between objects in the to-be-processed media information and punctuation marks of the to-be-processed media information. According to the relationship between the objects in the to-be-processed media information and the punctuation marks of the to-be-processed media information, the additional information related to the to-be-processed media information can be obtained. Alternatively, the additional information can be predefined information related to the to-be-processed media information. The to-be-processed media information and the additional information can be processed to obtain the phrase structure graph. For example, the to-be-processed media information and the additional information can be processed based on a syntax analysis method to obtain the phrase structure graph.

[0048] According to an embodiment of the present disclosure, after obtaining the phrase structure graph, the phrase structure graph can be processed to obtain a target object structure graph. For example, the phrase structure graph can be processed to obtain an intermediate object structure graph. According to the intermediate object structure graph, the target object structure graph can be obtained. For example, the intermediate object structure graph can include a first intermediate object structure graph and a second intermediate object structure graph. The phrase structure graph can be processed to obtain the first intermediate object structure graph. The first intermediate object structure graph can be converted in terms of object nodes and edges to obtain the second intermediate object structure graph. The second intermediate object structure graph can be processed to obtain the target object structure graph. Alternatively, the second intermediate object structure graph can be determined as the target object structure graph.

[0049] According to an embodiment of the present disclosure, the target object structure graph can be processed based on a word embedding method to obtain a target object representation vector set. For example, the target object structure graph can be processed by using a representation model to obtain the target object representation vector set. The representation vector model can be obtained by training a deep learning model by using sample media information.

[0050] According to an embodiment of the present disclosure, the target representation vector set can be processed to obtain a processing result of the to-be-processed media information. For example, in a case where the to-be-processed media information is media information related to an information extraction task, the processing result can be an information extraction result. In a case where the to-be-processed media information is media information related to a sentiment analysis task, the processing result can be a sentiment analysis result.

[0051] According to an embodiment of the present disclosure, the media information processing method can be executed by an electronic device. The electronic device can be a server or a terminal device. The electronic device can include at least one processor. The processor can be configured to execute the media information processing method provided by the embodiment of the present disclosure. For example, the media information processing method provided by the embodiment of the present disclosure can be executed by a single processor, or the media information processing method provided by the embodiment of the present disclosure can be executed by multiple processors in parallel.

[0052] According to an embodiment of the present disclosure, the media information processing method provided by the embodiment of the present disclosure can convert the phrase structure graph corresponding to the media information to be processed into a target object structure graph, and obtain a processing result of the media information to be processed by using a target object representation vector set obtained according to the target object structure graph. The media information processing method provided by the embodiment of the present disclosure can make the process that is difficult to be realized by using word embedding become possible, and provide a possibility for realizing syntax analysis by using a neural network. The media information processing method provided by the embodiment of the present disclosure can improve the quality of the processing result of the media information to be processed and reduce the difficulty of obtaining the processing result.

[0053] According to an embodiment of the present disclosure, the media information to be processed is media information related to a natural language processing task.

[0054] According to an embodiment of the present disclosure, the media information to be processed includes one of the following: text information to be processed and audio information to be processed.

[0055] According to an embodiment of the present disclosure, the natural language processing task can include one of the following: a natural language understanding task and a natural language generation task. The text information to be processed can be text information related to the natural language understanding task. The text information to be processed can be text information related to the natural language generation task. The audio information to be processed can be text information related to the natural language understanding task. The audio information to be processed can be audio information related to the natural language generation task.

[0056] According to an embodiment of the present disclosure, in a case where the media information to be processed is text information to be processed related to the natural language understanding task, the media information processing method provided by the embodiment of the present disclosure can improve the accuracy of the processing result of the natural language processing task for the text. In a case where the media information to be processed is audio information to be processed related to the natural language task, the media information processing method provided by the embodiment of the present disclosure can improve the accuracy of the processing result of the natural language processing task for the audio.

[0057] According to an embodiment of the present disclosure, the natural language processing task includes one of the following: an information extraction task, a sentiment analysis task, a knowledge reasoning task, an intent recognition task, a text classification task, a text summarization task, a machine translation task, and a coreference resolution task.

[0058] According to an embodiment of the disclosure, the information extraction task can refer to extracting predetermined event information or fact information from the to-be-processed media information, so as to realize automatic classification, extraction and reconstruction of content. The information extraction task can include one of the following: a relation extraction task, a named entity recognition task and an event extraction task.

[0059] According to an embodiment of the disclosure, the relation extraction task can refer to a task of extracting entities and relations between entities from the to-be-processed media information. The processing result of the to-be-processed media information related to the relation extraction task is the relation between entities in the to-be-processed media information. For example, the to-be-processed media information is “A is the son of B group manager C”. The processing result for the to-be-processed media information is that “A and C have a “parent-child” relationship”. The processing result of the to-be-processed media information related to the relation extraction task is the relation between entities in the to-be-processed media information.

[0060] According to an embodiment of the disclosure, the named entity recognition task can refer to a task of identifying named entities from the to-be-processed media information and classifying the named entities into predetermined categories. The predetermined categories can include at least one of the following: a person name, an organization name, a place name, a time, a date, a currency and a percentage. The processing result of the to-be-processed media information related to the named entity task is the named entity in the to-be-processed media information and the category of the named entity.

[0061] According to an embodiment of the disclosure, the event extraction task can refer to a task of extracting event-related information from the to-be-processed media information. The event-related information can include at least one of the following: a person, an event occurrence time and an event occurrence location. The processing result of the to-be-processed media information related to the event extraction task is the person, the event occurrence location and the event occurrence time in the to-be-processed media information.

[0062] According to an embodiment of the disclosure, the sentiment analysis task can refer to a task of determining the sentiment category of an entity from the to-be-processed media information. The sentiment category can include positive, negative or neutral. The processing result of the to-be-processed media information related to the sentiment analysis task is the sentiment classification of the entity in the to-be-processed media information.

[0063] According to an embodiment of the disclosure, the knowledge reasoning task can refer to a task of reasoning unknown knowledge according to existing knowledge (i.e., to-be-processed media information). The processing result of the to-be-processed media information related to the knowledge reasoning task is the knowledge reasoning result of the to-be-processed media information.

[0064] According to an embodiment of the disclosure, the intent recognition task can refer to a task of analyzing the to-be-processed media information to obtain an intent category. The intent category can include at least one of the following: a navigation type category, a transaction type category and an information type category. The processing result of the to-be-processed media information related to the intent recognition task is the intent category of the to-be-processed media information.

[0065] According to an embodiment of the present disclosure, the text classification task can refer to a task of classifying the to-be-processed media information based on a predetermined classification system or standard to obtain a classification result. The processing result of the to-be-processed media information related to the text classification task is the classification result of the to-be-processed media information.

[0066] According to an embodiment of the present disclosure, the text summarization task can refer to a task of converting the to-be-processed media information into a short summary including key information. The processing result of the to-be-processed media information related to the text summarization task is the summary information of the to-be-processed media information.

[0067] According to an embodiment of the present disclosure, the machine translation task can refer to a task of translating the to-be-processed media information from media information in a source language into media information in a target language. The processing result of the to-be-processed media information related to the machine translation task is the translation result of the to-be-processed media information.

[0068] According to an embodiment of the present disclosure, the anaphora resolution task can include the coreference resolution task. The anaphora resolution task can refer to a task of determining an antecedent for an anaphoric component appearing in the to-be-processed media information. The coreference resolution task can refer to a task of determining an entity in the real world to which a representation in the to-be-processed media information refers. The representation can include one of the following: a common noun phrase, a proper noun, and a pronoun. The processing result of the to-be-processed media information related to the anaphora resolution task is the anaphora resolution result of the to-be-processed media information.

[0069] According to an embodiment of the present disclosure, the operation S210 can include the following operations.

[0070] According to the phrase structure graph corresponding to the to-be-processed media information, a first intermediate object structure graph is obtained. A first target object node in the first intermediate object structure graph is converted into an edge to obtain a second intermediate object structure graph. According to the second intermediate object structure graph, a target object structure graph is obtained.

[0071] According to an embodiment of the present disclosure, the nodes in the first intermediate object structure graph are all object nodes. The first intermediate object structure graph can include a plurality of object nodes. The first target object node can refer to an object node in the first intermediate object structure graph representing relationship information. The second intermediate object structure graph can include part of the object nodes in the first intermediate object structure graph.

[0072] According to an embodiment of the present disclosure, the phrase structure graph corresponding to the media information to be processed can be processed to obtain a first intermediate object structure graph. For example, at least one phrase node in the phrase structure graph corresponding to the media information to be processed can be split into an object node to obtain a plurality of object nodes. According to the plurality of object nodes and the object node relationship between each of the plurality of object nodes, the first intermediate object structure graph is obtained. The object node relationship between each of the plurality of object nodes can be determined according to the phrase structure graph.

[0073] According to an embodiment of the present disclosure, the first intermediate object structure graph can be converted into an object node and an edge to obtain a second intermediate object structure graph. The second intermediate object structure graph can be determined as the target object structure graph. Alternatively, the second intermediate object structure graph can be converted into an object node and a node attribute to obtain the target object structure graph.

[0074] The media information processing method according to the embodiments of the present disclosure will be further described below with reference to the specific embodiments in combination with Figures 3-5

[0075] According to an embodiment of the present disclosure, the phrase structure graph corresponding to the media information to be processed can be processed to obtain a first intermediate object structure graph. For example, at least one phrase node in the phrase structure graph corresponding to the media information to be processed can be split into an object node to obtain a plurality of object nodes. According to the plurality of object nodes and the object node relationship between each of the plurality of object nodes, the first intermediate object structure graph is obtained. The object node relationship between each of the plurality of object nodes can be determined according to the phrase structure graph.

[0076] According to an embodiment of the present disclosure, the phrase structure graph corresponding to the media information to be processed can be processed to obtain a first intermediate object structure graph. For example, at least one phrase node in the phrase structure graph corresponding to the media information to be processed can be split into an object node to obtain a plurality of object nodes. According to the plurality of object nodes and the object node relationship between each of the plurality of object nodes, the first intermediate object structure graph is obtained. The object node relationship between each of the plurality of object nodes can be determined according to the phrase structure graph.

[0077] According to an embodiment of the present disclosure, the object node relationship between two object nodes can include at least one of the following: no relationship between the two object nodes, one of the two object nodes can be the next object node of the other object node, and one of the two object nodes can be the argument of the other object node. For example, one of the two object nodes being the argument of the other object node can include one of the following: one of the two object nodes can be an object node for indicating an argument and the other object node can be an object node for indicating a predicate, and the other object node of the two object nodes can be an object node for indicating a limiting information and one object node can be an object node for indicating a predicate. The object node for indicating an argument can include one of the following: an object node for indicating a subject and an object node for indicating an object.

[0078] ​For example, if one of the two object nodes can be the next object node of the other object node, the object node relationship between the two object nodes can be indicated by next_word. If one of the two object nodes can be the object node for indicating argument and the other object node can be the object node for indicating predicate, the object node relationship between the two object nodes can be indicated by pred.arg. If one of the two object nodes can be the object node for indicating qualification information and the other object node can be the object node for indicating predicate, the object node relationship between the two object nodes can be indicated by func.arg.

[0079] According to an embodiment of the present disclosure, each phrase node in the phrase structure graph corresponding to the media information to be processed can be split into a plurality of object nodes corresponding to the phrase nodes, thereby obtaining the plurality of object nodes. If the phrase structure graph includes object nodes, the plurality of object nodes include the object nodes in the original phrase structure graph and the object nodes obtained by splitting the phrase nodes in the phrase structure graph into object nodes. If the phrase structure graph does not include object nodes, the plurality of object nodes are the object nodes obtained by splitting the phrase nodes in the phrase structure graph.

[0080] According to an embodiment of the present disclosure, the phrase structure graph can be analyzed to determine the object node relationships between the plurality of object nodes. Then, according to the plurality of object nodes and the object node relationships between the plurality of object nodes, the first intermediate object structure graph is obtained.

[0081] According to an embodiment of the present disclosure, the phrase structure graph can further include at least one original node and at least one edge for connecting two nodes. The two nodes include one of the following: two additional nodes, one additional node and one original node, and two original nodes. The original node is determined according to the media information to be processed.

[0082] According to an embodiment of the present disclosure, determining the object node relationships between the plurality of object nodes according to the phrase structure graph can include the following operations.

[0083] According to at least one object relationship and at least one node relationship, the object node relationships between the plurality of object nodes are determined.

[0084] According to an embodiment of the present disclosure, the at least one object relationship is determined according to at least one of the following: the relationship between objects in the at least one original node and the relationship between objects in the at least one additional node. The at least one node relationship is determined according to the edge label of the at least one edge in the phrase structure graph.

[0085] According to an embodiment of the present disclosure, the phrase structure graph can include at least two nodes and at least one edge connecting the two nodes. The at least two nodes can include at least one additional node and at least one original node. The original node can be determined according to the media information to be processed. For each of the at least one edge connecting the two nodes, the two nodes connected by the edge can be one of two additional nodes, two original nodes, and one additional node and one original node.

[0086] According to an embodiment of the present disclosure, if the original node is a phrase node, the object relationship between at least two objects in the original node can be determined. If the additional node is a phrase node, the object relationship between at least two objects in the additional node can be determined. The object relationship between two objects can include one of no relationship between the two objects and one of the two objects being a next object of the other object.

[0087] According to an embodiment of the present disclosure, the node relationship between two nodes connected by each of the at least one edge in the phrase structure graph can be determined according to the edge label of the edge. The edge label of the edge can be used to indicate the node relationship between the two connected nodes. The node relationship between the two nodes can include at least one of no relationship between the two nodes, one of the two nodes being a next node of the other node, and one of the two nodes being a parameter of the other node. The edge label can include one of next_word, pred.arg, and func.arg. The next_word can be used to indicate the node relationship between one of the two nodes being a next node of the other node. The pred.arg can be used to indicate the node relationship between one of the two nodes being a node for indicating a parameter and the other node being a node for indicating a predicate. The func.arg can be used to indicate the node relationship between one of the two nodes being a node for indicating a limited information and the other node being a node for indicating a predicate.

[0088] According to an embodiment of the present disclosure, the object node relationship between the plurality of object nodes can be determined according to the at least one object relationship and the at least one node relationship.

[0089] According to an embodiment of the present disclosure, obtaining the first intermediate object structure graph according to the plurality of object nodes and the object node relationship between the plurality of object nodes can include the following operations.

[0090] According to the object node relationship between the plurality of object nodes, at least one edge connecting the plurality of object nodes is determined. According to the plurality of object nodes and the at least one edge connecting the plurality of object nodes, the first intermediate object structure graph is obtained.

[0091] According to embodiments of this disclosure, two object nodes with an object node relationship among multiple object nodes can be connected by an edge to obtain at least one edge. Then, based on the multiple object nodes and at least one edge, a first intermediate object structure diagram is obtained.

[0092] According to embodiments of this disclosure, a starting object node can be determined from a plurality of object nodes. Starting from the starting object node, based on the object node relationships between the plurality of object nodes, two object nodes with object node relationships are connected by an edge to obtain a first intermediate object node. The starting object node can be determined based on the media information to be processed. For example, the starting object node can be the object node corresponding to the first object in the media information to be processed. Alternatively, the starting object node can be the object node corresponding to the central object in the media information to be processed. The central object can refer to the object corresponding to the core information in the media information to be processed.

[0093] The following is for reference. Figure 3 The process of obtaining a first intermediate object structure diagram based on a phrase structure diagram corresponding to the media information to be processed, according to the embodiments of this disclosure, will be further explained in conjunction with specific embodiments.

[0094] Figure 3 The illustration shows an example diagram of obtaining a first intermediate object structure diagram based on a phrase structure diagram corresponding to the media information to be processed, according to an embodiment of the present disclosure.

[0095] like Figure 3 As shown in diagram 300, the media information to be processed can be "I will be present, I am responsible for stage management?" 301. The phrase structure diagram 302 corresponding to the media information to be processed includes node 302_1 "I", phrase node 302_2 "will be present", node 302_3 "I", phrase node 302_4 "responsible for", phrase node 302_5 "stage management", node 302_6 "parataxis", and node 302_7 "whether". Nodes 302_6 and 302_7 are both supplementary nodes. Node 302_6 can refer to a relation node. Node 302_7 can refer to a function node.

[0096] The edge label of the directed edge from phrase node 302_2 to node 302_1 is "pred.arg.1". The edge label of the directed edge from phrase node 302_4 to node 302_3 is "pred.arg.1". The edge label of the directed edge from phrase node 302_4 to phrase node 302_5 is "pred.arg.2". The edge label of the directed edge from node 302_6 to phrase node 302_2 is "pred.arg.1". The edge label of the directed edge from node 302_6 to node 302_7 is "func.arg".

[0097] The phrase nodes in the phrase structure graph 302 corresponding to the media information to be processed are split into object nodes, obtaining a plurality of object nodes. For example, for the phrase node 302_2 "will be present", the phrase node 302_2 is split into the object node 303_2 "will", the object node 303_3 "be", and the object node 303_4 "present". For the phrase node 302_4 "is responsible for", the phrase node 302_4 "is responsible for" is split into the object node 303_6 "is" and the object node 303_7 "responsible for". For the phrase node 302_5 "stage supervisor", the phrase node 302_5 "stage supervisor" is split into the object node 303_8 "stage", the object node 303_9 "supervisor". Thus, the object node 303_1 "I", the object node 303_2 "will", the object node 303_3 "be", the object node 303_4 "present", the object node 303_5 "I", the object node 303_6 "is", the object node 303_7 "responsible for", the object node 303_8 "stage", the object node 303_9 "supervisor", the object node 303_10 "monitor", the object node 303_11 "supervisor", the object node 303_12 "parataxis", and the object node 303_13 "whether" are obtained.

[0098] According to the phrase structure graph, object node relationships between the plurality of object nodes are determined. For example, for the phrase node 302_2 "will be present", the object node relationship between the object node 303_2 "will" and the object node 303_3 "be" is "next_word". The object node relationship between the object node 303_3 "be" and the object node 303_4 "present" is "next_word". For the node 302_6 "parataxis", the phrase node 302_2 "will be present", and the node 302_1 "I", according to the edge label of the directed edge between the phrase node 302_2 "will be present" and the node 302_1 "I" being "pred.arg.1" and the edge label of the directed edge between the phrase node 302_2 "will be present" and the node 302_6 "parataxis" being "pred.arg.1", the directed edge from the node 302_6 "parataxis" to the phrase node 302_2 "will be present" is an in-edge of the phrase node 302_2, and the directed edge from the phrase node 302_2 "will be present" to the node 302_1 "I" is an out-edge of the phrase node 302_2, obtaining the object node relationship between the object node 303_2 and the object node 303_1 being "pred.arg.1" and the object node relationship between the object node 303_12 and the object node 303_2 being "pred.arg.1".

[0099] The object node relationship between the object node 303_2 and the object node 303_1 is "pred.arg.1" based on the above-described manner. The object node relationship between the object node 303_2 and the object node 303_3 is "next_word". The object node relationship between the object node 303_3 and the object node 303_4 is "next_word". The object node relationship between the object node 303_6 and the object node 303_5 is "pred.arg.1". The object node relationship between the object node 303_6 and the object node 303_7 is "next_word". The object node relationship between the object node 303_6 and the object node 303_8 is "pred.arg.2". The object node relationship between the object node 303_8 and the object node 303_9 is "next_word". The object node relationship between the object node 303_9 and the object node 303_10 is "next_word". The object node relationship between the object node 303_10 and the object node 303_11 is "next_word". The object node relationship between the object node 303_12 and the object node 303_2 is "pred.arg.1". The object node relationship between the object node 303_12 and the object node 303_13 is "pred.arg.2". The object node relationship between the object node 303_13 and the object node 303_6 is "func.arg".

[0100] According to the object node relationship between the plurality of object nodes, at least one edge connecting the plurality of object nodes is determined. The edge can be a directed edge. An edge label of the edge can be determined according to the object node relationship between the object nodes. For example, an edge label of a directed edge from the object node 303_2 to the object node 303_1 is “pred.arg.1”. An edge label of a directed edge from the object node 303_2 to the object node 303_3 is “next_word”. An edge label of a directed edge from the object node 303_3 to the object node 303_4 is “next_word”. An edge label of a directed edge from the object node 303_6 to the object node 303_5 is “pred.arg.1”. An edge label of a directed edge from the object node 303_6 to the object node 303_7 is “next_word”. An edge label of a directed edge from the object node 303_6 to the object node 303_8 is “pred.arg.2”. An edge label of a directed edge from the object node 303_8 to the object node 303_9 is “next_word”. An edge label of a directed edge from the object node 303_9 to the object node 303_10 is “next_word”. An edge label of a directed edge from the object node 303_10 to the object node 303_11 is “next_word”. An edge label of a directed edge from the object node 303_12 to the object node 303_2 is “pred.arg.1”. An edge label of a directed edge from the object node 303_12 to the object node 303_13 is “pred.arg.2”. An edge label of a directed edge from the object node 303_13 to the object node 303_6 is “func.arg”.

[0101] According to the plurality of object nodes and the at least one edge, a first intermediate object structure graph 303 is obtained.

[0102] According to an embodiment of the present disclosure, the at least one additional node can include at least one relationship node. The first intermediate object structure graph can include at least two intermediate object structure subgraphs connected via the at least one relationship node. The first target object node can include the relationship node.

[0103] According to an embodiment of the present disclosure, converting the first target object node in the first intermediate object structure graph into an edge to obtain a second intermediate object structure graph can include the following operations.

[0104] According to a structure graph relationship between the at least two intermediate object structure subgraphs connected via the at least one relationship node, the at least one relationship node is converted into at least one edge to obtain the second intermediate object structure graph.

[0105] According to an embodiment of the present disclosure, the additional nodes can include relationship nodes. The relationship nodes can include one of the following: a parallel relationship node, a modification relationship node, a continuation relationship node, a progression relationship node, a selection relationship node, a transition relationship node, a hypothesis relationship node, a cause-effect relationship node, a condition relationship node, an explanation relationship node, and a purpose relationship node. The first target object node can include a relationship node. The relationship between the structure diagrams can include at least one of the following: an equal structure diagram relationship and an unequal structure diagram relationship. For example, the equal structure diagram relationship can include a parallel structure diagram relationship. The unequal structure diagram relationship can include a modification structure diagram relationship.

[0106] According to an embodiment of the present disclosure, for a relationship node of the at least one relationship node, for two intermediate object structure sub-diagrams of the at least two intermediate object structure sub-diagrams connected via the relationship node, the relationship node is converted into an edge according to a structure diagram relationship between the two intermediate object structure sub-diagrams connected via the relationship node, thereby obtaining a second intermediate object structure diagram.

[0107] According to an embodiment of the present disclosure, converting the at least one relationship node into at least one edge according to the structure diagram relationship between the at least two intermediate object structure sub-diagrams connected via the at least one relationship node to obtain the second intermediate object structure diagram can include the following operations.

[0108] According to an embodiment of the present disclosure, at least one first intermediate object structure sub-diagram and at least one second intermediate object structure sub-diagram are determined from the at least two intermediate object structure sub-diagrams according to the structure diagram relationship between the at least two intermediate object structure sub-diagrams connected via the at least one relationship node. The at least one relationship node is converted into at least one directed edge to obtain the second intermediate object structure diagram.

[0109] According to an embodiment of the present disclosure, the directed edge is directed from a second target object node of the first intermediate object structure sub-diagram to a third target object node of the second intermediate object structure sub-diagram corresponding to the first intermediate object structure sub-diagram.

[0110] According to an embodiment of the present disclosure, for a relationship node of the at least one relationship node, for two intermediate object structure sub-diagrams of the at least two intermediate object structure sub-diagrams connected via the relationship node, the relationship node can be converted into a directed edge for connecting between a second target object node of the first intermediate object structure sub-diagram and a third target object node of the second intermediate object structure sub-diagram according to a structure diagram relationship between the two intermediate object structure sub-diagrams connected via the relationship node. The directed edge is directed from the second target object node to the third target object node. An edge label of the directed edge can be determined according to semantic information indicated by the relationship node.

[0111] Reference will be made to the followingFigure 4 The process of converting a first target object node in a first intermediate object structure graph into an edge to obtain a second intermediate object structure graph, in conjunction with specific embodiments, will be further explained.

[0112] Figure 4 The illustration shows an example diagram of converting a first target object node in a first intermediate object structure graph into an edge to obtain a second intermediate object structure graph according to an embodiment of the present disclosure.

[0113] like Figure 4 As shown, in 400, object node 401_12 "parataxis" is a relation node. The first intermediate object structure graph 401 includes two intermediate object structure subgraphs connected via object node 401_12 "parataxis". Based on the structure graph relationship between the two intermediate object structure subgraphs, it is determined that the first intermediate object structure graph is an intermediate object structure subgraph including object node 303_1 "I", object node 303_2 "Will", object node 303_3 "To", and object node 303_4 "Field", and the second intermediate object structure subgraph is an intermediate object structure subgraph including object node 303_5 "I", object node 303_6 "Responsible", object node 303_7 "Responsible", object node 303_8 "Dance", object node 303_9 "Stage", object node 303_10 "Supervisor", object node 303_11 "Supervisor", and object node 303_13 "whether".

[0114] The relation node (i.e., object node 401_12 "parataxis") is transformed into a directed edge from the second target object node (i.e., object node 401_2 "will") in the first intermediate object structure subgraph to the third target object node (i.e., object node 401_13 "whether") in the second intermediate object structure subgraph, resulting in the second intermediate object structure graph 402. The edge label of the directed edge is determined based on the semantic information indicated by the relation node; therefore, the edge label of the directed edge is "parataxis".

[0115] According to embodiments of this disclosure, obtaining a target object structure diagram based on a second intermediate object structure diagram may include the following operations.

[0116] Identify the limiting object nodes and the fourth target object node in the second intermediate object structure diagram. Based on the limiting object nodes and the fourth target object node, perform a structural transformation on the second intermediate object structure diagram to obtain the target object structure diagram.

[0117] According to an embodiment of the present disclosure, the defining object node corresponds to the fourth target object node. The defining object node can be used to define the intermediate object structure subgraph included in the second intermediate object structure graph corresponding to the defining object node. The intermediate object structure subgraph included in the second intermediate object structure graph can include the fourth target object node.

[0118] According to an embodiment of the present disclosure, the defining object node can be used to define the intermediate object structure subgraph in the second intermediate object structure graph corresponding to the defining node. The defining object node can refer to the upper object node of the defined intermediate object structure subgraph.

[0119] According to an embodiment of the present disclosure, the second intermediate object structure graph can be converted between nodes and node attributes according to the defining object node and the fourth target object node to obtain the target object structure graph.

[0120] According to an embodiment of the present disclosure, the second intermediate object structure graph can be converted between nodes and node attributes according to the defining object node and the fourth target object node to obtain the target object structure graph.

[0121] According to the semantic information indicated by the defining object node and the semantic information indicated by the intermediate object structure subgraph included in the second intermediate object structure graph defined by the defining object node, the target attribute information is determined. The defining object node is converted into the node attribute of the fourth target object node corresponding to the defining object node to obtain the target object structure graph. The attribute information indicated by the node attribute is the target attribute information.

[0122] According to an embodiment of the present disclosure, the target attribute information can be determined according to the semantic information indicated by the defining object node and the semantic information indicated by the intermediate object structure subgraph corresponding to the defining object node in the second intermediate object structure graph. The defining object node is converted into the node attribute of the fourth target object node, so that the attribute information indicated by the node attribute is the target attribute information.

[0123] The process of obtaining the target object structure graph according to the second intermediate object structure graph according to an embodiment of the present disclosure will be further described below with reference to specific embodiments. Figure 5

[0124] Figure 5 An example schematic diagram of obtaining the target object structure graph according to the second intermediate object structure graph according to an embodiment of the present disclosure is schematically shown.

[0125] As Figure 5 ​As shown, in 500, it is determined that the qualification object node in the second intermediate object structure graph 501 is the object node 501 12 "whether", and the fourth target object node is the object node 501 6 "negative". The intermediate object structure subgraph in the second intermediate object structure graph that the qualification object node is used to limit includes the object node 501 5 "I", the object node 501 6 "negative", the object node 501 7 "responsibility", the object node 501 8 "dance", the object node 501 9 "stage", the object node 501 10 "monitor", and the object node 501 11 "supervise".

[0126] According to the semantic information indicated by the qualification object node and the semantic information indicated by the intermediate object structure subgraph included in the second intermediate object structure graph that the qualification object node is used to limit, it is determined that the target attribute information is "(whether = true)".

[0127] The qualification object node (that is, the object node 501 12 "whether") is converted into the node attribute of the fourth target object node (that is, the object node 501 6 "negative") corresponding to the qualification object node, and the target object structure graph 502 is obtained. The attribute information indicated by the node attribute is the target attribute information.

[0128] According to an embodiment of the present disclosure, obtaining the target object structure graph according to the second intermediate object structure graph can include the following operations.

[0129] The second intermediate object structure graph is determined as the target object structure graph.

[0130] According to an embodiment of the present disclosure, the second intermediate object structure graph can also not be processed, and the second intermediate object structure graph is directly taken as the target object structure graph.

[0131] The media information processing method provided by the embodiment of the present disclosure can realize the conversion of the phrase structure graph to the target object structure graph, and can realize the conversion of the target object structure graph to the phrase structure graph by using the inverse operation of the phrase structure graph to the target object structure graph. For example, according to the target object structure graph, the second intermediate object structure graph is obtained. The target edge in the second intermediate object structure graph is converted into the first target object node, and the first intermediate object structure graph is obtained. According to the first intermediate object structure graph, the phrase structure graph corresponding to the media information to be processed is obtained.

[0132] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution comply with the relevant legal regulations and do not violate public order and good customs.

[0133] The above is only an example embodiment, but is not limited thereto, and can also include other media information processing methods known in the art, as long as the quality of the processing result of the to-be-processed media information can be improved and the difficulty of obtaining the processing result can be reduced.

[0134] Figure 6 A block diagram of a media information processing apparatus according to an embodiment of the present disclosure is schematically shown.

[0135] As Figure 6 shown, the media information processing apparatus 600 can include a first obtaining module 610, a second obtaining module 620, and a third obtaining module 630.

[0136] The first obtaining module 610 is configured to obtain a target object structure graph according to a phrase structure graph corresponding to to-be-processed media information. The phrase structure graph includes at least one additional node corresponding to additional information, and the additional information includes information related to the to-be-processed media information.

[0137] The second obtaining module 620 is configured to obtain a target object representation vector set according to the target object structure graph.

[0138] The third obtaining module 630 is configured to obtain a processing result for the to-be-processed media information according to the target object representation vector set.

[0139] According to an embodiment of the present disclosure, the first obtaining module 610 can include a first obtaining sub-module, a second obtaining sub-module, and a third obtaining sub-module.

[0140] The first obtaining sub-module is configured to obtain a first intermediate object structure graph according to the phrase structure graph corresponding to the to-be-processed media information.

[0141] The second obtaining sub-module is configured to convert a first target object node in the first intermediate object structure graph into an edge to obtain a second intermediate object structure graph.

[0142] The third obtaining sub-module is configured to obtain the target object structure graph according to the second intermediate object structure graph.

[0143] According to an embodiment of the present disclosure, the at least one additional node includes at least one relationship node. The first intermediate object structure graph includes at least two intermediate object structure subgraphs connected via the at least one relationship node. The first target object node includes the relationship node.

[0144] According to an embodiment of the present disclosure, the second obtaining sub-module can include a first obtaining unit.

[0145] The first obtaining unit is configured to convert the at least one relationship node into at least one edge according to a structure graph relationship between the at least two intermediate object structure subgraphs connected via the at least one relationship node to obtain the second intermediate object structure graph.

[0146] According to an embodiment of the present disclosure, the first obtaining unit can include a first determining subunit and a first obtaining subunit.

[0147] The first determining subunit is configured to determine at least one first intermediate object structure subgraph and at least one second intermediate object structure subgraph from the at least two intermediate object structure subgraphs according to the structure graph relationship between the at least two intermediate object structure subgraphs connected via the at least one relationship node.

[0148] The first obtaining subunit is configured to convert the at least one relationship node into at least one directed edge to obtain a second intermediate object structure graph. The directed edge is directed from a second target object node of the first intermediate object structure subgraph to a third target object node of the second intermediate object structure subgraph corresponding to the first intermediate object structure subgraph.

[0149] According to an embodiment of the present disclosure, the third obtaining sub-module can include a first determining unit and a second obtaining unit.

[0150] The first determining unit is configured to determine a limit object node and a fourth target object node in the second intermediate object structure graph. The limit object node and the fourth target object node correspond to each other. The limit object node is used to limit the intermediate object structure subgraph included in the second intermediate object structure graph corresponding to the limit object node. The intermediate object structure subgraph included in the second intermediate object structure graph includes the fourth target object node.

[0151] The second obtaining unit is configured to perform structure transformation on the second intermediate object structure graph according to the limit object node and the fourth target object node to obtain the target object structure graph.

[0152] According to an embodiment of the present disclosure, the second obtaining unit can include a second determining subunit and a second obtaining subunit.

[0153] The second determining subunit is configured to determine target attribute information according to semantic information indicated by the limit object node and semantic information indicated by the intermediate object structure subgraph included in the second intermediate object structure graph limited by the limit object node.

[0154] The second obtaining subunit is configured to convert the limit object node into a node attribute of the fourth target object node corresponding to the limit object node to obtain the target object structure graph. The attribute information indicated by the node attribute is the target attribute information.

[0155] According to an embodiment of the present disclosure, the third obtaining sub-module can include a second determining unit.

[0156] The second determining unit is configured to determine the second intermediate object structure graph as the target object structure graph.

[0157] According to an embodiment of the present disclosure, the first obtaining sub-module can include a third obtaining unit, a third determining unit and a fourth obtaining unit.

[0158] The third obtaining unit is configured to split a phrase node in the phrase structure graph corresponding to the media information to be processed into an object node, to obtain a plurality of object nodes.

[0159] The third determining unit is configured to determine an object node relationship between the plurality of object nodes according to the phrase structure graph.

[0160] The fourth obtaining unit is configured to obtain a first intermediate object structure graph according to the plurality of object nodes and the object node relationship between the plurality of object nodes.

[0161] According to an embodiment of the present disclosure, the phrase structure graph further includes at least one original node and at least one edge for connecting two nodes. The two phrase nodes include one of the following: two additional nodes, one additional node and one original node, and two original nodes, wherein the original node is determined according to the media information to be processed.

[0162] According to an embodiment of the present disclosure, the third determining unit can include a third determining sub-unit.

[0163] The third determining sub-unit is configured to determine the object node relationship between the plurality of object nodes according to at least one object relationship and at least one node relationship. The at least one object relationship is determined according to at least one of the following: a relationship between objects in the at least one original node and a relationship between objects in the at least one additional node. The at least one node relationship is determined according to an edge label of the at least one edge in the phrase structure graph.

[0164] According to an embodiment of the present disclosure, the fourth obtaining unit can include a fourth determining sub-unit and a third obtaining sub-unit.

[0165] The fourth determining sub-unit is configured to determine at least one edge for connecting the plurality of object nodes according to the object node relationship between the plurality of object nodes.

[0166] The third obtaining sub-unit is configured to obtain the first intermediate object structure graph according to the plurality of object nodes and the at least one edge for connecting the plurality of object nodes.

[0167] According to an embodiment of the present disclosure, the media information to be processed is media information related to a natural language processing task. The media information to be processed includes one of the following: text information to be processed and audio information to be processed.

[0168] According to an embodiment of the present disclosure, the natural language processing task includes one of the following: an information extraction task, a sentiment analysis task, a knowledge reasoning task, an intent recognition task, a text classification task, a text summarization task, a machine translation task and a coreference resolution task.

[0169] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0170] According to an embodiment of the present disclosure, an electronic device comprises at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0171] According to an embodiment of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the method as described above.

[0172] According to an embodiment of the present disclosure, a computer program product comprises a computer program, and the computer program, when executed by a processor, implements the method as described above.

[0173] Figure 7 A block diagram schematically shows an electronic device suitable for implementing a media information processing method according to an embodiment of the present disclosure. The electronic device is intended to represent a variety of forms of digital object computers, such as laptop computers, desktop computers, workstations, personal digital object assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent a variety of forms of mobile devices, such as personal digital object processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections, and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0174] As Figure 7 shown, the electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0175] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0176] The computing unit 701 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the media information processing method. For example, in some embodiments, the media information processing method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the media information processing method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the media information processing method by any other appropriate means, such as by means of firmware.

[0177] Various implementations of the systems and techniques described above herein can be realized in electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0178] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0179] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0180] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0181] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0182] The computer system can include clients and servers. This description uses the terms "client" and "server" to refer to the roles of these computational entities. A client and a server can be distant from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0183] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the spirit and scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in series, or executed in different orders, as long as the desired results of the technical solutions of the present disclosure are achieved, and the present disclosure is not limited herein.

[0184] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A media information processing method, comprising: Based on the phrase structure diagram corresponding to the media information to be processed, a target object structure diagram is obtained, wherein the phrase structure diagram includes at least one additional node corresponding to additional information, and the additional information includes information related to the media information to be processed; The media information to be processed includes multiple objects, and the objects include characters or words; the object structure diagram includes: a structure diagram in which all nodes are object nodes; the object node is a node corresponding to an object; The additional information includes at least one of the following: relationship information and limiting information; the relationship information includes information related to specifying the relationship between the objects included in the media information to be processed; the limiting information includes information for limiting predetermined information in the media information to be processed. Based on the target object structure diagram, the target object representation vector set is obtained; and Based on the target object representation vector set, the processing result for the media information to be processed is obtained; The step of obtaining the target object structure graph based on the phrase structure graph corresponding to the media information to be processed includes: obtaining a first intermediate object structure graph based on the phrase structure graph corresponding to the media information to be processed; converting the first target object node in the first intermediate object structure graph into an edge to obtain a second intermediate object structure graph; the first target object node includes object nodes representing relational information in the first intermediate object structure graph; and obtaining the target object structure graph based on the second intermediate object structure graph.

2. The method according to claim 1, wherein, The at least one additional node includes at least one relation node, the first intermediate object structure graph includes at least two intermediate object structure subgraphs connected via the at least one relation node, and the first target object node includes the relation node; The step of converting the first target object node in the first intermediate object structure graph into an edge to obtain the second intermediate object structure graph includes: Based on the structural graph relationship between at least two intermediate object structural subgraphs connected via the at least one relation node, the at least one relation node is converted into at least one edge to obtain the second intermediate object structural graph.

3. The method according to claim 2, wherein, The step of converting the at least one relation node into at least one edge to obtain the second intermediate object structure graph based on the structure graph relationship between at least two intermediate object structure subgraphs connected via the at least one relation node includes: Based on the structural graph relationships between at least two intermediate object structural subgraphs connected via the at least one relation node, at least one first intermediate object structural subgraph and at least one second intermediate object structural subgraph are determined from the at least two intermediate object structural subgraphs; and The at least one relation node is converted into at least one directed edge to obtain the second intermediate object structure graph, wherein the directed edge points from the second target object node of the first intermediate object structure subgraph to the third target object node of the second intermediate object structure subgraph corresponding to the first intermediate object structure subgraph.

4. The method according to any one of claims 1 to 3, wherein, The step of obtaining the target object structure diagram based on the second intermediate object structure diagram includes: The defined object node and the fourth target object node in the second intermediate object structure graph are determined, wherein the defined object node and the fourth target object node correspond to each other, the defined object node is used to define the intermediate object structure subgraph included in the second intermediate object structure graph corresponding to the defined object node, and the intermediate object structure subgraph included in the second intermediate object structure graph includes the fourth target object node; the defined object node includes the upper-level object node of the defined intermediate object structure subgraph; and Based on the defined object node and the fourth target object node, the second intermediate object structure diagram is structurally transformed to obtain the target object structure diagram.

5. The method according to claim 4, wherein, The step of performing a structural transformation on the second intermediate object structure diagram based on the defined object node and the fourth target object node to obtain the target object structure diagram includes: Based on the semantic information indicated by the defined object node and the semantic information indicated by the intermediate object structure subgraph included in the second intermediate object structure graph defined by the defined object node, the target attribute information is determined; and The defined object node is converted into a node attribute of the fourth target object node corresponding to the defined object node to obtain the target object structure diagram, wherein the attribute information indicated by the node attribute is the target attribute information.

6. The method according to any one of claims 1 to 3, wherein, The step of obtaining the target object structure diagram based on the second intermediate object structure diagram includes: The second intermediate object structure diagram is determined as the target object structure diagram.

7. The method according to any one of claims 1, 2, 3, and 5, wherein, The step of obtaining the first intermediate object structure diagram based on the phrase structure diagram corresponding to the media information to be processed includes: The phrase nodes in the phrase structure graph corresponding to the media information to be processed are split into object nodes to obtain multiple object nodes; Based on the phrase structure diagram, determine the object node relationships among the multiple object nodes; and The first intermediate object structure diagram is obtained based on the multiple object nodes and the object node relationships between the multiple object nodes.

8. The method according to claim 7, wherein, The phrase structure graph also includes at least one original node and at least one edge for connecting two nodes. The two phrase nodes include one of the following: two additional nodes, one additional node and one original node, and two original nodes, wherein the original nodes are determined based on the media information to be processed. The step of determining the object node relationships among multiple object nodes based on the phrase structure diagram includes: Based on at least one object relationship and at least one node relationship, an object node relationship is determined among a plurality of said object nodes, wherein said at least one object relationship is determined based on at least one of the following: the relationship between objects in said at least one original node and the relationship between objects in said at least one additional node, and said at least one node relationship is determined based on the edge label of said at least one edge in said phrase structure graph.

9. The method according to claim 7, wherein, The step of obtaining the first intermediate object structure diagram based on the plurality of object nodes and the object node relationships between the plurality of object nodes includes: Based on the object node relationships between the multiple object nodes, determine at least one edge for connecting the multiple object nodes; and The first intermediate object structure diagram is obtained based on the plurality of object nodes and the at least one edge used to connect the plurality of object nodes.

10. The method according to any one of claims 1, 2, 3, 5, 8 and 9, wherein, The media information to be processed is media information related to the natural language processing task, and the media information to be processed includes one of the following: text information to be processed and audio information to be processed.

11. The method according to claim 10, wherein, The natural language processing tasks include one of the following: information extraction, sentiment analysis, knowledge reasoning, intent recognition, text classification, text summarization, machine translation, and referential resolution.

12. A media information processing device, comprising: The first obtaining module is configured to obtain a target object structure diagram based on a phrase structure diagram corresponding to the media information to be processed, wherein the phrase structure diagram includes at least one additional node corresponding to additional information, the additional information including information related to the media information to be processed; the media information to be processed includes multiple objects, the objects including characters or words; the object structure diagram includes: a structure diagram in which all nodes are object nodes; the object nodes are nodes corresponding to objects; the additional information includes at least one of the following: relationship information and limiting information; the relationship information includes information related to specifying the relationship between the objects included in the media information to be processed; the limiting information includes information for limiting predetermined information in the media information to be processed; The second obtaining module is used to obtain a target object representation vector set based on the target object structure diagram; and The third obtaining module is used to obtain the processing result for the media information to be processed based on the target object representation vector set; The first obtaining module includes: The first obtaining submodule is used to obtain a first intermediate object structure diagram based on the phrase structure diagram corresponding to the media information to be processed; The second obtaining submodule is used to convert the first target object node in the first intermediate object structure graph into an edge to obtain a second intermediate object structure graph; the first target object node includes object nodes representing relational information in the first intermediate object structure graph; and The third obtaining submodule is used to obtain the target object structure diagram based on the second intermediate object structure diagram.

13. The apparatus according to claim 12, wherein, The at least one additional node includes at least one relation node, the first intermediate object structure graph includes at least two intermediate object structure subgraphs connected via the at least one relation node, and the first target object node includes the relation node; The second obtaining submodule includes: The first obtaining unit is configured to convert the at least one relation node into at least one edge based on the structural graph relationship between at least two intermediate object structural subgraphs connected via the at least one relation node, thereby obtaining the second intermediate object structural graph.

14. The apparatus according to claim 13, wherein, The first obtaining unit includes: A first determining subunit is configured to determine at least one first intermediate object structure subgraph and at least one second intermediate object structure subgraph from the at least two intermediate object structure subgraphs based on the structure graph relationship between the at least two intermediate object structure subgraphs connected via the at least one relation node; and The first obtaining subunit is used to convert the at least one relation node into at least one directed edge to obtain the second intermediate object structure graph, wherein the directed edge points from the second target object node of the first intermediate object structure subgraph to the third target object node of the second intermediate object structure subgraph corresponding to the first intermediate object structure subgraph.

15. The apparatus according to any one of claims 12 to 14, wherein, The third obtaining submodule includes: A first determining unit is configured to determine a defined object node and a fourth target object node in the second intermediate object structure diagram, wherein the defined object node and the fourth target object node correspond to each other, the defined object node is used to define an intermediate object structure subgraph included in the second intermediate object structure diagram corresponding to the defined object node, and the intermediate object structure subgraph included in the second intermediate object structure diagram includes the fourth target object node; the defined object node includes a parent object node of the defined intermediate object structure subgraph; and The second obtaining unit is used to perform structural transformation on the second intermediate object structure diagram based on the defined object node and the fourth target object node to obtain the target object structure diagram.

16. The apparatus according to claim 15, wherein, The second obtaining unit includes: The second determining subunit is configured to determine target attribute information based on the semantic information indicated by the defined object node and the semantic information indicated by the intermediate object structure subgraph included in the second intermediate object structure graph defined by the defined object node; and The second obtaining subunit is used to convert the limited object node into a node attribute of a fourth target object node corresponding to the limited object node, thereby obtaining the target object structure diagram, wherein the attribute information indicated by the node attribute is the target attribute information.

17. The apparatus according to any one of claims 12 to 14, wherein, The third obtaining submodule includes: The second determining unit is used to determine the second intermediate object structure diagram as the target object structure diagram.

18. The apparatus according to any one of claims 12, 13, 14, and 16, wherein, The first obtaining submodule includes: The third obtaining unit is used to split the phrase node in the phrase structure graph corresponding to the media information to be processed into object nodes to obtain multiple object nodes. The third determining unit is configured to determine the object node relationships among the multiple object nodes based on the phrase structure diagram; and The fourth obtaining unit is used to obtain the first intermediate object structure diagram based on the plurality of object nodes and the object node relationships between the plurality of object nodes.

19. The apparatus according to claim 18, wherein, The phrase structure graph also includes at least one original node and at least one edge for connecting two nodes. The two phrase nodes include one of the following: two additional nodes, one additional node and one original node, and two original nodes, wherein the original nodes are determined based on the media information to be processed. The third determining unit includes: The third determining subunit is used to determine the object node relationship between a plurality of said object nodes based on at least one object relationship and at least one node relationship, wherein said at least one object relationship is determined based on at least one of the following: the relationship between objects in said at least one original node and the relationship between objects in said at least one additional node, and said at least one node relationship is determined based on the edge label of at least one edge in said phrase structure graph.

20. The apparatus according to claim 18, wherein, The fourth obtaining unit includes: The fourth determining subunit is configured to determine at least one edge for connecting the multiple object nodes based on the object node relationships between the multiple object nodes; and The third obtaining subunit is used to obtain the first intermediate object structure diagram based on the plurality of object nodes and at least one edge used to connect the plurality of object nodes.

21. The apparatus according to any one of claims 12, 13, 14, 16, 19, and 20, wherein, The media information to be processed is media information related to the natural language processing task, and the media information to be processed includes one of the following: text information to be processed and audio information to be processed.

22. The apparatus according to claim 21, wherein, The natural language processing tasks include one of the following: information extraction, sentiment analysis, knowledge reasoning, intent recognition, text classification, text summarization, machine translation, and referential resolution.

23. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 11.

25. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images