Key reference resolution method, device, medium and product in speech-to-text

Abstract

Embodiments of the present disclosure disclose a key reference resolution method and device in speech-to-text, a medium and a product. The method comprises: constructing a user's interpersonal relationship knowledge graph and splitting according to the interpersonal relationship type; when the third person pronoun is contained in the text sequence converted by the user's speech and meets the preset condition, obtaining the context information to obtain the reference object of the third person pronoun, the relationship type between the reference object and the user, and the confidence of the relationship type; sorting and traversing the relationship sub-graph based on the relationship type and the confidence to obtain the candidate relationship sub-graph; when the reference object matches any of the sub-nodes in the candidate relationship sub-graph and the first annotation field of the sub-nodes successfully, obtaining the text sequence containing the correct third person pronoun based on the second annotation field of the sub-nodes. The method obtains the reference object of the third person pronoun through semantic analysis, and matches the reference object in multiple dimensions combined with the interpersonal relationship knowledge graph to obtain the correct third person pronoun.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, and in particular to a method, apparatus, medium, and product for key reference parsing in speech-to-text conversion. Background Technology

[0002] With the widespread adoption of mobile internet, voice-to-text functionality has emerged in social applications, providing users with a more convenient way to communicate. In scenarios where typing is inconvenient, such as while driving or when hands are busy, this function quickly converts speech into text, allowing users to transmit information without manual input, greatly improving communication efficiency and meeting users' needs for instant communication in various complex situations.

[0003] However, when using the speech-to-text function in practice, once the key referential analysis stage is reached, if the user uses third-person pronouns, the system often has difficulty accurately distinguishing between "he," "she," and "it" based on the contextual semantic environment, which can lead to semantic confusion and affect the accurate transmission of information. Summary of the Invention

[0004] In view of this, embodiments of this disclosure provide a method, apparatus, medium, and product for key referential parsing in speech-to-text conversion. This method can obtain the referent of third-person pronouns through semantic analysis and combine interpersonal relationship knowledge graphs to match the referents in multiple dimensions, thereby solving the semantic confusion problem caused by the difficulty in distinguishing third-person pronouns and improving the quality of speech-to-text conversion and user experience.

[0005] In a first aspect, embodiments of this disclosure provide a method for parsing key references in speech-to-text conversion, employing the following technical solution: Construct a knowledge graph of user interpersonal relationships, and divide the knowledge graph into multiple subgraphs according to the type of interpersonal relationship; wherein, the interpersonal relationship type and the subgraph both include child nodes, and the child nodes include a first annotation field and a second annotation field; The user's voice is converted into a text sequence. When the text sequence contains a third-person pronoun and the third-person pronoun meets a preset condition, context information is obtained. Based on the context information, the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type are obtained. Based on the relationship type and the confidence level, all relationship subgraphs are sorted to form a set of relationship subgraphs; Traverse the set of relation subgraphs in sorted order to obtain candidate relation subgraphs; The referenced object is matched with the child nodes and the first annotation field of the child nodes in the candidate relation subgraph respectively; When the referent successfully matches the child node or the referent successfully matches the first annotation field of the child node, a text sequence containing the correct third-person pronoun is obtained based on the second annotation field of the child node.

[0006] Optionally, constructing the user's interpersonal relationship knowledge graph includes: Obtain the user's contact notes information, and construct an initial knowledge graph based on the contact notes information; Obtain historical conversation data of users on a preset chat platform, and extract interpersonal relationship information from the historical conversation data; Based on the interpersonal relationship information, the initial knowledge graph is optimized to obtain a recommended knowledge graph; The recommended knowledge graph is displayed to the user, and in response to the user's modification command, the recommended knowledge graph is modified to generate an interpersonal relationship knowledge graph.

[0007] Optionally, obtaining the user's contact remarks information and constructing an initial knowledge graph based on the contact remarks information includes: Extract the name remarks from the contact remarks information and identify whether the name remarks contain preset characters; If the preset characters are included, then the primary title and secondary title are extracted from the name remarks information based on the preset characters; If the preset characters are not included, then the name remarks information will be determined as the primary title; Using the user's basic information as the central node, the main title is used as a child node of the central node; Construct the first annotation field for the child node; If there is a secondary name corresponding to the child node, then the secondary name is stored in the first label field of the child node; If there is no secondary name corresponding to the child node, the first label field of the child node is empty; Detect the gender information contained in the contact remarks information, and store the third-person pronoun corresponding to the gender information in the second annotation field of the child node; Create directed edges to represent the relationships between users and their contacts; The central node, the child nodes, the first annotation field, and the directed edges constitute the initial knowledge graph.

[0008] Optionally, the step of obtaining context information, and based on the context information, obtaining the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type, includes: According to the preset first search range, short-range context information is obtained based on sentences containing third-person pronouns, and a first prompt word is constructed based on the short-range context information; If the AI ​​model outputs the referent based on the first prompt word, then the relationship type and the confidence level are obtained based on the short-distance context information and the AI ​​model. If the AI ​​model does not output the referent based on the first prompt word, then according to the preset second search range, long-distance context information is obtained with sentences containing third-person pronouns as the reference point, and a second prompt word is constructed based on the long-distance context information; If the AI ​​model outputs the referent based on the second prompt word, then the relationship type and the confidence level are obtained based on the long-distance context information and the AI ​​model. If the AI ​​model does not output the referent based on the second prompt word, then according to the preset third search range, cross-dialogue context information is obtained based on the timestamp of the statement containing the third-person pronoun, and a third prompt word is constructed based on the cross-dialogue context information. Based on the third prompt word and the AI ​​model, the referent, the relationship type, and the confidence level are obtained.

[0009] Optionally, the step of obtaining short-range contextual information based on a preset first search range, using sentences containing third-person pronouns as reference points, includes: Using a statement containing a third-person pronoun as a reference point, take N statements forward and N statements backward as short-range context information.

[0010] Optionally, the step of obtaining long-distance contextual information according to a preset second search range, using sentences containing third-person pronouns as reference points, includes: Using sentences containing third-person pronouns as reference points, dialogue content consistent with the theme of the sentences is extracted both forward and backward, and the extracted dialogue content is used as long-distance contextual information.

[0011] Optionally, the step of obtaining cross-dialogue context information according to a preset third search range, using the timestamps of statements containing third-person pronouns as reference points, includes: Using the timestamp of the statement containing the third-person pronoun as the reference point, set the first time interval forward and the second time interval backward; Within the time range formed by the first time interval and the second time interval, cross-dialogue context information is filtered out.

[0012] Optionally, the key reference parsing method in speech-to-text further includes: According to the preset fourth search range, downstream information is obtained based on statements containing third-person pronouns; If the downstream information contains a preset prompt word, it is determined that the third-person pronoun does not meet the preset condition. Based on the preset prompt word, a text sequence containing the correct third-person pronoun is obtained. If the downstream information does not contain a preset prompt word, then the third-person pronoun is determined to meet the preset condition.

[0013] Secondly, this disclosure also provides a key reference parsing system for speech-to-text conversion, employing the following technical solution: The knowledge graph construction module is used to construct a user's interpersonal relationship knowledge graph. According to the type of interpersonal relationship, the interpersonal relationship knowledge graph is divided into multiple relationship subgraphs. Each interpersonal relationship type and each relationship subgraph includes a child node, and each child node includes a first annotation field and a second annotation field. The speech-to-text module is used to convert the user's speech into a text sequence. When the text sequence contains a third-person pronoun, it obtains context information and, based on the context information, obtains the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type. The graph set formation module is used to sort all relation subgraphs based on the relation type and the confidence level to form a relation subgraph set; The candidate relation subgraph acquisition module is used to traverse the relation subgraph set in sorted order to obtain candidate relation subgraphs. The referential object matching module is used to match the referential object with the child nodes and the first annotation field of the child nodes in the candidate relation subgraph, respectively; The correct sequence acquisition module is used to acquire a text sequence containing the correct third-person pronoun based on the second annotation field of the child node when the referent successfully matches the child node or the referent successfully matches the first annotation field of the child node.

[0014] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution: The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the above-described key reference parsing methods in speech-to-text conversion.

[0015] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute any of the above-described key reference parsing methods in speech-to-text conversion.

[0016] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0017] The key referential parsing method in speech-to-text provided in this disclosure constructs a user's interpersonal relationship knowledge graph, enabling the system to integrate and organize various interpersonal relationships in the user's life. This provides a foundation for accurate parsing of the referential meaning of third-person pronouns, allowing the system to find suitable referential objects within a relatively complete interpersonal relationship system. The interpersonal relationship knowledge graph is divided into multiple sub-graphs based on interpersonal relationship types, allowing the system to perform more targeted searches when dealing with referential issues. Different sub-graphs correspond to different interpersonal relationship types, improving search efficiency and accuracy. Sub-nodes contain first and second annotation fields, which can store more detailed information about interpersonal relationships, such as the secondary titles of contacts and corresponding third-person pronouns. This helps the system make judgments based on richer information when matching referential objects, thereby improving matching accuracy. Converting the user's speech into a text sequence allows the system to perform text processing on the speech content, a prerequisite for subsequent referential parsing. Only by converting speech into text can the third-person pronouns and their referential relationships be further analyzed. When a text sequence contains a third-person pronoun, the system can obtain contextual information and, based on this, determine the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type. This allows the system to extract key information from the text, providing a basis for subsequent sorting and matching. All relation subgraphs are sorted based on relationship type and confidence level, forming a set of relation subgraphs. This sorting process allows the system to traverse the set of relation subgraphs in sorted order, prioritizing relation subgraphs with closer relationships to the referent and higher confidence levels. This narrows the search scope, reducing unnecessary search and matching processes and improving search accuracy and efficiency. The referent is then matched against child nodes and the first annotation field of each child node in the candidate relation subgraph. Through this multi-dimensional matching method, the system can more accurately find child nodes that match the referent, thus determining the correct referential relationship. When the referent successfully matches a child node or the first annotation field of a child node, the text sequence containing the correct third-person pronoun is obtained based on the second annotation field of the child node. This step ultimately solves the semantic confusion problem caused by the difficulty in distinguishing third-person pronouns in the prior art, ensuring the accurate transmission of information.

[0018] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the key reference parsing method in speech-to-text provided in this embodiment of the disclosure; Figure 2 A flowchart illustrating the method for constructing an interpersonal knowledge graph according to an embodiment of this disclosure; Figure 3 A flowchart illustrating the method for obtaining the referent, relation type, and confidence level provided in the embodiments of this disclosure; Figure 4 A flowchart illustrating the correct appellation resolution method provided in this embodiment of the disclosure; Figure 5 A flowchart illustrating the method for constructing a database of uncommon terms for appellations provided in this embodiment of the disclosure; Figure 6 A flowchart illustrating the correct title acquisition method provided in this embodiment of the disclosure; Figure 7 A schematic diagram of the key reference parsing system in speech-to-text provided in this embodiment of the disclosure; Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0021] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0022] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0023] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0024] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0025] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0026] Reference Figure 1 This disclosure provides a method for resolving key references in speech-to-text conversion, including the following steps: S1: Construct a knowledge graph of the user's interpersonal relationships, and divide the knowledge graph into multiple subgraphs according to the type of interpersonal relationship; where the interpersonal relationship type and the subgraph both include child nodes, and the child nodes include a first annotation field and a second annotation field; S2: When the text sequence contains a third-person pronoun and the third-person pronoun meets the preset conditions, obtain the context information, and based on the context information, obtain the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type. S3: Sort all relation subgraphs based on relation type and confidence level to form a set of relation subgraphs; S4: Traverse the set of relation subgraphs in sorted order to obtain candidate relation subgraphs; S5: Match the referent with the child nodes and the first label field of the child nodes in the candidate relation subgraph respectively; S6: When the referent matches the child node or the referent matches the first annotation field of the child node, obtain the text sequence containing the correct third-person pronoun based on the second annotation field of the child node.

[0027] The key referential parsing method for speech-to-text provided in this publication integrates and organizes various interpersonal relationships in a user's life by constructing a knowledge graph of the user's interpersonal relationships. This provides a foundation for accurately parsing the referential meaning of third-person pronouns, allowing the system to find suitable referents within a relatively complete interpersonal relationship system. By dividing the interpersonal relationship knowledge graph into multiple sub-graphs and classifying them according to interpersonal relationship types, the system can perform more targeted searches when dealing with referential problems. Different sub-graphs correspond to different interpersonal relationship types, improving the efficiency and accuracy of the search.

[0028] The child node contains a first annotation field and a second annotation field. These annotation fields can store more detailed information about interpersonal relationships, such as the secondary title of the contact person and the corresponding third-person pronoun. This helps the system to make judgments based on richer information when matching referents, thereby improving the accuracy of the matching.

[0029] Converting user speech into a text sequence enables the system to perform text processing on the speech content. This is a prerequisite for subsequent referential analysis; only by converting speech into text can the third-person pronouns and their referential relationships be further analyzed. When the text sequence contains third-person pronouns, the system can obtain contextual information and, based on this, determine the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type. This allows the system to extract key information from the text, providing a basis for subsequent sorting and matching. All relation subgraphs are sorted based on relationship type and confidence level, forming a set of relation subgraphs. The sorting process allows the system to traverse the set of relation subgraphs in sorted order, prioritizing relation subgraphs with closer relationships to the referents and higher confidence levels. This narrows the search scope, reducing unnecessary search and matching processes and improving the accuracy and efficiency of the search.

[0030] The system matches the referent object with the child nodes and the first annotation field of the child nodes in the candidate relation subgraph. Through this multi-dimensional matching method, the system can more accurately find the child nodes that match the referent object, thereby determining the correct referential relationship. When the referent object successfully matches a child node or the first annotation field of the child node, the system obtains the text sequence containing the correct third-person pronoun based on the second annotation field of the child node. This step ultimately solves the semantic confusion problem caused by the difficulty in distinguishing third-person pronouns in the existing technology, ensuring the accurate transmission of information.

[0031] In summary, by accurately analyzing the referential relationships of third-person pronouns, the system outputs more accurate and clearer text sequences, improving the quality of speech-to-text conversion. This allows users who rely on speech-to-text functions to obtain and process information more accurately, thereby enhancing the user experience.

[0032] In S1, refer to Figure 2 The flowchart illustrating the method for constructing an interpersonal relationship knowledge graph shows that "constructing a user's interpersonal relationship knowledge graph" includes the following steps: S11: Obtain the user's contact notes information and construct an initial knowledge graph based on the contact notes information; S12: Obtain the user's historical dialogue data on a preset chat platform and extract interpersonal relationship information from the historical dialogue data; S13: Optimize the initial knowledge graph based on interpersonal relationship information to obtain a recommended knowledge graph; S14: Display the recommended knowledge graph to the user, and in response to the user's modification operation command, modify the recommended knowledge graph to generate an interpersonal relationship knowledge graph.

[0033] In S11, the notes field content of all contacts is extracted from the contact management system of the user device. These notes include various information such as the contact's name, nickname, job title, and relationship with the user.

[0034] Extract name remarks from the contact's notes information and identify whether the name remarks contain preset characters (e.g., parentheses, quotation marks, hyphens, etc.). If preset characters are present, extract the primary and secondary titles based on these characters. If preset characters are not present, the name remarks are designated as the primary title. For example, if preset characters are present, remarks within parentheses or quotation marks are usually secondary titles, as are remarks after hyphens. Therefore, if the name remarks are "Teacher Li (Old Li)," "Teacher Li" outside the parentheses is the primary title, and "Old Li" inside the parentheses is the secondary title. If the name remarks are "Manager Zhang (Old Zhang)," "Old Zhang" inside the quotation marks is the secondary title, and "Manager Zhang" outside the quotation marks is the primary title. If the name remarks are "Boss Wang (Old Wang)," "Boss Wang" before the hyphen is the primary title, and "Old Wang" after the hyphen is the secondary title. As can be seen, this method can determine the extraction rules for name remarks based on the specific type of the preset characters, and extract the primary and secondary titles from the name remarks based on the extraction rules.

[0035] Using the user's basic information as the central node, the predetermined primary title is used as the child node of the central node. Each child node represents a contact person and is assigned a unique identifier in the system for subsequent management and retrieval. The first label field of the child node is constructed. If there is a corresponding secondary title for the child node, the secondary title is stored in the first label field of the child node; otherwise, the first label field of the child node is empty.

[0036] The system constructs a second annotation field for child nodes to check if the contact's remarks contain gender information. If it does, the corresponding third-person pronoun is stored in the child node's second annotation field. If not, semantic analysis is performed on the contact's remarks to retrieve the corresponding third-person pronoun. If retrieval fails, the second annotation field is initially set to empty; if successful, the corresponding third-person pronoun is stored in the child node's second annotation field. For example, if the contact's remarks contain "Ms. Zhang," when the system performs semantic analysis on this information, the keyword "Ms." clearly indicates a female gender. Therefore, the system can successfully retrieve the corresponding third-person pronoun "she" and store it in the child node's second annotation field.

[0037] Semantic analysis is performed on contact notes. When the relationship between a user and a contact can be directly determined from the contact notes, a directed edge is created to represent the relationship. For example, if the user is female and one contact note contains "father," a one-way edge is created from the central node to the "father" child node, labeled as "father-daughter relationship." If another contact note contains "Xiao Li" and "colleague," a two-way edge is created between the central node and the "Xiao Li" child node, labeled as "colleague relationship." When the relationship between a user and a contact cannot be determined from the contact notes, the edge between the central node and the contact is not created. Instead, the contact note is recorded for later refinement by combining historical dialogue data and user modifications.

[0038] This method allows for the initial acquisition of a user's interpersonal network based on contact notes, extracting contacts associated with the user and constructing an initial knowledge graph. This initial knowledge graph consists of a central node, child nodes, a first annotation field, a second annotation field, and directed edges. Within this initial knowledge graph, the user's frequently used and infrequently used terms of address for contacts are distinguished. Frequently used terms are used as primary terms to construct nodes, while infrequently used terms are used as secondary terms to enrich the node information. This facilitates rapid matching of the user's voice-related information with the knowledge graph in practical applications. Furthermore, in real-world applications, a user's way of addressing the same contact may vary. Existing technologies tend to overlook secondary terms such as nicknames, leading to matching failures with the knowledge graph and failing to fully utilize its value, thus affecting the accuracy of subsequent speech recognition. This method effectively solves this problem by using a first annotation field to store secondary terms.

[0039] In S12, all historical dialogue data of the user within a specified time period is obtained through an interface with a preset chat platform (such as WeChat, QQ, etc.). Named entity recognition algorithms from natural language processing are used to identify person entities such as names and titles appearing in the historical dialogue data. The dialogue records between the user and each person entity are extracted from the historical dialogue data. By analyzing these dialogue records, relationship information between the user and person entities is extracted. For example, if the dialogue includes phrases like "Xiao Li and I completed this project together," it can be determined that "I" and "Xiao Li" are colleagues. The relationship information between the user and each person entity is combined to construct the user's interpersonal relationship information.

[0040] In S13, the person entities in the interpersonal relationship information are compared with the child nodes and the first annotation field in the initial knowledge graph. If the first annotation field of a person entity and a child node or a person entity and a child node is the same, the edges between the central node and the child nodes are optimized based on the relationship information between the user and the person entity. For example, in the initial knowledge graph, some child nodes do not have edges. By using the relationship information between the user and the person entity, edges can be constructed for these nodes. Or, for edges that already exist in the initial knowledge graph, the relationship information between the user and the person entity can reveal errors in the original edges in their pointing or annotation, which need to be corrected. When analyzing relationship information, for those edges that are consistent with the relationships represented by existing edges, the original edges are directly retained without any modification. If the first annotation fields of the person entity and its child nodes are all different, and the first annotation fields of the person entity and its child nodes are also different, then it is determined whether the person referred to by the person entity and its child nodes are the same. If they are the same, the person entity is stored in the first annotation field of the child node. If they are different, it means that the person entity is not included in the initial knowledge graph. A new child node is constructed in the initial knowledge graph based on the person entity. The relationship information between the user and each person entity is extracted from the interpersonal relationship information. Based on the relationship information between the user and each person entity, an edge is constructed between the central node and the new child node. In addition, if the second annotation field of the child node is empty, the corresponding third-person pronoun of the contact is retrieved again by performing semantic analysis on the historical dialogue data. If the retrieval fails, the second annotation field of the child node remains empty and can be filled in by the user later. If the retrieval is successful, the corresponding third-person pronoun of the contact is stored in the second annotation field of the child node. Through this method, interpersonal relationship information can be fully utilized to improve the initial knowledge graph. The improved knowledge graph is a recommendation knowledge graph that can be presented to users.

[0041] In S14, the optimized recommended knowledge graph is presented to the user in a visual manner, providing an interactive interface that allows users to modify the knowledge graph, such as adding or deleting child nodes, modifying relationships, and supplementing the first and second annotation fields. Based on the user's modification commands, the recommended knowledge graph is corrected accordingly, ultimately generating an accurate interpersonal relationship knowledge graph.

[0042] Before splitting the interpersonal relationship knowledge graph, clarify the types of interpersonal relationships needed, such as family relationships (e.g., parents, children, siblings, etc.), work relationships (e.g., colleagues, superiors and subordinates, etc.), friendship relationships, and classmate relationships. These types can be further refined or merged according to specific business needs and data characteristics.

[0043] The process iterates through each child node and edge in the interpersonal relationship knowledge graph. For each edge, it determines the type of interpersonal relationship it represents. An empty subgraph is created for each interpersonal relationship type, with each central node representing a user. When an edge is found to belong to a certain type, the edge and its connected child nodes are added to the corresponding subgraph. For example, if an edge represents a parent-child relationship, the edge and its corresponding parent-child nodes are added to the family relationship subgraph. After the entire interpersonal relationship knowledge graph has been traversed, multiple decomposed subgraphs are obtained.

[0044] In S2, the user's speech signal is captured via microphone and converted into a digital signal. Speech features are then extracted from this digital signal. A pre-trained acoustic model (such as DNN, RNN, and their variants) is used to match these features with acoustic patterns. A language model (statistical or deep learning-based) then decodes the phoneme sequence into a text sequence. Finally, post-processing is performed to ensure the output conforms to common usage. During processing, the system identifies whether the text sequence contains third-person pronouns. If so, it needs to be corrected promptly to ensure the correct use of third-person pronouns.

[0045] Reference Figure 3 The flowchart illustrating the methods for obtaining the referent, relationship type, and confidence level of the third-person pronoun includes the following steps: "Obtaining contextual information, and based on the contextual information, obtaining the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type." S21: According to the preset first search range, obtain short-range context information based on sentences containing third-person pronouns, and construct the first prompt word based on the short-range context information; S22: Input the first prompt word into the AI ​​model. If the AI ​​model outputs the referent, then execute S23; if the AI ​​model does not output the referent, then execute S24. S23: Obtain relationship type and confidence level based on short-range contextual information and AI model; S24: According to the preset second search range, obtain long-distance context information based on sentences containing third-person pronouns, and construct second prompt words based on the long-distance context information; S25: Input the second prompt word into the AI ​​model. If the AI ​​model outputs the referent, then execute S26; if the AI ​​model does not output the referent, then execute S27. S26: Based on long-distance contextual information and AI models, obtain relationship type and confidence level; S27: According to the preset third search range, obtain cross-dialogue context information based on the timestamp of the statement containing the third-person pronoun, and construct the third prompt word based on the cross-dialogue context information; S28: Based on third-party prompts and AI models, obtain the referent, relationship type, and confidence level.

[0046] In steps S21-S23, using the statement containing the third-person pronoun as the baseline, N statements are taken both forward and backward as short-range context information. The value of N can be preset according to the specific application scenario; for example, in a typical chat scenario, N can be set to 3-5 statements. The reason for this setting is that within a short distance, the correlation between statements is strong, usually providing enough information to determine the referent of the third-person pronoun. Using this method, the statement containing the third-person pronoun is located in the text sequence. Then, according to the defined search range, the preceding and following statements are extracted. The obtained short-range context information is then organized, irrelevant symbols and interjections are removed, and it is combined into a coherent text as the first prompt word.

[0047] The first prompt word is input into the pre-trained AI model. The AI ​​model analyzes and processes the prompt word, attempting to output the referent of the third-person pronoun. If the AI ​​model successfully outputs the referent, proceed to S23; otherwise, it indicates insufficient short-range contextual information, requiring an expanded search scope, and proceed to S24.

[0048] If the AI ​​model successfully outputs the referent, it will also analyze the relationship between the referent and the current conversation subject based on short-range context information, such as whether it is a friend, colleague, or family member. The AI ​​model can determine the relationship type by recognizing keywords and names in the context, and the AI ​​model will give a confidence value, indicating the degree of certainty about the output relationship type.

[0049] In S24-S26, using the statement containing the third-person pronoun as the baseline, the dialogue content consistent with the topic of that statement is traced backward and backward respectively. The extracted dialogue content is used as long-distance contextual information, which can cover more background information and help to more accurately determine the referent. Regarding conversation boundary features, a dialogue often revolves around one or more coherent topics. When there is a clear semantic break or a shift to a completely new topic unrelated to the previous one, it can be considered the end of a user's current conversation and the beginning of a new one. For example, in a business negotiation, the price and specifications of a product are discussed first, and then the conversation suddenly shifts to weekend leisure activities. The change in topic can be used to distinguish different conversations. Using the statement containing the third-person pronoun as the baseline, the conversation is traced backward to the starting point that is semantically closely connected to that statement, and backward to the position where the semantic coherence ends. The content of this interval is used as long-distance contextual information.

[0050] The acquired long-range contextual information is organized, and irrelevant symbols and interjections are removed, forming a coherent text as the second cue word. This second cue word is input into the AI ​​model, and the next step is determined based on the model's output. If the AI ​​model outputs a referent, proceed to step S26; otherwise, the search scope is further expanded, proceeding to step S27. Similar to step S23, if the AI ​​model successfully outputs a referent, it analyzes the relationship type between the referent and the current dialogue subject based on the long-range contextual information and provides a corresponding confidence value.

[0051] In S27 and S28, using the timestamp of the statement containing the third-person pronoun as a reference point, a first time interval is set forward and a second time interval is set backward, such as 24 hours before and 2 minutes after. Within the time range formed by the first and second time intervals, cross-dialogue context information is filtered out. This information includes the user's current conversation and other conversations related to the current conversation. For example, in a chat platform, suppose in a conversation between a user and friend A at 15:30 on July 17, 2025, A says, "He's planning to have a party." Using 15:30 as a reference, cross-dialogue information is filtered out within a time range of 24 hours before and 2 minutes after. In a conversation between the user and friend B at 20:00 on July 16, B mentions, "C is planning to have a party this weekend"; at 15:31 on July 17, friend D sends a message, "I heard C is having a party." These conversations are related to the current conversation mentioning "him" and together constitute cross-dialogue context information, thus inferring that "he" may refer to C.

[0052] This method performs thematic analysis on cross-conversation context information, filtering out dialogue content that differs from the user's current conversation. Specifically, within a defined time frame, it filters all conversations outside the current one, removing those clearly irrelevant to the current topic. For example, if the current conversation is about a work project, conversations related to entertainment and leisure can be excluded. This method extracts statements relevant to the current topic, and combines the current conversation from the cross-conversation context information with the retained dialogue content consistent with the current topic into a coherent text, which serves as the third prompt word.

[0053] When the third prompt word is input into the AI ​​model, it contains a large amount of user dialogue information and is comprehensive. The AI ​​model can output the referent of the third-person pronoun, and correspondingly, it can also output the relationship type between the referent and the current dialogue subject, as well as the corresponding confidence value.

[0054] The AI ​​model comprises an embedding layer, a feature extraction layer, a referential object prediction layer, and a relation type determination layer. The embedding layer takes as input prompts (corresponding to different stages of input), which are in text form and constructed from contextual information obtained from different search scopes. The embedding layer transforms discrete prompts into a continuous vector space representation, facilitating subsequent neural network processing. Using pre-trained word embedding models (such as Word2Vec and GloVe) or task-specific fine-tuned embedding layers, each word in the text is mapped to a fixed-length vector. The embedding layer converts each word in the prompts into a low-dimensional vector representation, resulting in a sequence of word embedding vectors.

[0055] The word embedding vector sequence is input into the feature extraction layer. This layer uses a deep learning model (such as Long Short-Term Memory Network, Gated Recurrent Unit, or Convolutional Neural Network, CNN) to process the word embedding vector sequence and extract the contextual features of the text. These features reflect the relationships between words in the text and the overall semantics of the sentence. The feature vector output from the feature extraction layer is then fed into the referential object prediction layer. Based on the extracted feature vector, the referential object prediction layer predicts the referential object through a fully connected layer or other classifier (such as a Multilayer Perceptron, MLP). Simultaneously, additional output nodes or regression layers are used to calculate the confidence score of the referential object prediction, reflecting the model's certainty in judging the referential object. When the confidence scores of all referential object predictions are less than a preset threshold, the referential object is determined to be unpredictable, and the referential object prediction layer outputs an empty string or a specific identifier indicating that it cannot be predicted. When the confidence score of any referential object prediction is greater than or equal to the preset threshold, the referential object with the highest prediction confidence is output.

[0056] The relationship type determination layer is activated only when the referent object prediction is successful. It provides the feature vector output from the feature extraction layer and the referent object identifier output from the referent object prediction layer to the relationship type determination layer. Based on the determined referent object, the layer further analyzes the relationship between the referent object and the current dialogue subject. The feature vector and referent object information are processed through a fully connected layer or other classifiers to determine the relationship type. Simultaneously, additional output nodes or regression layers are used to calculate the confidence score of the relationship type prediction. Using this method, when the referent object is successfully output, the relationship type determination layer can obtain the relationship type between the referent object and the current dialogue subject, as well as the confidence score of the relationship type. The confidence score ranges from [0, 1].

[0057] Further, the above acquisition of the referent, the relation type and the confidence level is premised on that the third-person pronoun contained in the text sequence meets a preset condition, and the judgment method for whether the preset condition is met is: acquiring downstream information with the sentence containing the third-person pronoun as a reference point according to a preset fourth search range; if the downstream information contains a preset prompt word, determining that the third-person pronoun does not meet the preset condition; if the downstream information does not contain the preset prompt word, determining that the third-person pronoun meets the preset condition.

[0058] Downstream information refers to P sentences acquired backward with the sentence containing the third-person pronoun as the reference point, where P≤N. If the downstream information contains the preset prompt word, a text sequence containing the correct third-person pronoun is directly acquired based on the preset prompt word, wherein the preset prompt words include but are not limited to "the 'ta' with the female radical" and "the 'ta' with the human radical" and "the 'ta' with the roof radical". For example, when a user says in speech: "What will the child eat tonight, have you ordered takeout for ta?", and then the user follows with a sentence "the 'ta' with the female radical", at this time, through speech recognition, the "ta" with the male radical generated in the text sequence is automatically converted into the "ta" with the female radical.

[0059] In S3, relation subgraphs matching the relation types are acquired, and the matched relation subgraphs are sorted in descending order of confidence to form an ordered set of relation subgraphs. For example, it is assumed that there are 3 output relation types, namely work, family and friend, and there are also 3 corresponding relation subgraphs. Among them, the confidence of the work relation type is [X], the confidence of the family relation type is [Y], and the confidence of the friend relation type is [Z]. According to the sorting in descending order of confidence, if [X]>[Y]>[Z], the relation subgraph of work is ranked first, the relation subgraph of family is ranked second, and the relation subgraph of friend is ranked last, thereby forming the set of relation subgraphs.

[0060] In S4 to S6, the set of relation subgraphs is traversed according to the sorting order set in S3, and the currently traversed relation subgraph is used as a candidate relation subgraph. The referent is first matched with child nodes in the candidate relation subgraph; if the matching with the child node is successful, the third-person pronoun stored in a second annotation field of the child node is taken as the correct third-person pronoun, and the text sequence is corrected through the correct third-person pronoun, so that the text sequence containing the correct third-person pronoun can be obtained; if the matching with the child node is unsuccessful, the referent is first matched with a first annotation field of the child node in the candidate relation subgraph; if the matching with the first annotation field is successful, the third-person pronoun stored in the second annotation field of the child node is taken as the correct third-person pronoun; if the matching with the first annotation field is unsuccessful, the process returns to step S4 to continue traversing the set of relation subgraphs and acquire a new candidate relation subgraph until the matching is successful or the traversal of the set of relation subgraphs is completed.

[0061] If no match is found with the referent after the entire relational subgraph has been traversed, the third-person pronoun used in the text sequence is determined to be correct, and the text sequence is output directly. At the same time, the interpersonal relationship knowledge graph is updated based on the obtained referent, the relationship type between the referent and the user, and the third-person pronoun used in the text sequence.

[0062] Furthermore, in the speech recognition process, there is the problem of parsing uncommon characters in titles, especially when processing names containing uncommon characters. The system struggles to accurately identify these characters, leading to transcription errors. Therefore, this application utilizes a constructed interpersonal relationship knowledge graph to solve the problem of inaccurate use of uncommon characters in the title parsing process. (See reference...) Figure 4 The flowchart illustrates the correct appellation resolution method, which includes the following steps: S7: Build a database of uncommon words for user titles; S8: When a text sequence contains titles, and the titles are not commonly used and unique titles, obtain multiple title sequences; S9: When multiple title sequences contain uncommon characters, the correct title is determined from multiple title sequences based on the interpersonal relationship knowledge graph and the uncommon character lexicon. S10: Based on the correct title, obtain the text sequence containing the correct title sequence.

[0063] In S7, refer to Figure 5 The flowchart illustrating the method for constructing a database of uncommon terms of address shows that "constructing a database of uncommon terms of address for users" includes the following steps: S71: Extract uncommon characters for appellations from the standard character set, and build a basic lexicon based on these uncommon characters for appellations; S72: Extract commonly used titles from contact notes and historical dialogue data, optimize the basic vocabulary based on uncommon characters in the commonly used titles, and obtain a vocabulary of uncommon characters in titles.

[0064] In S71, an authoritative standard character set is selected, such as the "Information Technology Chinese Coded Character Set" released in 2022, which contains 87,900 characters. Uncommon characters used for appellations are selected from the standard character set and compiled into a dictionary as the basic dictionary.

[0065] In step S72, the contact remark information is analyzed to extract the name remark information therein. A natural language processing technique is used to extract appellation words from historical conversation data, and these name remark information and appellation words are the user's commonly used appellations. Rare characters are extracted from the user's commonly used appellations. For example, in "Teacher Xun (郇)", "郇" is a rare character. These rare characters are used to optimize the basic lexicon: remove rare characters that are no longer used or have errors, and supplement rare characters that are actually used by the user but not stored yet, and finally construct an appellation rare character lexicon. In addition, based on the historical conversation data, the use frequency of each rare character in the appellation rare character lexicon is counted.

[0066] An incremental learning method is used to update the appellation rare character lexicon, wherein with a cycle of 6 months, changes in user usage habits can be perceived according to changes in contact remark information and newly added conversation data, and then the appellation rare character lexicon is updated.

[0067] In step S8, during the speech-to-text processing, when an appellation is included in the text sequence, the appellation is matched with a pre-constructed common appellation lexicon to determine whether the appellation is a common unique appellation. If it can be accurately matched and corresponds to a unique person in the actual context, it is a common unique appellation, and the text sequence is output directly; if it is not in the common appellation lexicon or may correspond to multiple people, it is not a common unique appellation, and multiple appellation sequences will be obtained for further clarification, for example, "send a message to General Jing (荆)", "send a message to General Jing (经)", "send a message to General Jing (京)", etc.

[0068] In step S9, the difference words between the appellations contained in multiple appellation sequences are extracted, and the difference words are matched with the appellation rare character lexicon; if there is a rare character identical to the difference word in the appellation rare character lexicon, it is determined that the multiple appellation sequences contain a rare character; if there is no rare character identical to the difference word in the appellation rare character lexicon, it is determined that the multiple appellation sequences do not contain a rare character. For example, for the three appellation sequences "send a message to General Jing (荆)", "send a message to General Jing (经)" and "send a message to General Jing (京)", they respectively contain the three appellations "General Jing (荆)", "General Jing (经)" and "General Jing (京)", so the difference words are "荆", "经" and "京" respectively. As long as the appellation rare character lexicon contains any one of the difference words, it indicates that the multiple appellation sequences contain a rare character.

[0069] With reference to Figure 6 the schematic flow diagram of the correct appellation obtaining method shown, "determining a correct appellation from multiple appellation sequences based on an interpersonal relationship knowledge graph and the appellation rare character lexicon" comprises the following steps: S91: obtaining a target syllable based on the difference words between the appellations contained in multiple appellation sequences, and screening candidate rare characters with the same syllable from the appellation rare character lexicon according to the target syllable to form a candidate rare character set; S92: Traverse the set of candidate rare Chinese characters in descending order of usage frequency, determine the current candidate rare Chinese character, and obtain the current candidate title based on the current candidate rare Chinese character; S93: Acquire the speech end time of the user, extract a dialogue segment containing a title entity with the same pronunciation as the current candidate title from the user's current dialogue based on the speech end time and a preset time window size; S94: Determine the role attribution to which the current candidate title points based on the current candidate title and the dialogue segment, and determine a target relationship sub-graph from a plurality of relationship sub-graphs based on the role attribution; S95: Detect whether the target relationship sub-graph contains the current candidate title; if yes, execute S96; if no, execute S97; S96: Determine the current candidate title as the correct title; S97: Continue traversing the set of candidate rare Chinese characters, determine a new current candidate rare Chinese character, until the correct title is obtained or the traversal of the set of candidate rare Chinese characters is completed.

[0070] In S91, a pinyin input method or a pinyin annotation tool is used to obtain the syllable of the different word. For example, the syllables of "荆", "经" and "京" are all "jing". The rare character title lexicon adopts a storage mode classified based on syllables and sorted by usage frequency: all rare characters in the rare character title lexicon are classified according to the syllables of the rare characters, rare characters with the same syllable are grouped into one rare character set, and the same syllable is used as the index of the rare character set; within each rare character set, the rare characters are arranged in descending order of usage times. This storage method facilitates quick positioning of rare characters according to syllables, and can preferentially display rare characters that are used more frequently. Therefore, by comparing the target syllable with the indexes of the rare character sets, the rare character set whose index matches the target syllable is selected as the set of candidate rare Chinese characters.

[0071] In S92, each rare Chinese character contained in the set of candidate rare Chinese characters is a candidate rare Chinese character at this stage, and these candidate rare Chinese characters have been pre-sorted according to usage times, with candidate rare Chinese characters with higher usage times ranked first. The candidate rare Chinese character is sequentially selected as the current candidate rare Chinese character starting from the first element of the sorted set of candidate rare Chinese characters.

[0072] Replace the different word position with the current candidate rare Chinese character to generate the current candidate title. For example, if the current candidate rare Chinese character is "旌", and the original titles are "荆总", "经总" and "京总", then the current candidate title can be "旌总".

[0073] In step S93, for example, the system records that the user's speech end time is 14:30:12, March 18, 2025, and the preset time window size is 2 hours, then the system determines that the search time range is from 12:30:12 to 14:30:12, March 18, 2025. The system extracts dialogue segments containing address entities (e.g., "Jing Gong", "Jing Zong", etc.) that have the same pronunciation as the current candidate address (e.g., "Jing Zong") from the user's dialogues within the period from 12:30:12 to 14:30:12, March 18, 2025. These dialogue segments obtained by adopting the cross-session memory method include sentences describing interpersonal relationships, related events, etc., which are convenient for subsequent use by large language models.

[0074] In step S94, the representation form of the current candidate address is obtained, and the representation form is matched with each address structure form in the preset address structure form set, wherein these address structure forms are summarized according to common role address rules; if the representation form is the same as an address structure form, the role positioning corresponding to the address structure form is taken as the role attribution pointed to by the current candidate address. For example, when the address structure form is "rare character + Gong", the corresponding role positioning is "work role - technical personnel", when the current candidate address is "Gang Gong", since its representation form is the same as the address structure form "surname + Gong", it can be determined that the role attribution pointed to by "Gang Gong" is technical personnel in work roles. When the address structure form is "rare character + Zong", the corresponding role positioning is "work role - leader", when the current candidate address is "Jing Zong", it can be determined that the role attribution pointed to by it is leader in work roles. When the address structure form is "rare character + Jie", the corresponding role positioning is "social friend role - female friend or acquaintance", when the current candidate address is "Kuai Jie", it can be determined that the role attribution pointed to by it is female friend in friend roles.

[0075] If the presentation format and the form of address differ from the structure of the title, the current candidate titles and dialogue fragments are input into the large language model. The large language model extracts key elements from the dialogue fragments, including time information, scene description, key events, and themes. After extracting the key elements, the large language model performs an initial screening of possible role attributions based on the time information and scene description. During work hours and in a work setting, work roles are prioritized; during leisure time and in a family or relaxation setting, family or friend roles are prioritized. For example, in an office setting during work hours, for ambiguous titles, the focus is first on determining the work role. Combining key events and themes further narrows down the scope of role attribution. If the dialogue revolves around work tasks or business collaborations, the role attribution leans towards a work role; if it involves family matters or emotional care, it leans towards a family role; if it discusses entertainment activities or hobbies, it is likely a friend role. For example, in a dialogue about project bidding, the relevant titles are likely to point to a work role. The direct role cues provided by related vocabulary are used to revise and confirm the previous analysis results. If the related terms explicitly mention a role relationship, then this should be used as an important basis to determine the role to which the current candidate title belongs. For example, if the related term mentions "communicating order details with the client," then the related term "client" clearly indicates the role involved.

[0076] Based on the determined role affiliation, select the corresponding subgraph from the previously split relationship subgraphs. For example, if the determined role affiliation is family elder, select the family relationship subgraph as the target relationship subgraph.

[0077] In steps S95-S97, all nodes in the target relation subgraph are traversed to check if there is a node that matches the current candidate title. If a matching node exists, it means that the target relation subgraph contains the current candidate title, and the current candidate title is determined to be the correct title. If no matching node exists, all first label fields in the target relation subgraph are traversed to check if there is a first label field that matches the current candidate title. If a matching first label field exists, it means that the target relation subgraph contains the current candidate title, and the current candidate title is determined to be the correct title. If no matching first label field exists, it means that the target relation subgraph does not contain the current candidate title, and the current candidate title is determined to be the incorrect title. The process then returns to the candidate rare character set, selects the next candidate rare character as the new current candidate rare character, and repeats steps S92-S97 until the correct title is determined or the entire candidate rare character set is traversed.

[0078] If the correct title is still not determined after all candidate rare characters in the candidate rare character set have been traversed, the large language model is used to further analyze the interpersonal relationships and possible identity characteristics implied by the title in conjunction with the dialogue fragment, so as to determine the correct title, or the candidate rare character with the highest frequency of use in the candidate rare character set can be directly selected to form the correct title.

[0079] In multiple title sequences, relying solely on the rare characters already provided in these sequences is often insufficient to determine the correct title. To address this issue, this method selects a set of candidate rare characters based on the target syllable. This set encompasses a wider range of possible rare characters, significantly enriching the selection scope. Based on user habits, candidate rare characters are filtered and verified sequentially according to their frequency of use, taking into full account user preferences and frequency of use. This avoids overlooking rare characters that, while uncommon, appear frequently in actual usage. Furthermore, verification using an interpersonal relationship knowledge graph ensures a more accurate and correct title.

[0080] Furthermore, when multiple title sequences do not contain uncommon characters, for example, if the original title sequences are two, namely "Send a message to General Manager Yu" and "Send a message to General Manager Yu," the title words are extracted from each title sequence, such as "General Manager Yu" and "General Manager Yu." The role type to which each title word points is determined, such as the role type "work." Based on this role type, a specified relationship subgraph is selected from multiple relationship subgraphs, and the title words contained in the specified relationship subgraph are determined as the correct titles.

[0081] In S10, the correct title is combined with any one of the multiple title sequences to form a text sequence containing the correct title sequence. For example, if the original title sequences are three, namely "Send a message to General Manager Jing", "Send a message to General Manager Jing", and "Send a message to General Manager Jing", and the correct title obtained is "General Manager Jing", then the correct title sequence is "Send a message to General Manager Jing", thus obtaining the text sequence with the correct title recognition.

[0082] In summary, the present solution achieves significant optimization and improvement regarding the defects of existing speech-to-text technologies in the recognition of rare Chinese characters in appellations and third-person pronouns. By combining correct appellation sequences and correct third-person pronouns, a text sequence containing correct third-person pronouns can be finally obtained. In terms of recognizing rare Chinese characters in appellations, the present solution breaks through the limitation of general character libraries, and can comprehensively and accurately identify rare Chinese characters and rare names. Different from existing tools that rely on a general character library of about 30,000 Chinese characters which can hardly cover rare characters, the present solution constructs a richer and more comprehensive character library system to ensure that rare characters can also be accurately identified. For example, for rare Chinese characters with extremely low usage frequency such as "Jing", the present solution greatly improves the recognition rate and effectively solves the problem of low recognition rate in the prior art. At the same time, the present solution simplifies the memory correction process, which does not require a built-in input method and multiple manual corrections for memory, significantly improving use efficiency. Moreover, by associating specific contacts with character roles, the present solution accurately outputs names containing rare Chinese characters, avoids frequent manual corrections caused by recognition errors, and truly gives full play to the convenience of the speech-to-text function.

[0083] In terms of third-person pronoun recognition, the present solution has powerful contextual semantic analysis capability, and can accurately distinguish "he", "she" and "it" according to the contextual semantic environment of the conversation, effectively avoiding semantic confusion. In scenarios such as family communication, when a user uses "he" to refer to a family member, the present solution can accurately judge the referent, improving the recognition accuracy of personal pronouns. Taking a chat platform as an example, in the prior art, the recognition accuracy for scenarios involving "she" is relatively low, and when a user describes non-human things such as pets, the prior art usually uses "he" instead of "it", while the present solution can significantly improve this situation. Whether in different scenarios such as family communication, daily communication or formal office work, the present solution can provide users with accurate and clear text conversion results, ensure accurate information transmission, meet the communication requirements of formal communication, office collaboration and other scenarios, and truly realize efficient and accurate application of the speech-to-text function in various scenarios.

[0084] Reference Figure 7 , the present disclosure provides a key reference resolution system in speech-to-text, comprising: a knowledge graph construction module 101, configured to construct a user's interpersonal relationship knowledge graph, and split the interpersonal relationship knowledge graph into a plurality of relationship sub-graphs according to interpersonal relationship types; wherein both the interpersonal relationship types and the relationship sub-graphs comprise child nodes, and the child nodes comprise a first annotation field and a second annotation field; a speech-to-text conversion module 102, configured to convert a user's speech into a text sequence, acquire context information when the text sequence contains a third-person pronoun, and acquire the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type based on the context information; The graph set forming module 103 is used to sort all relation subgraphs based on relation type and confidence level to form a relation subgraph set; The candidate graph acquisition module 104 is used to traverse the set of relation subgraphs in sorted order to obtain candidate relation subgraphs; The referential object matching module 105 is used to match the referential object with the child nodes and the first annotation field of the child nodes in the candidate relation subgraph respectively; The correct sequence acquisition module 106 is used to acquire a text sequence containing the correct third-person pronoun based on the second annotation field of the child node when the referent successfully matches the child node or the referent successfully matches the first annotation field of the child node.

[0085] The various variations and specific examples of the speech-to-text key reference parsing method provided above are also applicable to the speech-to-text key reference parsing system provided in this disclosure. Through the foregoing detailed description of the speech-to-text key reference parsing method, those skilled in the art can clearly understand the implementation method of the speech-to-text key reference parsing system. For the sake of brevity, it will not be described in detail here.

[0086] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0087] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to execute computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the key reference parsing method in the speech-to-text conversion of the foregoing embodiments of this disclosure.

[0088] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0089] like Figure 8This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 8 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0090] like Figure 8 As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0091] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 8 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0092] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the key reference parsing method in speech-to-text conversion according to embodiments of this disclosure are performed.

[0093] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0094] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the methods described in the foregoing embodiments of the present disclosure are performed.

[0095] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0096] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0097] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0098] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0099] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0100] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0101] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0102] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0103] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for parsing key references in speech-to-text conversion, characterized in that, include: Construct a knowledge graph of user interpersonal relationships, and divide the knowledge graph into multiple subgraphs according to the type of interpersonal relationship; wherein, the interpersonal relationship type and the subgraph both include child nodes, and the child nodes include a first annotation field and a second annotation field; The user's voice is converted into a text sequence. When the text sequence contains a third-person pronoun and the third-person pronoun meets a preset condition, context information is obtained. Based on the context information, the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type are obtained. Based on the relationship type and the confidence level, all relationship subgraphs are sorted to form a set of relationship subgraphs; Traverse the set of relation subgraphs in sorted order to obtain candidate relation subgraphs; The referenced object is matched with the child nodes and the first annotation field of the child nodes in the candidate relation subgraph respectively; When the referent successfully matches the child node or the referent successfully matches the first annotation field of the child node, a text sequence containing the correct third-person pronoun is obtained based on the second annotation field of the child node.

2. The key reference parsing method in speech-to-text according to claim 1, characterized in that, The construction of the user's interpersonal relationship knowledge graph includes: Obtain the user's contact notes information, and construct an initial knowledge graph based on the contact notes information; Obtain historical conversation data of users on a preset chat platform, and extract interpersonal relationship information from the historical conversation data; Based on the interpersonal relationship information, the initial knowledge graph is optimized to obtain a recommended knowledge graph; The recommended knowledge graph is displayed to the user, and in response to the user's modification command, the recommended knowledge graph is modified to generate an interpersonal relationship knowledge graph.

3. The key reference parsing method in speech-to-text according to claim 2, characterized in that, The step of obtaining the user's contact remarks information and constructing an initial knowledge graph based on the contact remarks information includes: Extract the name remarks from the contact remarks information and identify whether the name remarks contain preset characters; If the preset characters are included, then the primary title and secondary title are extracted from the name remarks information based on the preset characters; If the preset characters are not included, then the name remarks information will be determined as the primary title; Using the user's basic information as the central node, the main title is used as a child node of the central node; Construct the first annotation field for the child node; If there is a secondary name corresponding to the child node, then the secondary name is stored in the first label field of the child node; If there is no secondary name corresponding to the child node, the first label field of the child node is empty; Detect the gender information contained in the contact remarks information, and store the third-person pronoun corresponding to the gender information in the second annotation field of the child node; Create directed edges to represent the relationships between users and their contacts; The central node, the child nodes, the first annotation field, the second annotation field, and the directed edges constitute the initial knowledge graph.

4. The key reference parsing method in speech-to-text according to claim 1, characterized in that, The step of obtaining context information, and based on the context information, obtaining the referent of the third-person pronoun, the relationship type between the referent and the user, and the confidence level of the relationship type, includes: According to the preset first search range, short-range context information is obtained based on sentences containing third-person pronouns, and a first prompt word is constructed based on the short-range context information; If the AI ​​model outputs the referent based on the first prompt word, then the relationship type and the confidence level are obtained based on the short-distance context information and the AI ​​model. If the AI ​​model does not output the referent based on the first prompt word, then according to the preset second search range, long-distance context information is obtained with sentences containing third-person pronouns as the reference point, and a second prompt word is constructed based on the long-distance context information; If the AI ​​model outputs the referent based on the second prompt word, then the relationship type and the confidence level are obtained based on the long-distance context information and the AI ​​model. If the AI ​​model does not output the referent based on the second prompt word, then according to the preset third search range, cross-dialogue context information is obtained based on the timestamp of the statement containing the third-person pronoun, and a third prompt word is constructed based on the cross-dialogue context information. Based on the third prompt word and the AI ​​model, the referent, the relationship type, and the confidence level are obtained.

5. The key reference parsing method in speech-to-text according to claim 4, characterized in that, The step of obtaining short-range contextual information based on a preset first search range, using sentences containing third-person pronouns as reference points, includes: Using a statement containing a third-person pronoun as a reference point, take N statements forward and N statements backward as short-range context information.

6. The key reference parsing method in speech-to-text according to claim 4, characterized in that, The step of obtaining long-distance contextual information according to a preset second search range, using sentences containing third-person pronouns as reference points, includes: Using sentences containing third-person pronouns as reference points, dialogue content consistent with the theme of the sentences is extracted both forward and backward, and the extracted dialogue content is used as long-distance contextual information.

7. The key reference parsing method in speech-to-text according to claim 4, characterized in that, The step of obtaining cross-dialogue context information according to a preset third search range, using the timestamps of statements containing third-person pronouns as reference points, includes: Using the timestamp of the statement containing the third-person pronoun as the reference point, set the first time interval forward and the second time interval backward; Within the time range formed by the first time interval and the second time interval, cross-dialogue context information is filtered out.

8. The key reference parsing method in speech-to-text according to claim 4, characterized in that, Also includes: According to the preset fourth search range, downstream information is obtained based on statements containing third-person pronouns; If the downstream information contains a preset prompt word, it is determined that the third-person pronoun does not meet the preset condition. Based on the preset prompt word, a text sequence containing the correct third-person pronoun is obtained. If the downstream information does not contain a preset prompt word, then the third-person pronoun is determined to meet the preset condition.

9. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the key reference parsing method in speech-to-text as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the key reference parsing method in speech-to-text as described in any one of claims 1-8.

11. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-8.

Images