Voice data processing method and device, electronic equipment, medium and program product

Abstract

The invention provides a voice data processing method and device, electronic equipment, a medium and a program product, relates to application of a large model in remote voice interaction, and can be applied to the technical field of artificial intelligence. The method comprises the following steps: receiving a target information fragment sent by a client, wherein the target information fragment is obtained by analyzing and extracting data corresponding to remote interaction voice by the client; generating a candidate element set based on the target information fragment; field constraint information corresponding to the target form is obtained, field mapping is executed on the candidate element set based on the field constraint information, and a structured field set is obtained; and performing element verification on the structured field set based on the field constraint information, executing conflict detection and / or missing element determination under the condition that the element verification is not passed, generating an updated structured field set, generating a backfill write-in request based on the updated structured field, and sending the backfill write-in request to the target system.

Description

TECHNICAL FIELD

[0001] The present application relates to the application of large models in remote voice interaction, and relates to the technical field of artificial intelligence, and more particularly to a voice data processing method and device, equipment, medium and program product. BACKGROUND

[0002] With the popularity of remote online agency scenarios, users often describe the agency matters through voice, and the server needs to convert the spoken content into structured field data that can be written into the business system to complete form filling and process flow.

[0003] At present, some existing solutions can realize voice-to-text conversion and text intent recognition, keyword extraction and other processing, but the output is mostly fragmented semantic results, which is difficult to form a definite mapping with the field set, field type and verification rule of the target form. At the same time, information in multi-round dialogue is provided in a non-linear manner and there is reference, repetition and self-correction, which is prone to field missing, value conflict and insufficient consistency. Due to the lack of field-level constraint verification, conflict detection, value change tracking, and automatic completion and error correction mechanism in abnormal situations, the extraction result often still needs to be manually checked, formatted and backfilled, and it is difficult to form an automatic closed-loop processing flow. SUMMARY

[0004] In view of the above problems, the embodiments of the present application provide a voice data processing method, device, equipment, medium and program product.

[0005] According to a first aspect of the present application, a voice data processing method is provided, applied to a server side, the method comprising: receiving a target information segment sent by a client, the target information segment being obtained by the client analyzing and extracting data corresponding to a remote interactive voice; generating a candidate element set based on the target information segment; obtaining field constraint information corresponding to a target form, performing field mapping on the candidate element set based on the field constraint information, and obtaining a structured field set; performing element verification on the structured field set based on the field constraint information, performing conflict detection and / or missing element determination if the element verification fails, and generating an updated structured field set, wherein if the conflict detection indicates that there is a conflict value in the same field, a target field value is determined based on a time priority or a confidence priority, and the structured field set is updated; if it is determined that there is a missing element, generating missing element identification information, the missing element identification information being used to trigger supplementary processing of the client; and generating a backfill write request based on the updated structured field and sending it to a target system.

[0006] According to an embodiment of the present application, the target information segment at least includes a key information segment and corresponding associated information, and the generating a candidate element set based on the target information segment includes: aggregating the target information segment according to a preset information type to obtain a field value candidate item set; inputting the field value candidate item set into a pre-trained large language model for semantic processing to obtain a first semantic check result and a first error correction suggestion, wherein the first semantic check result is used to represent semantic consistency or conflict of the field value candidate item; obtaining rule information in a target business rule library, performing rule check on the field value candidate item set based on the rule information to obtain a first rule check result, wherein the first rule check result is used to represent one or more of format compliance, value range compliance and mandatory element satisfaction of the field value candidate item; based on the first semantic check result, the first error correction suggestion and the first rule check result, performing error correction processing or rejection processing on the field value candidate item set; and associating corresponding associated information with the field value candidate item in the processed field value candidate item set to generate the candidate element set.

[0007] According to an embodiment of the present application, the obtaining field constraint information corresponding to the target form and performing field mapping on the candidate element set based on the field constraint information to obtain a structured field set includes: determining a field set based on the target form, and generating a field semantic representation for each field in the field set respectively; configuring a field type rule, a format rule, a mandatory rule and a cross-field consistency rule for each field to obtain the field constraint information; and based on the field constraint information, calculating a mapping score of a field value candidate item in the candidate element set with the field semantic representation, and when the mapping score meets a preset threshold condition, mapping the field value candidate item to a corresponding field to generate the structured field set.

[0008] According to an embodiment of the present application, the performing element check on the structured field set based on the field constraint information includes: performing a first element check on the structured field set based on the mandatory rule, the field type rule and the format rule, and in the case that the first element check fails, generating a missing field set and an illegal field set; and performing a second element check on the structured field set based on the cross-field consistency rule, and in the case that the second element check fails, generating a consistency conflict set, wherein the missing field set, the illegal field set and the consistency conflict set are used to generate the missing element identification information or as an input for performing conflict detection.

[0009] According to a second aspect of this application, a voice data processing method is provided, applied to a client. The method includes: acquiring voice data during a remote interaction process and converting the voice data into text data; performing data analysis and extraction based on the text data to obtain a target information fragment, and sending the target information fragment to a server, so that the server performs element extraction and element verification based on the target information fragment; obtaining missing element identification information sent by the server, wherein the acquisition of the missing element identification information indicates that the element verification failed and that a missing element exists; and performing supplementary processing based on the missing element identification information to obtain a supplementary target information fragment, and sending the supplementary target information fragment to the server, wherein the supplementary target information fragment is used to cause the server to update the generated structured field set and drive the backfilling of the target form.

[0010] According to an embodiment of this application, the step of performing data analysis and extraction based on the text data to obtain the target information fragment includes: inputting the text data into a pre-trained large language model for semantic processing to obtain a second semantic verification result and a second error correction suggestion, wherein the second semantic verification result is used to characterize the semantic consistency or conflict of key information; obtaining rule information in a target business rule base, performing rule verification on the text data based on the rule information to obtain a second rule verification result, wherein the second rule verification result is used to characterize one or more of the following: format compliance, value range compliance, and mandatory element satisfaction of the key information; and performing error correction processing or removal processing on the key information based on the semantic verification result, the second error correction suggestion, and the second rule verification result to obtain the target information fragment.

[0011] According to an embodiment of this application, sending the supplementary target information fragment to the server includes: associating the supplementary target information fragment with the current session identifier to generate initial data to be sent; writing time information into the initial data to be sent to obtain the data to be sent, wherein the time information is used to characterize the generation order information of the supplementary target information fragment in the remote interaction process; and sending the data to be sent to the server.

[0012] According to an embodiment of this application, the step of performing supplementary processing based on the missing element identification information to obtain the supplementary target information fragment includes: parsing the missing element identification information to obtain the content to be supplemented and the corresponding format constraints; generating a supplementary inquiry prompt based on the format constraints, and collecting supplementary data based on the supplementary inquiry prompt; and performing input validity verification on the supplementary data, and if the input validity verification passes, performing data analysis and extraction based on the supplementary data to obtain the supplementary target information fragment.

[0013] A third aspect of this application provides a voice data processing apparatus applied to a server. The apparatus includes: a target information fragment acquisition module, configured to: receive a target information fragment sent by a client, wherein the target information fragment is obtained by the client through analysis and extraction of data corresponding to remote interactive voice; generate a candidate element set based on the target information fragment; a field mapping module, configured to: acquire field constraint information corresponding to a target form; perform field mapping on the candidate element set based on the field constraint information to obtain a structured field set; an element verification module, configured to: perform element verification on the structured field set based on the field constraint information; perform conflict detection and / or missing element determination if the element verification fails; generate an updated structured field set, wherein, if conflict detection indicates that there are conflicting values ​​in the same field, the target field value is determined based on time priority or confidence priority and the structured field set is updated; if a missing element is determined, generate missing element identification information, wherein the missing element identification information is used to trigger supplementary processing by the client; and a backfilling request generation module, configured to: generate a backfilling request based on the updated structured field set and send it to the target system.

[0014] A fourth aspect of this application provides a voice data processing apparatus applied to a client. The apparatus includes: a voice-text data acquisition module, configured to: acquire voice data during remote interaction and convert the voice data into text data; a target information fragment sending module, configured to: perform data analysis and extraction based on the text data to obtain a target information fragment, and send the target information fragment to a server, so that the server performs element extraction and element verification based on the target information fragment; a missing information acquisition module, configured to: acquire missing element identification information sent by the server, wherein the acquisition of the missing element identification information indicates that the element verification fails and that a missing element is determined to exist; and a supplementary processing module, configured to: perform supplementary processing based on the missing element identification information to obtain a supplementary target information fragment, and send the supplementary target information fragment to the server, wherein the supplementary target information fragment is used to enable the server to update the generated structured field set and drive the backfilling of the target form.

[0015] A fifth aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] A sixth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0017] A seventh aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0018] According to embodiments of this application, field mapping is performed based on field constraint information corresponding to the target form, which can automatically align candidate elements to the target form fields, reducing the workload of manual field-by-field screening, comparison, and entry, thereby improving the efficiency of information organization and processing throughput in remote agency scenarios. At the same time, the server performs element verification on the structured field set based on field constraint information, and performs conflict detection and / or missing element determination when the verification fails: On the one hand, when there are conflicting values ​​in the same field, the target field value is determined and the structured field set is updated by time priority or confidence priority, which can effectively suppress the risk of incorrect filling caused by multiple rounds of dialogue, slips of the tongue, or fluctuations in voice recognition, and improve the consistency and accuracy of field values; on the other hand, when it is determined that there are missing elements, the client-side supplementation processing is triggered by generating missing element identification information, forming a closed loop of targeted completion for missing information, reducing backfilling failures and repeated communication caused by incomplete information. Finally, based on the updated structured fields, a backfilling request is generated and sent to the target system, enabling key elements to form a closed-loop automated link from collection, mapping, verification to backfilling. This shortens business processing time, improves backfilling success rate and data quality, and reduces operational risks caused by incorrect or missing data. Attached Figure Description

[0019] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 The illustration shows an application scenario diagram of a voice data processing method, apparatus, device, medium, and program product according to embodiments of this application;

[0021] Figure 2 A flowchart illustrating a voice data processing method applied to a server according to an embodiment of this application is shown schematically.

[0022] Figure 3 A flowchart illustrating a voice data processing method applied to a client according to an embodiment of this application is shown schematically;

[0023] Figure 4 This illustration schematically shows a remote online business processing system according to some exemplary embodiments of this application;

[0024] Figure 5A This schematically illustrates a structural block diagram of a voice data processing apparatus applied to a server according to an embodiment of the present application;

[0025] Figure 5B This schematically illustrates a structural block diagram of a voice data processing apparatus applied to a client according to an embodiment of this application; and

[0026] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a voice data processing method according to an embodiment of this application. Detailed Implementation

[0027] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0031] In one or more embodiments described herein, the term "large model" can refer to a deep learning model with a large number of model parameters, which can include hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. Large models can also be called foundational models or basic models. They are pre-trained using large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability, such as large language models and multimodal pre-trained models. It should be understood that in practical applications, large models only require a small number of samples to fine-tune the pre-trained model before being applied to different tasks. Large models can be widely used in natural language processing, computer vision, and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering, image captioning, and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. Major application scenarios for large models can include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0032] With the widespread adoption of remote communication, voice recognition, and online business systems, remote errand processing for various industries is gradually migrating from offline counters to online service models. Currently, users can describe the matters they need to handle via voice. The server needs to obtain multiple structured elements corresponding to the matter and write these elements into the form fields of the business system to trigger subsequent validation, submission, and workflow. Due to the wide range of errand-handling matters, high field complexity, and inter-field constraints, as well as differences in field definitions, formatting rules, and required fields across different systems, how to stably and accurately convert natural language expressions into structured data that can be processed by the system has become a key issue in remote errand-handling scenarios.

[0033] In some solutions, speech recognition technology can convert speech signals into text sequences, while natural language processing (NLP) technology can perform intent recognition, keyword extraction, or slot filling on the text. However, these technologies primarily target dialogue understanding or text understanding, and their output is typically non-deterministic semantic labels, keyword sets, or fragmented information, making it difficult to directly map them to the field sets, field types, and validation rules of specific business systems. Especially in cross-business and cross-system agency scenarios, the same semantic element may correspond to different field names, data types, and value ranges in different systems, often accompanied by mandatory field constraints, format constraints, logical constraints, and consistency constraints, such as constraints between amount and currency, document number and document type, and time / location and event type. Without structured modeling and constraint-driven parsing mechanisms for form fields, speech-to-text transcription still requires manual field alignment, formatting, and completion.

[0034] Meanwhile, remote dialogues are characterized by their conversational style, incomplete expression, and frequent contextual references. Users often provide information in a non-linear manner, exhibiting phenomena such as flashbacks, insertions, repeated confirmations, and self-corrections. The same field may be mentioned multiple times in different rounds, with its value changing. For example, a user might initially provide an approximate amount and then correct it to an exact amount, or describe the event time and then change it again when supplementing materials. Existing technologies, when handling multi-round dialogues, typically lack the ability to manage field-level versions, detect conflicts, and verify consistency. This can easily lead to inconsistencies between the extracted results and the user's final confirmation, thus compromising the traceability and reliability of the backfilled data.

[0035] Furthermore, existing solutions often fail to establish a closed-loop technical chain from information collection to form completion. On one hand, speech recognition and text extraction modules are often loosely integrated with business system forms, resulting in a lack of linkage between extraction results and field validation rules. This necessitates manual verification, format conversion, and supplementation of missing items before form completion. On the other hand, when fields are missing or conflicting, the system struggles to automatically generate field-specific clarification questions to drive users to supplement information, and it also fails to adaptively adjust extraction strategies based on validation feedback, thus hindering automatic error correction and iterative convergence.

[0036] Based on this, embodiments of this application provide a voice data processing method applied to a server. The method includes: receiving a target information segment sent by a client, the target information segment being obtained by the client through analysis and extraction of data corresponding to remote interactive voice; generating a candidate element set based on the target information segment; obtaining field constraint information corresponding to a target form; performing field mapping on the candidate element set based on the field constraint information to obtain a structured field set; performing element verification on the structured field set based on the field constraint information; performing conflict detection and / or missing element determination if the element verification fails, generating an updated structured field set; wherein, if conflict detection indicates that there are conflicting values ​​in the same field, determining the target field value based on time priority or confidence priority and updating the structured field set; generating missing element identification information if missing elements are determined, the missing element identification information being used to trigger supplementary processing by the client; and generating a backfilling request based on the updated structured fields and sending it to the target system. According to embodiments of this application, field mapping is performed based on field constraint information corresponding to the target form, which can automatically align candidate elements to the target form fields, reducing the workload of manual field-by-field screening, comparison, and entry, thereby improving the efficiency of information organization and processing throughput in remote agency scenarios. At the same time, the server performs element verification on the structured field set based on field constraint information, and performs conflict detection and / or missing element determination when the verification fails: On the one hand, when there are conflicting values ​​in the same field, the target field value is determined and the structured field set is updated by time priority or confidence priority, which can effectively suppress the risk of incorrect filling caused by multiple rounds of dialogue, slips of the tongue, or fluctuations in voice recognition, and improve the consistency and accuracy of field values; on the other hand, when it is determined that there are missing elements, the client-side supplementation processing is triggered by generating missing element identification information, forming a closed loop of targeted completion for missing information, reducing backfilling failures and repeated communication caused by incomplete information. Finally, based on the updated structured fields, a backfilling request is generated and sent to the target system, enabling key elements to form a closed-loop automated link from collection, mapping, verification to backfilling. This shortens business processing time, improves backfilling success rate and data quality, and reduces operational risks caused by incorrect or missing data.

[0037] It should be noted that the voice data processing methods, apparatus, devices, media, and program products defined in this application can be used in the fields of artificial intelligence technology and fintech, and can also be used in a variety of other fields besides artificial intelligence technology and fintech. The application fields of the voice data processing methods, apparatus, devices, media, and program products provided in the embodiments of this application are not limited.

[0038] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0039] In scenarios where personal information is used for automated decision-making, the methods, devices, and systems provided in this application all provide users with corresponding operation entry points for users to choose to agree to or reject the automated decision results; if the user chooses to reject, the process enters the expert decision-making process.

[0040] Figure 1 The illustration shows an application scenario diagram of the voice data processing method, apparatus, device, medium, and program product according to embodiments of this application.

[0041] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc. For example, a user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send information, etc.

[0042] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal devices. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, and it can include, but is not limited to, applications, mini-programs, etc., installed on the terminal devices, and can also be in the form of web pages.

[0043] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and basic cloud computing services such as big data. The server can be the backend server of the aforementioned target application, used to provide backend services to the clients of the target application.

[0044] It should be noted that the voice data processing method provided in this application embodiment can generally be executed by the server 105. Accordingly, the voice data processing device provided in this application embodiment can generally be located in the server 105.

[0045] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0046] The following will be based on Figure 1 The described scene, through Figures 2-4 The voice data processing method of the embodiments of this application will be described in detail.

[0047] Figure 2 A flowchart illustrating a voice data processing method applied to a server according to an embodiment of this application is shown schematically.

[0048] like Figure 2 As shown, the voice data processing method 200 of this embodiment includes steps S210 to S240, and the voice data processing method 200 can be executed by the server 105.

[0049] In step S210, a target information segment sent by the client is received. The target information segment is obtained by the client through analysis and extraction of data corresponding to the remote interactive voice. A set of candidate elements is generated based on the target information segment.

[0050] In the embodiments of this application, after receiving the target information fragment sent by the client, the server can process the target information fragment as a phase summary of a remote interaction. The target information fragment can be formed by the client analyzing and extracting the data corresponding to the voice during the voice interaction process. It can come from single-turn question and answer or from the cumulative results of multi-turn dialogue. The target information fragment can include key fragments related to the business, such as name, contact information, amount, time, location, business type, application items, etc., and can also include contextual information associated with the key fragments, such as the order of appearance of the key fragments in the dialogue, the number of rounds of appearance, the position of the sentence fragment, whether it is the result after user confirmation, etc. After obtaining the target information fragment, the server can first perform format regularization and noise filtering on the target information fragment, such as removing irrelevant greetings, removing duplicate fragments, and merging adjacent and semantically consistent fragments.

[0051] In some embodiments, generating a candidate element set based on a target information fragment can be accomplished through aggregation and merging. The server can aggregate the target information fragments according to preset information types, merging key fragments of the same or similar types into the same candidate pool to form multiple field value candidate sets. These multiple field value candidate sets are then aggregated into a candidate element set. Taking bank transfer processing as an example, the target information fragment may contain two monetary expressions: "transfer five thousand" and "transfer five thousand five." The server can include both in the monetary candidate pool for subsequent verification and conflict handling.

[0052] In the embodiments of this application, the generation of the candidate element set can also combine multiple extraction granularities to improve adaptability to complex scenarios. For highly structured information, such as phone numbers, ID numbers, dates, and amounts, the server can prioritize rule-based normalization, such as uniformly removing spaces and separators, standardizing date representation, and standardizing monetary units and decimal places. For less structured information, such as business requests, accident descriptions, and reasons for processing, the server can retain relatively complete sentence-level fragments as candidates, and, if necessary, attach keyword summaries or core sentence extraction results, making it easier to determine their association with the target form fields in subsequent processing stages.

[0053] In embodiments of this application, the candidate element set may also include candidate combinations with cross-type associations to address the mixed narratives commonly found in spoken language. A user may mention multiple types of elements simultaneously in the same sentence, such as "My name is Zhang San, and today I transferred 5,000 to a bank card at xx bank," which includes name, time, amount, and also implicitly suggests bank type and account type. During aggregation, the server can not only categorize various fragments into their corresponding candidate pools but also record the co-occurrence relationships between fragments, such as co-occurrence within the same sentence, co-occurrence between adjacent sentences, or co-occurrence in the same round, and save this relationship as association information for the candidate element set. Based on this, subsequent processing stages can more accurately distinguish easily confused fields such as "receiving bank" and "paying bank," and can better determine the pairing relationships between amount and currency, and between account and bank.

[0054] In some embodiments, the generation of the candidate element set can incorporate confidence information to enhance the interpretability of subsequent decisions. The client may have already generated some confidence information during analysis and extraction, such as speech recognition clarity, segment matching strength, and user confirmation markers. The server can receive this information along with key segments and store it as related information in the candidate element set. Even if the client does not provide confidence information, the server can still generate a confidence assessment result based on the segment's normality, its matching degree with common patterns, and its coherence with the context.

[0055] In the embodiments of this application, to adapt to the diverse needs of different businesses, the generation of the candidate element set can also employ configurable element templates. The server can select different element templates based on the business type or target form type. The template defines the types of information to focus on, acceptable expression formats, and the organization method of the candidate pool. Taking money transfer agency as an example, the template can emphasize the amount, receiving account, recipient's name, purpose, etc. Through the template-driven aggregation strategy, the candidate element set can cover general scenarios and also focus more on key elements in specific scenarios, thereby making it easier to form structured backfill results in subsequent steps.

[0056] In step S220, the field constraint information corresponding to the target form is obtained, and field mapping is performed on the candidate element set based on the field constraint information to obtain a structured field set.

[0057] In the embodiments of this application, before entering the field mapping stage, the server can first obtain the field constraint information corresponding to the target form. The field constraint information can originate from form configurations provided by the business system, pre-defined field definition files on the server, field lists generated by business process orchestration, or statistical summaries of historical processing records. The field constraint information can be used to describe which fields the target form contains, the semantic meaning of each field, the hierarchical relationships and filling order between fields, and the format and value requirements that fields must meet when filling them out. By obtaining the field constraint information, the server can clearly define the target form of the structured output, avoiding incomplete fields or semantic confusion caused by freely organizing based solely on dialogue fragments, thus providing a unified reference for the conversion of the candidate element set into a structured field set.

[0058] In some embodiments, field constraint information may include field type information and format specification information. Field type information may indicate whether a field belongs to different types such as text, number, date, document, account, or address. Format specification information may indicate field length, character set range, fixed prefix or suffix, date representation method, and decimal place requirements for amounts. Taking a money transfer scenario as an example, the target form may include fields such as transfer amount, recipient name, recipient account, and recipient bank. The transfer amount needs to be a parsable numerical expression and meet the value range requirements; the recipient account needs to meet the number of digits and validation requirements; and the bank name needs to match a predefined set of institution names or a recognizable set of abbreviations. Based on these constraints, the server can normalize fragments of different forms in the candidate element set and prioritize candidates that meet the type and format requirements during mapping, thereby improving mapping accuracy and backfillability.

[0059] In embodiments of this application, the field constraint information may further include field semantic description information to assist in the semantic alignment between the candidate element set and the target form field. The field semantic description information may include the field's Chinese name, common aliases, explanations of business definitions, and example filling styles.

[0060] In some embodiments, field constraint information may include dependencies or linkage constraints between fields to support mapping strategies that better align with business logic. Dependencies can indicate that certain fields can only be determined after other fields are determined. For example, document type and document number are mutually constrained; the set of required fields can only be determined after the business type is determined; and the branch selection field can only be determined after the regional information is determined. When performing field mapping, the server can first determine the mapping results of key fields based on dependencies, and then dynamically adjust the candidate range of other fields based on the results of the key fields.

[0061] In the embodiments of this application, field mapping can be implemented using an alignment method based on matching scores. The server can calculate the degree of matching between each candidate option in the candidate element set and each field of the target form, taking into account field constraint information, and determine the mapping result accordingly. Matching scores can incorporate various factors, such as the degree of type matching, format fulfillment, similarity to the semantic description of the field, co-occurrence with contextual keywords, and source confidence information of the candidate option. Taking bank account opening agency services as an example, "Zhang San" has a high semantic and type matching score with the customer name field, but may also match the contact person's name field. In this case, further differentiation can be achieved by considering whether the phrase "contact person" appears in the context.

[0062] In some embodiments, to improve adaptability to colloquial expressions, abbreviations, and diverse writing styles, field constraint information can include an extensible thesaurus or mapping table. For example, the "Bank of Account" field can maintain a mapping table from the institution's abbreviation to its standardized name, the "Document Type" field can maintain a mapping table from common colloquialisms to standard types, and the "Region" field can maintain a mapping table from alternative names to standard administrative division names. During field mapping, the server can first standardize or normalize candidate options before matching them with the field constraint information.

[0063] In the embodiments of this application, the results of field mapping can be output in the form of a structured field set. This structured field set can record the correspondence between field identifiers and field values, and optionally record the source information and mapping basis information of the field values. The source information may include which dialogue round the candidate comes from, the corresponding sentence fragment, or time sequence information. The mapping basis information may include the matching score, the type and format rules satisfied, and the triggered synonym normalization rules, etc. By retaining this auxiliary information in the structured field set, the server can more easily locate the source of the problem when performing element verification, conflict detection, and missing element determination. Furthermore, it can generate more focused feedback content when client-side supplementary processing is required, thereby improving closed-loop processing efficiency and user experience.

[0064] In step S230, the structured field set is validated based on field constraint information. If the validation fails, conflict detection and / or missing element determination are performed to generate an updated structured field set. If conflict detection indicates that there are conflicting values ​​in the same field, the target field value is determined based on time priority or confidence priority and the structured field set is updated. If missing elements are determined, missing element identification information is generated, which is used to trigger supplementary processing on the client side.

[0065] In the embodiments of this application, after obtaining the structured field set, the server can perform element validation on the structured field set based on field constraint information to determine whether the current field set meets the filling requirements of the target form. Element validation can cover multiple dimensions such as field completeness, field type and format, field value range, and logical consistency between fields. Among them, field completeness validation can be used to determine whether all required fields have valid field values; field type and format validation can be used to determine whether field values ​​conform to the expression form specified by the field constraint information, such as whether the amount is a parsable numerical value, whether the date meets the specified representation method, and whether the document number meets the length and character composition requirements. Logical consistency validation between fields can be used to determine whether there are obvious contradictions between multiple field values, such as mismatch between document type and document number rules, mismatch between the relationship between the region field and the branch field, or a business type of loss reporting but with field combinations related to account opening, etc.

[0066] In some embodiments, when element validation fails, the server can perform conflict detection to identify situations where multiple mutually exclusive values ​​exist for the same field. Conflicting values ​​may arise from repeated statements in multiple rounds of dialogue, or from differences in expression due to slip-ups or fluctuations in speech recognition. For example, a user might first say "transfer 5,000," then add, "I misspoke, it's 5,500." After initial mapping, the structured field set may retain two candidate monetary values ​​simultaneously. Conflict detection can identify conflicting values ​​in the monetary field. After detecting conflicting values ​​in the same field, the server can determine the target field value based on time priority, for example, by using the expression that appears later and is closer to the current dialogue as the valid value. This allows the structured field set to automatically converge to the more likely correct result as the dialogue progresses.

[0067] In the embodiments of this application, in addition to time priority, the server can also determine the target field value based on confidence priority to address situations where the user has not explicitly corrected the data but multiple candidate expressions exist. Confidence can be derived from fragment quality indicators provided by the client, or it can be generated by the server based on the matching degree between field constraint information and candidate values. Taking ID number as an example, a candidate value that meets the length and verification rules and is consistent with the ID type can have a higher confidence level; another candidate value with missing characters or containing abnormal characters will have a lower confidence level. When conflict detection indicates that there are conflicting values ​​for the same field, the server can prioritize the candidate value with higher confidence as the target field value and update the structured field set accordingly, thereby improving the accuracy of backfilling fields without increasing additional interaction.

[0068] In some embodiments, when element validation fails, the server can also perform missing element determination to identify which fields have not yet obtained available field values. Missing element determination can be based on mandatory requirements in field constraint information or on dynamic mandatory rules driven by business type. For example, if the business type is personal account opening, fields such as name, document type, document number, and contact information may be mandatory. After determining the existence of missing elements, the server can generate missing element identification information to indicate the category, quantity, or priority of the missing fields, and send the missing element identification information to the client, thereby triggering the client to perform targeted supplementary processing.

[0069] In some embodiments, conflict detection and missing element determination can be performed in parallel or in a priority-based combination to adapt to different failure reasons and processing paths. For scenarios where conflicting values ​​exist in the same field and a missing field also exists, the server can first resolve the conflict based on time priority or confidence priority to ensure the convergence of key fields as quickly as possible, and then generate missing element identification information based on missing element determination to trigger client-side supplementation. For scenarios with only conflicts but no missing elements, the server can directly update the structured field set after resolving the conflict on the server side, reducing client interaction. For scenarios with only missing elements but no conflicts, the server can directly output missing element identification information to trigger supplementation processing, avoiding unnecessary conflict calculations. By flexibly selecting a combination strategy for conflict detection and missing element determination when element validation fails, the updated structured field set can reach a fillable state more quickly, reducing overall processing time and interaction rounds.

[0070] In the embodiments of this application, when the element verification passes, the server can directly enter the subsequent backfilling stage based on the structured field set.

[0071] In step S240, a backfill request is generated based on the updated structured fields and sent to the target system.

[0072] In the embodiments of this application, after obtaining the updated structured field set, the server can enter the backfilling stage to complete the linkage processing with the target system. The updated structured field set has usually passed element validation and completed conflict resolution and missing data completion, meeting the field requirements and data specifications of the target form. The server can generate a backfilling request based on the updated structured field set. The backfilling request may include the target form identifier, the correspondence between field identifiers and field values, the business type identifier, and backfilling sequence information. The organization method of the backfilling request can match the interface requirements of the target system, for example, using field key-value pairs to carry field values, or using form data objects to carry the entire table field set, so that the target system can directly receive and complete the write processing in the existing process.

[0073] In some embodiments, before generating the fill-in request, the server can perform adaptation processing on the updated structured field set to ensure that the field definitions are consistent with the target system. Adaptation processing may include field naming mapping, field type conversion, unit and precision normalization, and enumeration value standardization.

[0074] In the embodiments of this application, the server can select different backfill strategies according to the business process requirements of the target system. For target systems that support writing the entire table at once, the server can assemble the updated structured field set into complete form data according to the target form structure and initiate a write request all at once. For target systems that require step-by-step submission, the server can split the backfill request into multiple stage requests. For example, first submit identity information and business type to complete the acceptance registration, then submit fields such as amount, account, and purpose to complete the business element entry, and finally submit confirmation information to complete the submission.

[0075] In some embodiments, after sending a write request, the server can receive the write result returned by the target system and perform subsequent processing based on the write result. The write result may include information such as whether the write was successful, the type of failure reason, the failure field identifier, and the business transaction identifier generated by the target system. When the write is successful, the server can send the business transaction identifier back to the client for subsequent querying or to notify the user of the processing progress. When the write fails, the server can decide whether to trigger supplementary processing on the client again or re-execute field adaptation based on the failure reason.

[0076] In the embodiments of this application, the sending of the backfill request can employ a configurable routing strategy to adapt to scenarios where multiple target systems coexist. The server can select different target system interfaces based on business type, region, institution, or system version, and generate a matching backfill request. Taking a multi-branch system of a financial institution as an example, different branches may use different versions of the business system or different field definitions. The server can select the corresponding field mapping rules and interface addresses when generating the backfill request.

[0077] According to embodiments of this application, the server receives target information fragments extracted by the client and generates a candidate element set. Combined with field constraint information corresponding to the target form, field mapping is completed. This automatically aligns key information in voice interaction into a writable structured field set, reducing manual screening and field-by-field entry workload and improving the efficiency of remote processing. Furthermore, element validation is performed on the structured field set based on field constraint information. If validation fails, conflict detection and missing element determination are performed: when conflicting values ​​exist for the same field, the target field value is determined based on time priority or confidence priority, and the field set is updated. This reduces the risk of incorrect filling due to slips of the tongue, corrections, and recognition fluctuations. When missing elements exist, missing element identification information is generated to trigger client-side supplementation processing, forming a targeted completion loop and improving the completeness and success rate of backfilling. Finally, a backfilling request is generated and sent to the target system, achieving an automated closed loop from collection, mapping, validation to backfilling, shortening processing time and reducing operational risks.

[0078] The following describes in detail the voice data processing method applied to the server side of this application through preferred embodiments.

[0079] In existing remote voice interaction scenarios, key information extracted from the client side often suffers from colloquial expressions, multiple rounds of correction, synonyms, and recognition errors. This leads to multiple candidate values ​​for the same field, unclear field boundaries, or missing required elements. Furthermore, different business requirements have strict stipulations regarding field format, value range, and mandatory field rules. Relying solely on simple type aggregation or rule matching can easily result in candidates that appear semantically reasonable but do not meet business rules, or candidates that meet the format but have incorrect semantic meaning. This instability in the quality of the candidate element set further leads to field mapping deviations and backfilling failures. To address these issues, the embodiments of this application introduce a dual constraint mechanism of semantic verification and rule verification.

[0080] Specifically, the target information fragment includes at least key information fragments and their associated information. The server can first aggregate the target information fragments according to preset information types to form a set of candidate field values. Then, the set of candidate field values ​​is input into a pre-trained large language model for semantic processing to obtain a first semantic verification result and a first error correction suggestion. The first semantic verification result is used to characterize the semantic consistency or conflict between candidate options to identify candidate options with mutually exclusive values, unclear semantic references, or inconsistent contexts under the same field. At the same time, rule information can be obtained from the target business rule base and rule verification can be performed on the set of candidate field values ​​to obtain a first rule verification result, which is used to characterize the degree to which candidate options meet requirements such as format compliance, value range compliance, and fulfillment of mandatory elements. Finally, based on the first semantic verification result, the first error correction suggestion, and the first rule verification result, error correction or elimination processing is performed on the set of candidate field values, and the processed candidate field values ​​are associated with the corresponding associated information to generate a set of candidate elements.

[0081] The above approach enables semantic conflict identification and correction of candidate options before field mapping, and combines business rules to perform format, scope, and mandatory field validation. This achieves dual validation and cleansing of key information, reducing the probability of erroneous candidates entering subsequent field mapping and element validation processes, improving the accuracy and consistency of the candidate element set, thereby reducing the number of missing element replacement rounds and conflict resolution overhead, and enhancing the success rate and stability of structured field generation and backfilling.

[0082] In existing voice-interactive form filling processes, even after candidate elements have been extracted from the dialogue, problems such as unclear field boundaries, mismapping due to similar field semantics, and alignment difficulties caused by differences in field definitions across different business forms are still common. For example, when multiple names appear in the same dialogue, they may correspond to the applicant's name, contact person's name, or payee's name, respectively; when multiple time expressions appear, they may correspond to the occurrence time, application time, or confirmation time, respectively. Relying solely on keyword matching or fixed template mapping can easily lead to field mismatches, missing fields, or redundant fields, thereby affecting the success rate of subsequent validation and data entry.

[0083] To address the aforementioned issues, embodiments of this application provide a field constraint-driven mapping scheme for target forms, enabling field mapping to possess interpretable semantic alignment criteria and executable rule constraints. Specifically, the server can first determine a set of fields based on the target form and generate a semantic representation for each field in the set. This semantic representation can be used to characterize the field name, business meaning, and typical value characteristics. Subsequently, field type rules, format rules, mandatory field rules, and cross-field consistency rules are configured for each field to form field constraint information, clarifying the acceptable data forms, necessities, and logical relationships between fields. Based on this, a mapping score between the candidate field values ​​and their semantic representations can be calculated for the candidate element set based on the field constraint information. The mapping score comprehensively reflects the degree of matching between the candidate value and the field semantics, as well as the degree of satisfaction with the field type and format requirements. When the mapping score meets a preset threshold condition, the candidate field value is mapped to the corresponding field, generating a structured field set.

[0084] The above scheme eliminates the reliance on single keywords or fixed-position rules for field mapping. Instead, it uses the semantic representation of the target form fields as the core for alignment and establishes constraints through type, format, required fields, and cross-field consistency rules. This allows the mapping process to effectively distinguish between semantically similar fields, reducing the probability of incorrect mapping. At the same time, the threshold mechanism can suppress the accidental entry of low-confidence candidates, improve the overall consistency and backfillability of the structured field set, and thus provide a more reliable data foundation for subsequent element verification, missing data completion, and backfilling, thereby improving the success rate and stability of automatic form filling.

[0085] In the embodiments of this application, after the server completes field mapping and obtains a structured field set, it can perform element validation on the structured field set based on field constraint information to determine whether the structured field set meets the filling conditions of the target form, and provide a locatable basis for subsequent supplementary processing or conflict resolution. In this embodiment, element validation is performed in two levels: first element validation and second element validation. The first element validation focuses on the completeness and standardization of the fields themselves, while the second element validation focuses on the logical consistency between fields. Through the hierarchical design of the two-level validation, missing and format-related problems can be quickly identified first without adding unnecessary calculations, and then cross-field contradictions can be further located, thereby improving the efficiency of closed-loop processing.

[0086] Specifically, the first-factor validation can be performed based on mandatory field rules, field type rules, and format rules. For example, the target form contains fields such as transfer amount, recipient account, recipient name, and purpose, where the transfer amount and recipient account are mandatory fields. When the server performs first-factor validation on the structured field set, it first checks whether all mandatory fields have valid field values. If the recipient account field is found to be empty or contains only incomplete fragments, it is added to the missing field set. Then, it checks whether the field value type matches the field type rules. For example, if the transfer amount field contains expressions like "approximately five thousand" that cannot be parsed into a specific numerical value, the transfer amount field is added to the non-compliant field set. Finally, it checks whether the field value meets the format requirements based on the format rules. For example, if the recipient account field does not meet the number of digits requirement or contains abnormal characters, it is also added to the non-compliant field set. By generating the missing field set and the non-compliant field set, the server can clearly distinguish between the two types of problems: "missing fields" and "fields with values ​​but no usability," providing a basis for subsequent targeted supplementation.

[0087] In the embodiments of this application, the second element verification is performed based on cross-field consistency rules to detect logical conflicts between fields. For example, the structured field set may include fields such as document type, document number, date of birth, and contact number. Cross-field consistency rules may include verification items such as document type matching with document number rules and date of birth matching with the date segment in the document number. When the second element verification finds that the document type is a resident ID card but the document number length does not meet the corresponding rule, or that the date of birth parsed from the document number is inconsistent with the date of birth field, the server adds the relevant field pair to the consistency conflict set.

[0088] After generating the sets of missing fields, non-compliant fields, and consistency conflicts, the server can use these sets to generate missing element identification information or as input for conflict detection, thus forming a closed-loop processing chain. Through the above embodiments, hierarchical verification and problem localization of structured field sets can be achieved, making missing data completion and conflict resolution more targeted, thereby improving the success rate of data backfilling and data consistency.

[0089] Figure 3 A flowchart illustrating a voice data processing method applied to a client according to an embodiment of this application is shown schematically.

[0090] like Figure 3 As shown, the voice data processing method 300 includes steps S310 to S340, and the voice data processing method 300 can be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0091] In step S310, voice data during the remote interaction process is collected and converted into text data.

[0092] In embodiments of this application, voice data may originate from telephone calls, terminal device microphones, or other remote interaction channels. The client converts the voice data into text data for subsequent semantic analysis and key information extraction. During the conversion process, the client can segment the voice segments and record the generation order information to ensure that the text data remains consistent with the interaction rounds, facilitating subsequent location and supplementation.

[0093] In step S320, data analysis and extraction are performed based on text data to obtain target information fragments, and the target information fragments are sent to the server so that the server can perform feature extraction and feature verification based on the target information fragments.

[0094] In embodiments of this application, data analysis and extraction based on text data may include key information location and classification processing, such as identifying key segments from text, including names, ID numbers, contact information, amounts, times, locations, and business types, and simultaneously generating associated information to describe the contextual attributes of the key segments in the dialogue. The associated information may include the number of times the key segments appear, their order of appearance, corresponding sentence segments, whether they are statements confirmed by the user, or their co-occurrence relationships with other segments.

[0095] After the client sends the target information fragment, the server can perform feature extraction and feature validation based on the target information fragment, construct a structured field set and determine whether the fields meet the backfill conditions, thereby converting the natural language information collected by the client into structured data that can be used for target form processing.

[0096] In step S330, the missing element identification information sent by the server is obtained, wherein obtaining the missing element identification information indicates that the element verification fails and that a missing element is determined to exist.

[0097] During interaction with the server, the client receives missing element identification information. Receiving this information indicates that the server's element verification failed and that a missing element has been identified. This missing element identification information can indicate the missing field category or name in the current business process, and may also include corresponding format requirements or priority for supplementation. For example, in a bank transfer service scenario, missing element identification information might indicate a missing receiving account or purpose information; in a government service service scenario, it might indicate a missing application type or address information. After receiving this information, the client can determine the scope of content that needs to be supplemented and initiate the supplementation process.

[0098] In step S340, supplementary processing is performed based on the missing element identification information to obtain supplementary target information fragments, and the supplementary target information fragments are sent to the server. The supplementary target information fragments are used to enable the server to update the generated structured field set and drive the backfilling of the target form.

[0099] The supplementary processing may include generating supplementary prompts based on missing fields and collecting supplementary audio or text, then analyzing and extracting the supplementary data to form supplementary target information fragments. These supplementary target information fragments are used to update the generated structured field set on the server side and further drive the backfilling of the target form. If gaps still exist after supplementation, the client can continue to supplement multiple times based on new missing element identifiers until the server determines that the backfilling conditions are met and completes the writing.

[0100] According to the embodiments of this application, through interactive collection, analysis, extraction and supplementary processing mechanisms, remote voice interaction can form a collaborative closed loop with the field mapping, element verification and target system backfilling on the server side, thereby improving the information processing efficiency, field filling accuracy and overall processing stability in remote agency scenarios.

[0101] The following describes in detail, by way of preferred embodiments, the voice data processing method applied to the client in this application.

[0102] In the embodiments of this application, after the client converts voice data into text data, to improve the accuracy and stability of extracting key information from the text data, a dual constraint mechanism of semantic processing and rule verification can be introduced in the data analysis and extraction stages to generate higher-quality target information fragments. Specifically, the client can input the text data into a pre-trained large language model for semantic processing, outputting a second semantic verification result and a second error correction suggestion. The second semantic verification result is used to characterize the semantic coherence and consistency of key information; the second error correction suggestion is used to indicate possible correction directions. Simultaneously, rule information can be obtained from the target business rule base, and rule verification can be performed on the text data based on the rule information to obtain a second rule verification result. The second rule verification result is at least used to characterize the degree to which the key information meets the requirements of format compliance, value range compliance, and mandatory element satisfaction. Furthermore, based on the second semantic verification result, the second error correction suggestion, and the second rule verification result, error correction or elimination processing can be performed on key information: for key information that can be converted into compliant expression through error correction suggestions, the client performs normalization correction and retains it; for key information that is characterized as unreliable or non-compliant by both rule verification and semantic verification, the client eliminates it or marks it as a low-confidence candidate, thereby preventing erroneous information from entering the subsequent processing chain. Finally, the client can organize the error-corrected or filtered key information along with necessary related information into target information fragments and send them to the server, enabling the server to perform element extraction, element verification, and backfilling processing based on more consistent input, thereby reducing subsequent supplementary rounds and improving the overall backfilling success rate.

[0103] In the embodiments of this application, to ensure that the server can correctly associate supplementary information and perform fusion updates according to the generation order during multi-round remote interaction, the client can send supplementary target information fragments to the server using a sending method with session association and sequence identifier. Specifically, the client can first associate the supplementary target information fragment with the current session identifier to generate initial data to be sent. The current session identifier can be used to represent the interaction session corresponding to the same remote task, enabling the server to group the supplementary target information fragment with previously sent target information fragments into the same processing link, avoiding information mixing or misuse between different sessions. Subsequently, the client writes time information into the initial data to be sent to obtain the data to be sent, where the time information is used to represent the generation order information of the supplementary target information fragments during the remote interaction process. By writing time information, the server can distinguish the sequential relationship of the same field in different rounds, making it easier to determine the target field value based on time priority when performing conflict detection or field updates in the future, and to achieve a more robust value decision by combining confidence information when necessary. Finally, the client sends the data to be sent to the server, enabling the server to incrementally update the structured field set according to the generation order of the supplementary information while maintaining session consistency. This reduces field overwriting errors caused by out-of-order reception, duplicate sending, or multiple rounds of correction, and improves the stability and controllability of closed-loop completion and backfilling.

[0104] In the embodiments of this application, after the client receives the missing element identification information sent by the server, in order to improve the targeting of the supplementary interaction and reduce invalid supplementation, the client can perform supplementary processing based on the missing element identification information to obtain the supplementary target information fragment. Specifically, the client parses the missing element identification information to obtain the content to be supplemented and the format constraints corresponding to the content to be supplemented. The content to be supplemented can represent the field category or field name that needs to be filled, and the format constraints can represent length requirements, character composition requirements, numerical range requirements, or date representation requirements, etc. Subsequently, the client generates supplementary inquiry prompts based on the format constraints, so that the supplementary inquiry can clearly guide the user to give input that meets the requirements, and collects supplementary data based on the supplementary inquiry prompts. The supplementary data can be supplementary voice data or supplementary text data. After obtaining the supplementary data, the client first performs input validity verification on the supplementary data to determine whether the supplementary data meets the format constraints or whether it is parsable; if the input validity verification passes, the client then performs data analysis and extraction based on the supplementary data, converts the supplementary data into supplementary target information fragments that can be used for field updates, and sends them to the server to support the server in updating the structured field set and continuing to advance the backfilling process.

[0105] Figure 4 The illustration shows a schematic diagram of a remote online business processing system according to some exemplary embodiments of this application.

[0106] like Figure 4 As shown, the remote online business processing system based on intelligent semantic processing includes a client and a server. The server further includes an intelligent analysis center and a backfilling and anomaly handling link that interfaces with the target system.

[0107] The client is used to carry out front-end data acquisition and preliminary processing capabilities for remote interaction. It includes functional units such as voice acquisition, voice transcription, data analysis, and network communication. It is used to acquire voice data and convert it into text data during remote interaction, extract target information fragments from the text data, and send the target information fragments to the server through network communication.

[0108] The intelligent analysis center is used to carry out structured processing capabilities for target forms, including modules such as field constraint processing, field mapping, and element validation and conflict / missing element handling. Field constraint processing is used to obtain and maintain field constraint information corresponding to the target form. Field constraint information may include field sets, field type rules, format rules, mandatory field rules, and cross-field consistency rules. Field mapping is used to map the field values ​​of candidate elements in the candidate element set to the target field based on the field constraint information, generating a structured field set. Element validation and conflict / missing element handling is used to perform element validation on the structured field set, and when the validation fails, it performs conflict detection and missing element determination, updates the conflict field values ​​according to time priority or confidence priority, and generates missing element identification information to trigger client-side supplementation processing, thus forming a closed-loop completion mechanism.

[0109] The backfilling and exception handling link is used to realize the writing of structured fields to the target system and the feedback of results. It includes information receiving, field matching and backfilling execution. When the structured field set meets the backfilling conditions, a backfilling input request is generated and sent to the target system to complete the form writing. When an exception such as missing fields, non-compliant format or consistency conflict is found, a supplementary request is generated and fed back to the client, guiding the continued collection of supplementary data and updating of the field set until the backfilling conditions are met, realizing a collaborative closed loop of remote interaction, intelligent judgment and writing to the target system.

[0110] Corresponding to the above-described voice data processing method, embodiments of this application also provide a voice data processing apparatus.

[0111] Figure 5A The diagram illustrates a structural block diagram of a voice data processing apparatus applied to a server according to an embodiment of this application.

[0112] like Figure 5AAs shown, the voice data processing device 500 of this embodiment includes a target information fragment acquisition module 501, a field mapping module 502, an element verification module 503, and a backfilling request generation module 504.

[0113] The target information fragment acquisition module 501 can be used to receive target information fragments sent by the client. The target information fragments are obtained by the client through analysis and extraction of data corresponding to remote interactive voice. A candidate element set is generated based on the target information fragments. In one embodiment, the target information fragment acquisition module 501 can be used to perform the operation S210 described above, which will not be repeated here.

[0114] The field mapping module 502 can be used to obtain field constraint information corresponding to the target form, and perform field mapping on the candidate element set based on the field constraint information to obtain a structured field set. In one embodiment, the field mapping module 502 can be used to perform the operation S220 described above, which will not be repeated here.

[0115] The feature validation module 503 can be used to perform feature validation on the structured field set based on field constraint information. If the feature validation fails, it performs conflict detection and / or missing feature determination, generating an updated structured field set. Specifically, if conflict detection indicates conflicting values ​​for the same field, the target field value is determined based on time priority or confidence priority, and the structured field set is updated. If missing features are determined, missing feature identification information is generated, which is used to trigger supplementary processing on the client side. In one embodiment, the feature validation module 503 can be used to perform the operation S230 described above, which will not be repeated here.

[0116] The backfill request generation module 504 can be used to generate a backfill request based on the updated structured fields and send it to the target system. In one embodiment, the backfill request generation module 504 can be used to perform the operation S240 described above, which will not be repeated here.

[0117] According to an embodiment of this application, the target information fragment acquisition module 501 can also be used to aggregate target information fragments according to preset information types to obtain a set of candidate field values; input the set of candidate field values ​​into a pre-trained large language model for semantic processing to obtain a first semantic verification result and a first error correction suggestion, wherein the first semantic verification result is used to characterize the semantic consistency or conflict of candidate field values; acquire rule information in the target business rule base, perform rule verification on the set of candidate field values ​​based on the rule information to obtain a first rule verification result, wherein the first rule verification result is used to characterize one or more of the following: format compliance, value range compliance, and mandatory element satisfaction of candidate field values; perform error correction processing or elimination processing on the set of candidate field values ​​based on the first semantic verification result, the first error correction suggestion, and the first rule verification result; and associate corresponding association information with candidate field values ​​in the processed set of candidate field values ​​to generate a set of candidate elements.

[0118] According to an embodiment of this application, the field mapping module 502 can also be used to determine a field set based on a target form, generate a field semantic representation for each field in the field set, configure field type rules, format rules, mandatory field rules and cross-field consistency rules for each field to obtain field constraint information, and calculate the mapping score between the field value candidates in the candidate element set and the field semantic representation based on the field constraint information. When the mapping score meets a preset threshold condition, the field value candidates are mapped to the corresponding fields to generate a structured field set.

[0119] According to an embodiment of this application, the element verification module 503 can also be used to perform a first element verification on the structured field set based on mandatory rules, field type rules, and format rules. If the first element verification fails, a missing field set and a non-compliant field set are generated. Furthermore, based on cross-field consistency rules, a second element verification is performed on the structured field set. If the second element verification fails, a consistency conflict set is generated. The missing field set, the non-compliant field set, and the consistency conflict set are used to generate missing element identification information or as input for performing conflict detection.

[0120] According to embodiments of this application, any multiple modules among the target information fragment acquisition module 501, field mapping module 502, feature verification module 503, and backfill request generation module 504 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the target information fragment acquisition module 501, field mapping module 502, feature verification module 503, and backfill request generation module 504 can be at least partially implemented as hardware circuitry, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the target information fragment acquisition module 501, field mapping module 502, feature verification module 503, and backfilling request generation module 504 can be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0121] Figure 5B A schematic block diagram of a voice data processing apparatus applied to a client according to an embodiment of this application is shown.

[0122] like Figure 5B As shown, the voice data processing device 510 of this embodiment includes a voice text data acquisition module 511, a target information segment sending module 512, a missing information acquisition module 513, and a supplementary processing module 514.

[0123] The voice-to-text data acquisition module 511 can be used to collect voice data during remote interaction and convert the voice data into text data. In one embodiment, the voice-to-text data acquisition module 511 can be used to perform the operation S310 described above, which will not be repeated here.

[0124] The target information fragment sending module 512 can be used to perform data analysis and extraction based on text data to obtain target information fragments, and send the target information fragments to the server, so that the server can perform feature extraction and feature verification based on the target information fragments. In one embodiment, the target information fragment sending module 512 can be used to perform the operation S320 described above, which will not be repeated here.

[0125] The missing information acquisition module 513 can be used to acquire missing element identification information sent by the server. Acquiring the missing element identification information indicates that the element verification failed and that a missing element is confirmed to exist. In one embodiment, the missing information acquisition module 513 can be used to perform the operation S330 described above, which will not be repeated here.

[0126] The supplementary processing module 514 can be used to perform supplementary processing based on the missing element identification information, obtain supplementary target information fragments, and send the supplementary target information fragments to the server. The supplementary target information fragments are used to enable the server to update the generated structured field set and drive the backfilling of the target form. In one embodiment, the supplementary processing module 514 can be used to perform the operation S340 described above, which will not be repeated here.

[0127] According to an embodiment of this application, the target information fragment sending module 512 can also be used to input text data into a pre-trained large language model for semantic processing to obtain a second semantic verification result and a second error correction suggestion, wherein the second semantic verification result is used to characterize the semantic consistency or conflict of key information; obtain rule information in the target business rule base, perform rule verification on the text data based on the rule information to obtain a second rule verification result, wherein the second rule verification result is used to characterize one or more of the following: format compliance, value range compliance, and fulfillment of required elements of key information; and perform error correction processing or removal processing on key information based on the semantic verification result, the second error correction suggestion, and the second rule verification result to obtain the target information fragment.

[0128] According to an embodiment of this application, the supplementary processing module 514 can also be used to associate the supplementary target information fragment with the current session identifier to generate initial data to be sent; write time information into the initial data to be sent to obtain the data to be sent, wherein the time information is used to characterize the generation order information of the supplementary target information fragment in the remote interaction process; and send the data to be sent to the server.

[0129] According to an embodiment of this application, the supplementary processing module 514 can also be used to parse the missing element identification information to obtain the content to be supplemented and the corresponding format constraints; generate supplementary query prompts based on the format constraints, and collect supplementary data based on the supplementary query prompts; and perform input validity verification on the supplementary data. If the input validity verification passes, perform data analysis and extraction based on the supplementary data to obtain supplementary target information fragments.

[0130] According to embodiments of this application, any multiple modules among the voice-text data acquisition module 511, target information fragment transmission module 512, missing information acquisition module 513, and supplementation processing module 514 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the voice-text data acquisition module 511, target information fragment transmission module 512, missing information acquisition module 513, and supplementation processing module 514 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuits, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations, or in a suitable combination of any of these. Alternatively, at least one of the voice and text data acquisition module 511, the target information fragment sending module 512, the missing information acquisition module 513, and the supplementary processing module 514 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0131] Figure 6 A block diagram schematically illustrates an electronic device suitable for implementing a voice data processing method according to an embodiment of this application.

[0132] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage portion 608 into a random access memory 603. The processor 601 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0133] Random access memory 603 stores various programs and data required for the operation of electronic device 600. Processor 601, read-only memory 602, and random access memory 603 are interconnected via bus 604. Processor 601 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 602 and / or random access memory 603. It should be noted that the programs may also be stored in one or more memories other than read-only memory 602 and random access memory 603. Processor 601 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0134] According to embodiments of this application, the electronic device 600 may further include an input / output interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0135] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0136] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 602 described above, and / or random access memory 603, and / or one or more memories other than read-only memory 602 and random access memory 603.

[0137] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0138] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0139] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0140] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A voice data processing method, applied on a server side, characterized in that, The method includes: Receive a target information fragment sent by a client, the target information fragment being obtained by the client through analysis and extraction of data corresponding to remote interactive voice; generate a candidate element set based on the target information fragment; Obtain the field constraint information corresponding to the target form, and perform field mapping on the candidate element set based on the field constraint information to obtain a structured field set; Based on the field constraint information, the structured field set is validated for elements. If the element validation fails, conflict detection and / or missing element determination are performed, generating an updated structured field set. Specifically, if conflict detection indicates conflicting values ​​for the same field, the target field value is determined based on time priority or confidence priority, and the structured field set is updated. If missing elements are determined, missing element identification information is generated, which is used to trigger supplementary processing on the client side. A backfill request is generated based on the updated structured fields and sent to the target system.

2. The method according to claim 1, characterized in that, The target information fragment includes at least key information fragments and corresponding related information. The step of generating a candidate element set based on the target information fragment includes: The target information fragments are aggregated according to preset information types to obtain a set of candidate field values; The set of field value candidates is input into a pre-trained large language model for semantic processing to obtain a first semantic verification result and a first error correction suggestion. The first semantic verification result is used to characterize the semantic consistency or conflict of the field value candidates. Obtain rule information from the target business rule base, perform rule verification on the field value candidate set based on the rule information, and obtain a first rule verification result, wherein the first rule verification result is used to characterize one or more of the following: format compliance, value range compliance, and mandatory element satisfaction of the field value candidate. Based on the first semantic verification result, the first error correction suggestion, and the first rule verification result, the candidate set of field values ​​is subjected to error correction or elimination processing; and Associate the corresponding association information with the field value candidates in the processed field value candidate set to generate the candidate element set.

3. The method according to claim 1, characterized in that, The step of obtaining field constraint information corresponding to the target form, and performing field mapping on the candidate element set based on the field constraint information to obtain a structured field set includes: Based on the target form, a set of fields is determined, and a semantic representation of each field in the set of fields is generated. Configure field type rules, format rules, required field rules, and cross-field consistency rules for each field to obtain the field constraint information; and Based on the field constraint information, a mapping score between the field value candidates in the candidate element set and the field semantic representation is calculated. When the mapping score meets a preset threshold condition, the field value candidates are mapped to the corresponding fields to generate the structured field set.

4. The method according to claim 3, characterized in that, The step of performing feature validation on the structured field set based on the field constraint information includes: Based on the required field rules, the field type rules, and the format rules, a first element validation is performed on the structured field set. If the first element validation fails, a missing field set and a non-compliant field set are generated; and Based on the cross-field consistency rules, a second element check is performed on the structured field set. If the second element check fails, a consistency conflict set is generated. The missing field set, the non-compliant field set, and the consistency conflict set are used to generate the missing element identification information or as input for performing conflict detection.

5. A voice data processing method, applied to a client, characterized in that, The method includes: Collect voice data during remote interaction and convert the voice data into text data; Data analysis and extraction are performed on the text data to obtain target information fragments, and the target information fragments are sent to the server so that the server can perform element extraction and element verification based on the target information fragments. Obtain missing element identification information sent by the server, wherein obtaining the missing element identification information indicates that the element verification failed and that a missing element is confirmed to exist; and Based on the missing element identification information, supplementary processing is performed to obtain supplementary target information fragments, and the supplementary target information fragments are sent to the server. The supplementary target information fragments are used to enable the server to update the generated structured field set and drive the backfilling of the target form.

6. The method according to claim 5, characterized in that, The process of analyzing and extracting data based on the text data to obtain the target information fragment includes: The text data is input into a pre-trained large language model for semantic processing to obtain a second semantic verification result and a second error correction suggestion. The second semantic verification result is used to characterize the semantic consistency or conflict of key information. Obtain rule information from the target business rule base, perform rule validation on the text data based on the rule information, and obtain a second rule validation result. The second rule validation result is used to characterize one or more of the following: format compliance, value range compliance, and fulfillment of required elements for the key information. Based on the semantic verification result, the second error correction suggestion, and the second rule verification result, error correction processing or removal processing is performed on the key information to obtain the target information fragment.

7. The method according to claim 5, characterized in that, Sending the supplementary target information fragment to the server includes: Associate the supplementary target information fragment with the current session identifier to generate initial data to be sent; Time information is written into the initial data to be sent to obtain the data to be sent, wherein the time information is used to characterize the generation order information of the supplementary target information fragment during the remote interaction process; and The data to be sent is sent to the server.

8. The method according to claim 5, characterized in that, The step of performing supplementation processing based on the missing element identification information to obtain supplemented target information fragments includes: Parse the missing element identification information to obtain the content to be supplemented and the corresponding format constraints; Based on the format constraints, supplementary query prompts are generated, and supplementary data is collected based on the supplementary query prompts; and The supplementary data is subjected to input validity verification. If the input validity verification passes, data analysis and extraction are performed based on the supplementary data to obtain the supplementary target information fragment.

9. A voice data processing device, applied to a server, characterized in that, The device includes: The target information fragment acquisition module is used to: receive a target information fragment sent by a client, wherein the target information fragment is obtained by the client through analysis and extraction of data corresponding to remote interactive voice; and generate a candidate element set based on the target information fragment. The field mapping module is used to: obtain field constraint information corresponding to the target form, and perform field mapping on the candidate element set based on the field constraint information to obtain a structured field set; The feature validation module is used to: perform feature validation on the structured field set based on the field constraint information; if feature validation fails, perform conflict detection and / or missing feature determination; and generate an updated structured field set. Specifically, if conflict detection indicates conflicting values ​​for the same field, the target field value is determined based on time priority or confidence priority, and the structured field set is updated. If missing features are determined, missing feature identification information is generated, which triggers supplementary processing on the client side. The backfill request generation module is used to: generate a backfill request based on the updated structured fields and send it to the target system.

10. A voice data processing device, applied to a client, characterized in that, The device includes: The voice and text data acquisition module is used to: collect voice data during remote interaction and convert the voice data into text data; The target information fragment sending module is used to: perform data analysis and extraction based on the text data to obtain target information fragments, and send the target information fragments to the server, so that the server can perform element extraction and element verification based on the target information fragments; The missing element information acquisition module is configured to: acquire missing element identification information sent by the server, wherein the acquisition of the missing element identification information indicates that the element verification failed and that a missing element is confirmed to exist; and The supplementary processing module is used to: perform supplementary processing based on the missing element identification information to obtain supplementary target information fragments, and send the supplementary target information fragments to the server. The supplementary target information fragments are used to enable the server to update the generated structured field set and drive the backfilling of the target form.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images