Task instruction generation method and device, storage medium and electronic device

Abstract

The application discloses a task instruction generation method and device, a storage medium and an electronic device, relates to the technical field of voice conversation, and comprises the following steps: receiving a voice task execution request; in the case that a task assignment key field and an object indication key field are identified in a voice task text converted from target voice data, determining a target object according to the respective associated confidence of each conversation object in a plurality of conversation objects; replacing the object indication key field in the voice task text with an object identifier of the target object to obtain a target text; and generating a task instruction of a target task based on the target text, thereby solving the technical problem that the accuracy of task instruction generation is low.

Description

Technical Field

[0001] This application relates to the field of voice conversation technology, and in particular to methods, apparatus, storage media and electronic devices for generating task instructions. Background Technology

[0002] In instant messaging and intelligent collaboration systems, users often initiate task assignments via voice. Existing technologies typically use traditional automatic speech recognition technology to transcribe speech signals into text, and then identify the execution object through simple string matching. However, this method cannot accurately determine the target object when there is a referent, and therefore cannot automatically generate the correct task execution instructions. In other words, existing technologies have the problem of low accuracy in generating task instructions. Summary of the Invention

[0003] This application provides a method, apparatus, storage medium, and electronic device for generating task instructions, in order to at least solve the problem of low accuracy in task instruction generation in related technologies.

[0004] This application provides a task instruction generation method, comprising: receiving a voice task execution request, wherein the voice task execution request includes target voice data in a voice session and object identifiers of multiple session objects participating in the voice session; identifying a task assignment key field and an object indication key field in the voice task text converted from the target voice data, and determining a target object based on the association confidence of each of the multiple session objects, wherein the object indication key field is used to indicate the object executing the target task indicated by the task assignment key field, and the association confidence is determined based on the historical interaction records of the session object in the voice session, and is used to indicate the degree of association between the session object and the target task; replacing the object indication key field in the voice task text with the object identifier of the target object to obtain target text; and generating a task instruction for the target task based on the target text.

[0005] This application also provides a task instruction generation apparatus, comprising: a receiving unit for receiving a voice task execution request, wherein the voice task execution request includes target voice data in a voice session and object identifiers of multiple session objects participating in the voice session; a recognition unit for identifying a task assignment key field and an object indication key field in a voice task text converted from the target voice data, and determining a target object based on the association confidence of each of the multiple session objects, wherein the object indication key field is used to indicate the object executing the target task indicated by the task assignment key field, and the association confidence is determined based on the historical interaction records of the session object in the voice session, and is used to indicate the degree of association between the session object and the target task; a replacement unit for replacing the object indication key field in the voice task text with the object identifier of the target object to obtain target text; and a generation unit for generating a task instruction for the target task based on the target text.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described task instruction generation methods when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described task instruction generation methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described task instruction generation methods.

[0009] The embodiments provided in this application obtain a set of session objects containing historical interaction records, and when task assignment words and indicator pronouns are detected, the association confidence is calculated based on the historical interaction records of each session object to determine the target object. Then, the indicator pronouns are rewritten to generate the target text, so that the target object can be accurately determined even when there is an indicator object, thereby improving the accuracy of task instruction generation and solving the technical problem of low accuracy of task instruction generation. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a hardware structure block diagram of a server device provided in an embodiment of this application;

[0012] Figure 2 This is a schematic diagram of the flow of an optional task instruction generation method according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an optional task instruction generation method according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram of another optional task instruction generation method according to an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of another optional task instruction generation method according to an embodiment of this application;

[0016] Figure 6 This is a schematic diagram of an optional task instruction generation apparatus according to an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0018] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0019] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device according to an embodiment of this application. Figure 1 As shown, the server device may include one or more ( Figure 1Only one is shown in the image. A processor 102 (which may include, but is not limited to, a central processing unit (CPU), microprocessor (MCU), or programmable logic device (FPGA), etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0021] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data storage method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0023] Example 1:

[0024] Embodiments of this application provide a method for generating task instructions, such as... Figure 2 As shown, it specifically includes:

[0025] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the task instruction generation method can be executed by an electronic device, such as... Figure 1 The user equipment or server shown includes the following specific steps:

[0026] S202, Receive a voice task execution request, wherein the voice task execution request includes the target voice data in the voice session and the object identifiers of multiple session objects participating in the voice session;

[0027] S204, In the case that the task assignment key field and the object indication key field are identified in the speech task text converted from the target speech data, the target object is determined according to the association confidence of each of the multiple conversation objects. The object indication key field is used to indicate the object that performs the target task indicated by the task assignment key field, and the association confidence is determined based on the historical interaction records of the conversation object in the speech conversation and is used to indicate the degree of association between the conversation object and the target task.

[0028] S206, replace the object indicator key field in the speech task text with the object identifier of the target object to obtain the target text;

[0029] S208, Generate task instructions for the target task based on the target text.

[0030] Optionally, in this embodiment, the voice task execution request can be voice data containing a specific task intent initiated by the user through a voice input interface, or it can be automatically collected voice data containing imperative meaning. The voice task execution request may include the raw audio signal to be processed, as well as the identity information of all participants in the current session environment.

[0031] Optionally, in this embodiment, the target speech data can be the original audio signal generated when the user initiates a task command or a feature representation after preliminary processing. For example, the target speech data can be an audio waveform containing an imperative sentence, which is converted into a feature vector that can be processed by a neural network after acoustic encoding, and can represent the words or meanings contained in the speech data.

[0032] Optionally, in this embodiment, a voice session can be an interactive process based on voice signals, involving multiple participants. It is understood that voice sessions are continuous and context-dependent, allowing participants to exchange information and collaborate on tasks via voice.

[0033] Optionally, in this embodiment, multiple session objects can be all or some entities participating in the current voice session, including real users, intelligent agents, or other interactive software entities. Each session object has a unique identity feature, such as an object identifier. Furthermore, session objects can also have other identifiers that distinguish them from the object identifier, which can also help identify the identity of the session object. It should be noted that while the object identifier of a session object is unique, other identifiers are not unique; other session objects can still use the same other identifier. For further example, suppose user A's object identifier is XX, and user A's other identifier is pear, while user B's object identifier is YY. In this case, user B's other identifier can also be pear. In this situation, user A and user B's other identifiers can be the same. In addition, non-user objects can also use the same other identifier; for example, intelligent agent A's other identifier can also be pear as mentioned above.

[0034] Optionally, in this embodiment, the object identifier can be a string or numerical code used to uniquely distinguish each session object, and the object identifier can accurately point to a specific user or intelligent agent.

[0035] Optionally, in this embodiment, the voice task text can be a text record converted from the target voice data by voice recognition technology, and the voice task text can be used to retain the semantic content in the user's voice.

[0036] Optionally, in this embodiment, the task assignment key field can be a word, phrase or text fragment in the voice task text that indicates the intention to assign a task. The task assignment key field can be expressed as an imperative verb or a request sentence. For example, the task assignment key field can include words such as let, call, help, handle, etc.

[0037] Optionally, in this embodiment, the object indicator key field may be a word, phrase or text fragment in the voice task text used to point to the performer. The object indicator key field may include an explicit name, nickname, pronoun or category word.

[0038] Optionally, in this embodiment, the association confidence score can be a quantitative value that measures the degree of matching between each conversation object and the current target task. The association confidence score is dynamically calculated based on the historical interaction records of the conversation object in the voice conversation, enabling the selection of the object most likely to be referred to by the user's intent from multiple conversation objects. The higher the association confidence score, the more frequently the object has performed similar tasks in the past or the stronger the interaction habit with the user. Therefore, objects with high confidence scores are preferentially selected as target objects, thereby improving the accuracy of task assignment.

[0039] Optionally, in this embodiment, the historical interaction record can be the interaction data accumulated by the current session object in past voice sessions or task executions, which may include information such as task type, execution frequency, response speed, and user feedback. By analyzing the historical behavior patterns of each session object, the target object can be inferred. For example, if object A has successfully executed data query tasks multiple times in the past, its association confidence when encountering similar tasks in the present will be increased accordingly, and vice versa. As another example, assuming that object B participated in communication or interaction with the account that initiated the voice task execution request before the voice task execution request was issued, the confidence of object B can be set higher, or object B can be set as the target object.

[0040] Optionally, in this embodiment, the target task can be a specific action or operation that needs to be performed, as determined by the task assignment key field. The target task can have a clear functional definition and execution requirements, such as organizing files, checking the weather, or sending messages.

[0041] Optionally, in this embodiment, the target text can be standardized text obtained by replacing the object indicator key field in the voice task text with the object identifier of the target object. For example, the target text could be "Please ask UID_001 to check the data".

[0042] Optionally, in this embodiment, the task instruction can be structured data generated based on the target text and containing complete execution elements. The task instruction can be used to define the executor, execution action, and execution parameters.

[0043] Optionally, in this embodiment, task assignment key fields and object indication key fields are identified from the voice task text converted from the target voice data, thereby filtering out verbs that indicate task intent and nouns or pronouns pointing to the executor. Then, based on the association confidence of each of the multiple conversation objects, the target object is determined, and the object identifier of the target object replaces the object indication key field to obtain the target text. Finally, task instructions are generated based on the target text.

[0044] To illustrate further, suppose we receive a request containing the voice data along with the identifiers of user A, colleague B, and agent C. The voice is converted to text, such as "Have that assistant check the meeting minutes." We identify "have" as the key field for task assignment and "that assistant" as the key field for object indication. A search of historical interaction records reveals that user A has repeatedly used the word "assistant" to refer to agent C, and agent C has query capabilities. Therefore, agent C has the highest association confidence. Thus, "that assistant" is replaced with agent C's object identifier, generating the target text "Please have agent C check the meeting minutes," ultimately generating a query task instruction for agent C.

[0045] It should be noted that by receiving a request containing voice data and conversation object identifiers, after identifying the key fields of task assignment and object indication, the association confidence of each object is calculated based on historical interaction records to determine the target object, and the pronouns in the text are replaced with specific identifiers to generate target text. Finally, task instructions are generated based on this, thereby achieving accurate parsing of pronoun fields in complex voice conversation scenarios and converting natural language into executable structured instructions.

[0046] The embodiments provided in this application obtain a set of session objects containing historical interaction records, and when task assignment words and indicator pronouns are detected, the association confidence is calculated based on the historical interaction records of each session object to determine the target object. Then, the indicator pronouns are rewritten to generate the target text, so that the target object can be accurately determined even when there is an indicator object, thereby improving the accuracy of task instruction generation and solving the technical problem of low accuracy of task instruction generation.

[0047] As an optional approach, before determining the target object based on the association confidence of each session object in the set of session objects, the following steps are taken:

[0048] Obtain multiple candidate constraints, where the constraints are used to determine the candidate object identifier;

[0049] Iterate through multiple constraints and perform the following operations:

[0050] If the historical interaction record or session configuration of a voice session matches the i-th constraint, at least one candidate object identifier is determined from the historical interaction record or session configuration according to the i-th constraint, where i is a natural number greater than or equal to 1, and the priority of the i-th constraint is greater than the priority of the (i+1)-th constraint.

[0051] Determine the candidate type for each candidate object identifier in at least one candidate object identifier;

[0052] The association confidence of each session object is determined based on at least one candidate object identifier and the candidate type of each candidate object identifier.

[0053] Optionally, in this embodiment, the constraint can be a condition used to determine the candidate object identifier of the target task. The constraint can include constraints on the history of the voice session and / or session configuration constraints. Different constraints have different priorities. When obtaining the candidate object identifier, the constraint with higher priority is executed first.

[0054] To further illustrate, when a user says "Can you check this for me?" or "Can you organize this?", if the session configuration of the voice conversation indicates that there is a currently awakened or responding agent, it can be determined that the first constraint condition is met, and "you" can be identified as this agent. At this time, the object identifier of this agent or other identifiers mentioned above can be candidate object identifiers. It is understood that the first constraint condition is the constraint condition with the highest priority.

[0055] If the first constraint is not met, the process continues to traverse downwards according to priority. If the user is still saying "Can you check for me?" or "Can you organize it?", and there are multiple intelligent agents or multiple real people having a continuous conversation, it can be determined that the first constraint is met at this time, and "you" is identified as the object with the closest interaction time and still valid conversation state.

[0056] Optionally, in this embodiment, the candidate object identifier can be the identifier of a session object that meets the constraints and has the potential to perform a task. For example, the candidate object identifier can be the dialogue identifier of user A, such as XX, or it can be other identifiers of user A, such as pear.

[0057] Optionally, in this embodiment, the candidate type can be a category label based on the session object attribute to which the candidate object identifier belongs, used to distinguish execution entities of different natures, such as human users, intelligent agents, groups, or devices. Candidate types may include human type, intelligent agent type, or device type, etc.

[0058] Optionally, in this embodiment, the session configuration and historical interaction information can also be understood as the context information of the voice session. The context information can be a comprehensive dataset used to describe the current voice session environment state and historical behavioral characteristics. The context information can cover the preset parameter rules in the session configuration and the past behavioral data accumulated in the historical interaction records. For example, the context information can include the default agent settings of the current session, the historical interaction frequency between the user and different objects, the referential habits of specific terms in the session, and the real-time online status of the objects.

[0059] Optionally, in this embodiment, multiple candidate constraints are obtained, wherein the constraints are used to determine candidate object identifiers; the multiple constraints are traversed, and the following operations are performed: if the historical interaction record or session configuration of the voice session matches the i-th constraint, at least one candidate object identifier is determined from the historical interaction record or session configuration according to the i-th constraint, wherein i is a natural number greater than or equal to 1, and the priority of the i-th constraint is greater than the priority of the (i+1)-th constraint; the candidate type of each candidate object identifier among the at least one candidate object identifier is determined; and the association confidence of each session object is determined according to the at least one candidate object identifier and the candidate type of each candidate object identifier.

[0060] To further illustrate, suppose a user says "Have that assistant check the meeting minutes" in a voice conversation with colleague B and agent C. First, candidate constraints are obtained, with the highest priority constraint being "currently active agent" and the second highest priority being "recent interaction records". Traversing the constraints, agent C is found to be the currently active agent, and its type is "agent", which matches the semantics of the key field indicating the "assistant" object. Then, based on the default weight of the "agent" type in the conversation configuration and the high-frequency interaction data of user C in the recent interaction records, the association confidence of agent C is calculated to be 0.95, while the association confidence of colleague B, whose type is "human user", is calculated to be 0.3. Therefore, agent C is determined to be the target object.

[0061] The embodiments provided in this application obtain multiple candidate constraints with priorities, traverse and determine the candidate object identifiers and their types based on historical interaction records or session configurations, and then calculate the association confidence of each session object based on the identifier and type. This achieves structured rule filtering and quantitative evaluation before determining the target object, thereby improving matching accuracy and thus enhancing the accuracy of task instruction generation.

[0062] As an optional approach, the association confidence of each session object is determined based on at least one candidate object identifier and the candidate type of each candidate object identifier, including:

[0063] Iterate through at least one candidate object identifier and multiple session objects, and perform the following operations:

[0064] If the j-th candidate object identifier matches the object identifier of the k-th session object, the first identifier coefficient is determined as the identifier matching score of the k-th session object, where j and k are natural numbers greater than or equal to 1;

[0065] If the j-th candidate object identifier matches other identifiers of the k-th session object, the second identifier coefficient is determined as the identifier matching score of the k-th session object;

[0066] If the candidate type of the j-th candidate object matches the object type of the k-th session object, the type coefficient is determined as the type matching score of the k-th session object.

[0067] The association confidence of the k-th session object is obtained by weighted summation of the identifier matching score and the type matching score. The association confidence of the k-th session object is used to indicate the probability that the k-th session object is the object indicated by the j-th candidate object identifier.

[0068] Optionally, in this embodiment, the identifier matching score can be a quantitative value used to measure the degree of direct matching between the candidate object identifier and the object identifier of the session object, reflecting whether the candidate object identifier corresponds to the object identifier of the session object or other identifiers. For example, if the candidate object identifier directly matches the object identifier of a certain session object, such as the candidate object identifier being XX and the object identifier of session object A also being XX, then the matching score for user A can be 1; if the candidate object identifier is a peach and the object identifier of session object A is also a peach, then the identifier matching score for user A can be 0.8; if neither matches, then the identifier matching score for user A can be 0.

[0069] Optionally, in this embodiment, the first identification coefficient may be a matching score value assigned when the candidate object identifier is completely consistent with the object identifier of the target session object.

[0070] Optionally, in this embodiment, other identifiers may be other identity information possessed by the target session object besides the object identifier, such as nickname, alias, abbreviation, or auxiliary ID. This information is used to expand the identification scope of the object. For example, the nickname of session object A is "Alice", and other identifiers include "AI assistant". If the candidate object identifier is "AI assistant", then session object A is matched through other identifiers.

[0071] Optionally, in this embodiment, the second identifier coefficient can be a matching score value assigned when the candidate object identifier matches other identifiers of the target session object. The second identifier coefficient is lower than the first identifier coefficient, indicating indirect matching or alias matching.

[0072] Optionally, in this embodiment, the object type can be an inherent category attribute of the target session object. The object type defines the application scenario and functional role of the object in the system. For example, the object type can be "human" or "intelligent agent". If the candidate type is "intelligent agent" and the object type is also "intelligent agent", the type matching score is higher.

[0073] Optionally, in this embodiment, the type matching score can be a quantitative value that measures the degree of matching between the candidate type and the object type of the session object, reflecting the consistency between the candidate object type indicated by the candidate object identifier and the actual type of the target session object.

[0074] Optionally, in this embodiment, the type coefficient can be a score value assigned when the candidate type matches the object type of the session object, used to quantify the degree of type matching and provide a specific score value for the matching of the type dimension, so as to combine it with the identifier matching score in the weighted summation. For example, the type coefficient can be set to 0.6, 0.8 or 1 to indicate that the types are consistent, thereby reflecting the fit between the object type and the candidate intent in the association confidence calculation.

[0075] Optionally, in this embodiment, during the weighted summation of the identifier matching score and the type matching score, the weights corresponding to the identifier matching score and the type matching score can be determined according to the actual situation. If the object identifier is more important, the weight of the identifier matching score can be set to 0.7, and the weight of the type matching score can be set to 0.3.

[0076] Optionally, in this embodiment, the following operations are performed: if the j-th candidate object identifier matches the object identifier of the k-th session object, a first identifier coefficient is determined as the identifier matching score of the k-th session object, where j and k are natural numbers greater than or equal to 1; if the j-th candidate object identifier matches other identifiers of the k-th session object, a second identifier coefficient is determined as the identifier matching score of the k-th session object; if the candidate type of the j-th candidate object identifier matches the object type of the k-th session object, a type coefficient is determined as the type matching score of the k-th session object; the identifier matching score and the type matching score are weighted and summed to obtain the association confidence of the k-th session object, where the association confidence of the k-th session object is used to indicate the probability that the k-th session object is the object indicated by the j-th candidate object identifier.

[0077] To further illustrate, suppose the candidate object is identified as "Assistant" and the candidate type is "Agent". The system iterates through the session objects. For Agent C, its object identifier is "Agent_C", which matches other identifiers "Assistant". Therefore, it obtains a second identifier coefficient as the identifier matching score. At the same time, its object type is "Agent", which matches the candidate type "Agent". Therefore, it obtains a type coefficient as the type matching score. The association confidence of Agent C is obtained by weighted summing of these two scores. However, for colleague B, its object identifier is "User_B", and other identifiers are "colleague". It does not match the candidate identifier "Assistant", so the identifier matching score is 0. Its object type is "Human", which does not match the candidate type "Agent", so the type matching score is 0.

[0078] The embodiments provided in this application traverse candidate object identifiers and multiple session objects, determine the corresponding identifier matching score and type matching score in the cases of direct identifier matching, alias matching and type matching respectively, and obtain the association confidence by weighted summation of the two, thereby realizing the quantitative evaluation of the matching degree between session objects and candidate identifiers through multi-dimensional feature fusion, and improving the accuracy of task object identification.

[0079] As an optional approach, the target object is determined based on the association confidence of each session object in the session object set, including:

[0080] Determine the set of association confidence scores that match the j-th candidate object identifier, wherein the set of association confidence scores includes multiple association confidence scores, and the number of association confidence scores in the set of association confidence scores is the same as the number of multiple session objects;

[0081] If the difference between the first association confidence and the second association confidence in the association confidence set is greater than or equal to the first preset threshold, and the first association confidence is greater than the second preset threshold, the session object corresponding to the first association confidence is determined as the target object. Here, the first association confidence is the association confidence with the highest value in the association confidence set, and the second association confidence is the association confidence with the second highest value in the association confidence set.

[0082] If the difference between the first association confidence and the second association confidence in the association confidence set is less than the first preset threshold and the first association confidence is greater than the second preset threshold, a prompt message is sent to the user account. The prompt message includes the session objects corresponding to the first association confidence and the second association confidence respectively. The prompt message is used to prompt the user account to select the target object. The task instruction generation request is sent by the user account.

[0083] Optionally, in this embodiment, the association confidence set may be a set containing all session object association confidence values ​​calculated for the j-th candidate object identifier, where each value in the association confidence set corresponds to a specific session object.

[0084] Optionally, in this embodiment, the first association confidence score in the association confidence score set can be the association confidence score value with the largest value in the association confidence score set, representing the session object with the highest matching degree with the current candidate object identifier among all session objects.

[0085] Optionally, in this embodiment, the second association confidence score can be the association confidence score value in the association confidence score set that is second only to the first association confidence score, representing the session object with the second highest matching degree with the current candidate object identifier among all session objects.

[0086] Optionally, in this embodiment, the first preset threshold can be a numerical limit used to determine whether the difference between the first association confidence level and the second association confidence level is large enough. Automatic matching is only performed when the best choice is significantly better than the second-best choice, thereby reducing errors caused by the similar matching degree of the two candidate objects.

[0087] Optionally, in this embodiment, the second preset threshold may be used to determine whether the highest association confidence level itself has reached a sufficiently high numerical limit, so as to set the minimum confidence threshold for automatically confirming the target object, so that even if the difference is significant, automatic matching will not be performed when the overall matching degree is not high, thereby preventing the task from being forcibly assigned when the overall recognition effect is poor.

[0088] Optionally, in this embodiment, the user account can be a user identity identifier or account entity that initiates the voice task execution request. It can receive prompt information and make manual interventions. As the final decision-maker when automatic decision-making cannot identify, it ensures that when the target object cannot be determined autonomously, the user can manually specify the executor, thereby ensuring the accuracy of task assignment.

[0089] Optionally, in this embodiment, the prompt information may be interactive data or a message sent to the user account that contains information about the candidate target object, informing the user that there are currently multiple high-confidence candidate objects and requesting the user to manually confirm or select one.

[0090] Optionally, in this embodiment, a set of association confidence scores matching the j-th candidate object identifier is determined, wherein the set of association confidence scores includes multiple association confidence scores, and the number of association confidence scores in the set is the same as the number of multiple session objects; if the difference between the first association confidence score and the second association confidence score in the set of association confidence scores is greater than or equal to a first preset threshold, and the first association confidence score is greater than a second preset threshold, the session object corresponding to the first association confidence score is determined as the target object, wherein the first association confidence score is the highest value association confidence score in the set of association confidence scores, and the second association confidence score is the second highest value association confidence score in the set of association confidence scores; if the difference between the first association confidence score and the second association confidence score in the set of association confidence scores is less than the first preset threshold, and the first association confidence score is greater than the second preset threshold, a prompt message is sent to the user account, wherein the prompt message includes the session objects corresponding to the first association confidence score and the second association confidence score, respectively, and the prompt message is used to prompt the user account to select the target object, and the task instruction generation request is issued by the user account.

[0091] To further illustrate, suppose the calculated association confidence score for agent C is 0.85, for colleague B it is 0.70, and for conversation object D it is 0.10, with the association confidence score set being [0.85, 0.70, 0.10]. The first association confidence score is 0.85, the second association confidence score is 0.70, and the difference is 0.15. If the first preset threshold is 0.10 and the second preset threshold is 0.70, since 0.15 is greater than 0.10 and 0.85 is greater than 0.70, the system automatically determines agent C as the target object and generates a task instruction. If the association confidence score for colleague B is 0.82, the difference is 0.03, which is less than 0.10, but 0.85 is greater than 0.70, the system sends a prompt message to the user account, listing agent C and colleague B, and the user selects agent C.

[0092] The embodiments provided in this application determine the set of associated confidence levels and compare the difference and absolute value between the highest and second-highest confidence levels. When the difference is significant and the confidence level meets the standard, the target object is automatically determined. When the difference is insufficient but the confidence level meets the standard, the user is given a prompt message containing candidate objects for selection. This reduces the user's operations in unambiguous scenarios and avoids task misassignment through manual confirmation when there is ambiguity.

[0093] As an optional approach, before determining the target object based on the association confidence of each session object in the set of session objects, the following steps are taken:

[0094] Iterate through each session object in the collection of session objects and perform the following operations:

[0095] Based on the matching status between the object type of the nth session object and the task assignment term, the semantic matching score of the nth session object is determined, where n is a natural number greater than or equal to 1;

[0096] The activity score of the nth session object is determined based on the historical interaction records of the nth session object;

[0097] Based on the historical selection record of the nth session object, determine the historical selection score of the nth session object, where the historical selection score is used to indicate the number of times the nth session object has been identified as the target object;

[0098] The association confidence of the nth session object is obtained by weighted summation of semantic matching score, activity score and historical selection score.

[0099] Optionally, in this embodiment, the semantic matching score can be a quantitative value calculated based on the degree of semantic association between the object type of the nth session object and the task assignment word. It can reflect the degree of fit between the functional attributes of the object and the current task intent. For example, if the task assignment word is "search for data" and the object type is "database agent", the semantic matching score is high because the two have a high degree of functional matching.

[0100] Optionally, in this embodiment, the activity score can be a quantitative value calculated based on the historical interaction records of the nth conversation object, which can reflect the activity level and responsiveness of the object in the current or recent conversation. For example, if an agent frequently participates in dialogue or processes tasks in the last few minutes, the activity score is high.

[0101] Optionally, in this embodiment, the historical selection score can be a quantitative value calculated based on the historical selection records of the nth session object. This value indicates the number of times the nth session object has been identified as a target by the system or user in the past. It can utilize historical preference data to predict the likelihood of an object being selected again, thus replicating the user's habitual selection tendencies. For example, if object A has been selected ten times in similar task scenarios in the past, while object B has only been selected once, then object A has a higher historical selection score.

[0102] Optionally, the identifier matching score and type matching score mentioned above, as well as the semantic matching score, activity score and historical selection score mentioned in this embodiment, can be combined and weighted to obtain the final association confidence of the session object. The weights corresponding to each score can be dynamically configured according to the actual situation.

[0103] It is understood that, in this embodiment, the association confidence of the session object can be calculated using only the identifier matching score and the type matching score, or it can be calculated by weighted summation using only the semantic matching score, the activity score, or the historical selection score; or the final association confidence can be obtained by weighted summation of the identifier matching score, the type matching score, the semantic matching score, the activity score, or the historical selection score.

[0104] To further illustrate, suppose a user says, "Have that assistant check the weather." The system iterates through the conversation objects. For agent A, the object type "weather query assistant" highly matches the task assignment phrase "check the weather," with a semantic matching score of 0.9; it responded to multiple queries in the past hour, with an activity score of 0.8; and it was selected as the target object 5 times in the past week, with a historical selection score of 0.7. The association confidence of agent A is obtained by weighting and summing these three scores. For agent B, the type is "news assistant," with a lower semantic matching score of 0.2, an activity score of 0.5, and a historical selection score of 0.1. The association confidence of agent B is calculated to be lower, and agent A is ultimately determined as the target object.

[0105] The embodiments provided in this application traverse the set of session objects, determine the semantic matching score based on the matching status of the object type and the task assignment word, determine the activity score based on the historical interaction record, determine the historical selection score based on the historical selection record, and obtain the association confidence score by weighted summation of the three. This realizes the evaluation of the task execution capability of the session object from three dimensions: functional adaptability, real-time response capability, and user historical preferences, avoiding the bias that may be caused by single-dimensional evaluation and improving the accuracy of task instruction generation.

[0106] As an optional approach, after receiving the task instruction generation request, the following steps are included:

[0107] The target speech data, object identifiers corresponding to multiple conversation objects, and the context feature set of the speech conversation are input into the trained task instruction generation model to obtain the task instruction. The context feature set is used to indicate the historical interaction records of the conversation objects in the speech conversation.

[0108] Optionally, in this embodiment, the task instruction generation model can be a machine learning algorithm or neural network architecture trained on a specific dataset that can directly output structured task instructions based on the multimodal features of the input. The task instruction generation model can include deep network layers that can process speech sequences, object identifiers, and contextual features.

[0109] Optionally, in this embodiment, the context feature set can be a comprehensive data vector or matrix used to characterize the current voice conversation environment state and historical behavioral features. The context feature set can be generated by encoding the historical interaction records of the conversation objects in the voice conversation, which can provide rich contextual semantic information for the task instruction generation model, thereby more accurately inferring the target object and task details. For example, the context feature set can include the interaction frequency between user A and agent B in the past week, the type of the most recent task, the user's commonly used instruction format, etc.

[0110] Through the embodiments provided in this application, after receiving a task instruction generation request, the target speech data, the set of conversation objects, and the set of contextual features reflecting historical interaction records are fed into the trained task instruction generation model as joint inputs. This enables the end-to-end fusion of speech semantics, object identity, and historical context information using a deep learning model, and automatically completes the conversion from speech reference to precise structured task instructions.

[0111] As an optional approach, before receiving the task instruction generation request, the following steps are included:

[0112] Obtain the speech training data set;

[0113] The speech training dataset is input into the initial task instruction generation model to obtain text recognition loss, task assignment triggering classification loss, segment start and end position loss, object type classification loss and candidate object ranking loss.

[0114] The target loss is obtained by weighted summing of the text recognition loss, task assignment triggering classification loss, fragment start and end position loss, object type classification loss, and candidate object ranking loss.

[0115] The initial task instruction generation model is iteratively trained based on the target loss until the target loss meets the preset convergence condition, thus obtaining the trained task instruction generation model.

[0116] Optionally, in this embodiment, the speech training dataset can be a large-scale labeled dataset used to supervise and optimize the task instruction generation model. The speech training dataset can consist of a large number of paired samples, each sample containing target speech data, corresponding speech task text, task assignment label, start and end positions of the referential segment, target object identifier, object type, and correct candidate object sorting order.

[0117] Optionally, in this embodiment, the text recognition loss can be a quantitative indicator that measures the difference between the speech task text generated by the model and the real labeled text. The text recognition loss can be calculated using the cross-entropy loss function to evaluate the accuracy of the model on the automatic speech recognition subtask.

[0118] Optionally, in this embodiment, the task assignment trigger classification loss can be a quantitative indicator to measure the accuracy of the model in judging whether the current speech contains task assignment intent. The task assignment trigger classification loss can be calculated using binary cross-entropy loss to evaluate the model's ability to recognize imperative sentences or request sentences.

[0119] Optionally, in this embodiment, the segment start and end position loss can be a quantitative indicator that measures the accuracy of the model in locating the start and end positions of key fields indicating objects in the text of a speech task. The segment start and end position loss can be calculated using sequence labeling loss or boundary regression loss, and is used to evaluate the model's ability to accurately find the positions of pronouns such as "you" and "he" in the text.

[0120] Optionally, in this embodiment, the object type classification loss can be a quantitative indicator that measures the accuracy of the model in judging the object type referred to by the referential fragment. The object type classification loss can be calculated using multi-class cross-entropy loss, which is used to evaluate the model's ability to distinguish between different types of execution subjects.

[0121] Optionally, in this embodiment, the candidate object ranking loss can be a quantitative indicator that measures the accuracy of the model in correctly ranking candidate objects. The candidate object ranking loss can be calculated using a ranking loss function to evaluate the model's ability to prioritize candidate objects based on association confidence.

[0122] Optionally, in this embodiment, the target loss can be a comprehensive evaluation index obtained by weighted summation of text recognition loss, task assignment trigger classification loss, fragment start and end position loss, object type classification loss, and candidate object ranking loss. The target loss reflects the overall performance of the model across all subtasks. For example, target loss = L_asr + L_trigger + L_span + L_type + L_rank, where L_asr is the text recognition loss, L_trigger is the task assignment trigger classification loss, L_span is the fragment start and end position loss, L_type is the object type classification loss, and L_rank is the candidate object ranking loss.

[0123] The embodiments provided in this application obtain a speech training dataset, merge it into the initial model to calculate the multi-task loss, and then calculate the target loss by weighting it. Based on this loss, the model is iteratively trained until convergence. This realizes the joint optimization of multiple sub-tasks such as speech recognition, intent triggering, referential localization, type discrimination and object sorting using a multi-task learning framework. This enables the model to learn the complex mapping relationship from speech to structured task instructions end-to-end, improving the accuracy of the model in comprehensive task scenarios.

[0124] As an alternative approach, to better understand the above task instruction generation method, another embodiment will be described below.

[0125] This embodiment provides an edge-side automatic mention tag generation method. The method receives user speech signals and conversation context, constructs a candidate object set, jointly identifies task assignment intent, referential expressions, target object types, and candidate object scores in the speech, and finally outputs mention-tagged display text and structured results.

[0126] Optionally, such as Figure 3 As shown, it includes the following steps:

[0127] S302, acquire the voice signal and conversation context. The conversation context includes a member list, an agent list, candidate object aliases, recently interacted objects, the current default agent, historical mention selection records, and user error correction records.

[0128] S304, Construct a set of candidate objects. Each candidate object includes at least the following attributes: object ID (object_id), display name (display_name), alias list (alias_list), object type (object_type), active state (active_state), last interaction time (last_interaction_time), role tags (role_tags), and priority weight (priority_weight). The object type is used to distinguish between real users, agents, groups, or other objects.

[0129] S306, Task assignment trigger gating. The system detects whether the speech contains actions or phrases such as "help me," "let," "call," "check," "take a look," "organize," "process," and "reply." If it does not belong to a task assignment sentence, the original semantic transcription result is directly output without triggering mention markers.

[0130] S308, Perform referential expression recognition. The system recognizes explicit names, nicknames, pronouns, category words, and action trigger words, and records source_audio_span and text_span for subsequent interpretation and rollback.

[0131] S310, Perform candidate object sorting. The system calculates candidate scores based on name matching, alias matching, pronoun resolution, object type, action semantics, recent interaction state, and historical personalized weights.

[0132] S312, perform mention tag generation and rollback. If the highest candidate score exceeds the automatic generation threshold and the difference between it and the second candidate score exceeds the conflict threshold, then automatically generate a mention tag; otherwise, preserve the original text or provide candidate confirmation.

[0133] In one specific embodiment, edge-side joint speech and semantic recognition can adopt a structure of "acoustic encoder + context / candidate object encoder + cross-attention fusion layer + joint decoder + multi-task task head". This structure is not limited to a specific neural network type. The acoustic encoder can be a Conformer, Transformer, or lightweight convolutional network. In the edge implementation, quantization, pruning, or caching of candidate object vectors can be used to reduce computational cost.

[0134] The model input consists of three categories: the first is the speech feature X_audio, which can be represented as a T×80 log-mel acoustic feature, where T is the number of speech frames; the second is the conversation context C, which can be represented as an M×D_c context representation, where M is the number of most recent conversation turns; and the third is the candidate object set E, which can be represented as an N×128 candidate object vector, where N is the number of member and agent candidates. Internally, the model maps the acoustic coding results, context coding results, and candidate object coding results to a unified hidden dimension D, for example, D=256.

[0135] like Figure 4 As shown, speech features are input into the acoustic encoder, the conversation context is input into the context encoder, and the candidate objects are input into the candidate object encoder. The outputs of the acoustic encoder, the context encoder, and the candidate object encoder are then fused in a cross-attention fusion layer, and finally the text is output by the joint decoder.

[0136]

[0137] Table 1

[0138] Optionally, Table 1 shows the task headers of the model. During training, a multi-task loss can be used: L = λ1L_asr + λ2L_trigger + λ3L_span + λ4L_type + λ5L_rank, where L_asr is the text recognition loss, L_trigger is the task assignment trigger classification loss, L_span is the loss for the start and end positions of the referred fragment, L_type is the object type classification loss, and L_rank is the candidate object ranking loss. The weights λ1 to λ5 can be set according to the scale of the training data and the resource constraints on the device. During deployment, only a portion of the task headers can be retained for a lightweight implementation.

[0139] When a user does not explicitly state the object name but uses expressions such as "you," "he," "it," "this," "that," or "help me," the system eliminates them according to the priority of the candidate objects. The following rules can be configured, but the default priorities are shown in Table 2:

[0140]

[0141] Table 2

[0142] The system does not rely solely on name lookup to determine the type; instead, it comprehensively considers the object table, category terms, action semantics, and contextual state. If the denotation matches the list of real human members, the candidate type is a real human; if it matches an agent's registered name, agent alias, or category terms such as "lobster," "assistant," "agent," or "robot," the candidate type is an agent; if the same name exists in both the list of real human members and the list of agents, conflict resolution is initiated. Optionally, Table 3 provides an example of type determination.

[0143]

[0144] Table 3

[0145] The overall score of a candidate object consists of multiple reproducible sub-scores. For any candidate object i, the system calculates the name matching score f_name(i), semantic matching score f_sem(i), context score f_ctx(i), type score f_type(i), and historical personalization score f_hist(i), with each sub-score's value range normalized to [0,1]. The specific calculation method is shown in Table 4.

[0146]

[0147] Table 4

[0148] The overall score can be expressed as S_i = w1f_name(i) + w2f_sem(i) + w3f_ctx(i) + w4f_type(i) + w5f_hist(i), where w1 to w5 are non-negative weights and w1 + w2 + w3 + w4 + w5 = 1. An example of a default configuration on the client side is: w1 = 0.30, w2 = 0.20, w3 = 0.25, w4 = 0.15, w5 = 0.10; when the system relies more on historical personalization, w5 can be increased.

[0149] The confidence score of a candidate object can be obtained by softmax normalization: confidence_i = exp(S_i) / Σ_jexp(S_j). In the lightweight client-side implementation, the highest candidate score S_1 can also be directly used as an approximation of the confidence score.

[0150] like Figure 5 As shown, it may include:

[0151] S502, Detection task assignment sentence;

[0152] S504, Extracting referential expressions;

[0153] S506, Construct the candidate set;

[0154] S508, query the candidate alias table;

[0155] S510, resolve according to context priority;

[0156] S512, distinguish between category words and action semantics;

[0157] S514, Calculate the overall score;

[0158] S516: If the overall score exceeds the threshold, an instruction will be generated.

[0159] Optionally, Table 5 provides examples of decision conditions, judgment rules, and outputs, as follows:

[0160]

[0161] Table 5

[0162] In one embodiment, the task assignment trigger threshold T_trigger can be set to 0.65, the automatic generation threshold T_auto can be set to 0.78, and the conflict threshold Δ can be set to 0.12. These values ​​are example thresholds and can be adjusted based on device performance, session scenarios, user error correction feedback, and security policies.

[0163] The system records user actions on the client side, including accepting, deleting, replacing, and reselecting auto-mention tags. If a user repeatedly modifies "You help me" to a certain default agent, the priority_weight of that agent in the same category of expressions is increased; if a user deletes a certain type of auto-mention tag, the weight of the corresponding triggering mode is decreased. This mechanism can be implemented through local caching and weight tables, or through lightweight model parameter updates on the client side, but does not require uploading the original session content.

[0164] It's important to note that Automatic Speech Recognition (ASR) is a technology that converts user speech signals into text or intermediate semantic representations. A mention is a marker used in instant messaging, collaboration systems, or agent platforms to explicitly point to a user, group member, agent, or other interactive object, typically represented by the "@" symbol. An AI agent is a software entity capable of dialogue, tool invocation, task execution, or automatic response. In this example, "Vox" is the agent's registered name, and "Lobster" is the agent's colloquial category word or alias. Imperative / task assignment sentences are sentences expressing commands, requests, suggestions, arrangements, or task assignments, such as "Check for me," "Have so-and-so do it," or "Tell so-and-so to take a look." Conversation context refers to environmental information related to the current voice input, including conversation members, agent list, recent messages, recent interactions, user identity, conversation type, and historical selection records. Structured semantic transcription results: In addition to the displayed text, the results also include fields such as mention_target_id, mention_target_type, mention_span, source_audio_span, confidence, and reason_code.

[0165] Optionally, in this embodiment, in the end-side speech-semantic joint recognition embodiment, the system inputs acoustic features, candidate object features, and conversation context features into the same end-side processing link. This link can employ a shared acoustic encoder, context encoder, and joint decoder, or it can be implemented using multiple lightweight modules connected in series. The joint decoder, while generating text units, predicts the imperative intent label, referential fragment, target object type, and candidate object score.

[0166] During the training or configuration phase, labeled samples can be used. The samples must contain at least the speech, transcribed text, whether a mention marker is triggered, the reference fragment, the target object ID, the object type, and the mention insertion position. If multi-task training is used, the transcription loss, intent classification loss, object type classification loss, and candidate ranking loss can be weighted and combined. If a combination of rules and a lightweight model is used, the candidate constraints and priority rules are used as post-decoding validation conditions.

[0167] Optionally, in this embodiment, when the existing ASR model cannot be directly modified, the system can first output text, timestamps, and intermediate confidence scores from the edge or local ASR, and then the edge semantic module can read the same session context to complete task assignment triggering, pronoun resolution, object type identification, and mention tag generation. This embodiment is a compatible implementation method, parallel to the edge-side speech-semantic joint recognition embodiment, and does not limit the scope of protection of this embodiment.

[0168] Optionally, in this embodiment, when a user says "Could you check tomorrow's meeting schedule for me?", the system first identifies "Could you check for me?" as a task assignment triggering mode; then it extracts the substitute word "you" and the action "check for me". If there is a unique active agent Vox in the current session, the candidate agent Vox has the highest context score and type score. The system outputs the text "@Vox, could you check tomorrow's meeting schedule for me?", and outputs the structured fields mention_target_id=Vox's object ID, mention_target_type=agent, source_audio_span=the audio segment corresponding to "you", confidence=candidate score, and reason_code=unique active agent.

[0169] Optionally, in this embodiment, when a user says "Let Lobster check industry data," the system recognizes "Lobster" as an agent category term or alias, rather than a real person's name. If the default agent for the current session is Vox, and Vox's alias_list contains "Lobster" or its role_tags contain "data query," then the system maps the target object to Vox and outputs "@Vox check industry data." If multiple agents match the "Lobster" category, they are sorted according to recent interactions, role tags, and user history selections; if the scores are close, they proceed to candidate confirmation.

[0170] Optionally, in this embodiment, when a user says "Let XX take a look at this solution," the system matches the real member "XXX" in the member list. The action "take a look" is more inclined towards human confirmation and does not match the agent category word, so it outputs "@XXX take a look at this solution." If an agent also has an alias "XXX," the system will compare the real member score and the agent score; when the difference between the two is less than the conflict threshold, it will not automatically generate a mention tag, but will provide candidate confirmation to avoid incorrect assignment.

[0171] The embodiments provided in this application, targeting multi-person conversations or multi-agent collaboration scenarios, directly transform implicit task assignment relationships in natural speech into mention tags that can be recognized and executed by the collaborative system on the edge through joint speech and semantic recognition. This solution differs from traditional ASR faithful transcription, contextual ASR hotword / entity enhancement, ordinary named entity recognition, ordinary reference resolution, and ordinary mention matching, and can improve the voice collaboration experience under low latency and privacy-friendly conditions.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0173] Embodiments of this application also provide a task instruction generation apparatus, such as... Figure 6 As shown, it includes:

[0174] The receiving unit 602 is used to receive a voice task execution request, wherein the voice task execution request includes target voice data in the voice session and object identifiers of multiple session objects participating in the voice session.

[0175] The determining unit 604 is used to determine the target object based on the association confidence of each of the multiple conversation objects when the task assignment key field and the object indication key field are identified in the speech task text converted from the target speech data. The object indication key field is used to indicate the object that performs the target task indicated by the task assignment key field, and the association confidence is determined based on the historical interaction records of the conversation object in the speech conversation and is used to indicate the degree of association between the conversation object and the target task.

[0176] Replacement unit 606 is used to replace the object indicator key field in the speech task text with the object identifier of the target object to obtain the target text;

[0177] The generation unit 608 is used to generate task instructions for the target task based on the target text.

[0178] As an optional approach, the device is further configured to: acquire multiple constraints, wherein the constraints are used to determine candidate object identifiers; traverse the multiple constraints and perform the following operations: if the historical interaction record or session configuration of the voice session matches the i-th constraint, determine at least one candidate object identifier from the historical interaction record or session configuration according to the i-th constraint, wherein i is a natural number greater than or equal to 1, and the priority of the i-th constraint is greater than the priority of the (i+1)-th constraint; determine the candidate type of each candidate object identifier among the at least one candidate object identifier; and determine the association confidence of each session object based on the at least one candidate object identifier and the candidate type of each candidate object identifier.

[0179] As an optional scheme, the apparatus is further configured to: traverse at least one candidate object identifier and multiple session objects, and perform the following operations: if the j-th candidate object identifier matches the object identifier of the k-th session object, determine a first identifier coefficient as the identifier matching score of the k-th session object, where j and k are natural numbers greater than or equal to 1; if the j-th candidate object identifier matches other identifiers of the k-th session object, determine a second identifier coefficient as the identifier matching score of the k-th session object; if the candidate type of the j-th candidate object identifier matches the object type of the k-th session object, determine a type coefficient as the type matching score of the k-th session object; and perform a weighted summation of the identifier matching score and the type matching score to obtain the association confidence of the k-th session object, wherein the association confidence of the k-th session object is used to indicate the probability that the k-th session object is the object indicated by the j-th candidate object identifier.

[0180] As an optional scheme, the device is further configured to: determine a set of association confidence scores matching the j-th candidate object identifier, wherein the set of association confidence scores includes multiple association confidence scores, and the number of association confidence scores in the set of association confidence scores is the same as the number of multiple session objects; when the difference between the first association confidence score and the second association confidence score in the set of association confidence scores is greater than or equal to a first preset threshold, and the first association confidence score is greater than a second preset threshold, the session object corresponding to the first association confidence score is determined as the target object, wherein the first association confidence score is the association confidence score with the highest value in the set of association confidence scores, and the second association confidence score is the association confidence score with the second highest value in the set of association confidence scores; when the difference between the first association confidence score and the second association confidence score in the set of association confidence scores is less than the first preset threshold, and the first association confidence score is greater than the second preset threshold, a prompt message is sent to the user account, wherein the prompt message includes the session objects corresponding to the first association confidence score and the second association confidence score respectively, and the prompt message is used to prompt the user account to select the target object, and the task instruction generation request is sent by the user account.

[0181] As an optional approach, the device is further configured to: before determining the target object based on the association confidence of each session object in the session object set, traverse each session object in the session object set and perform the following operations: determine the semantic matching score of the nth session object based on the matching status between the object type of the nth session object and the task assignment term, where n is a natural number greater than or equal to 1; determine the activity score of the nth session object based on the historical interaction records of the nth session object; determine the historical selection score of the nth session object based on the historical selection records of the nth session object, where the historical selection score is used to indicate the number of times the nth session object has been identified as the target object; and perform a weighted summation of the semantic matching score, the activity score, and the historical selection score to obtain the association confidence of the nth session object.

[0182] As an optional approach, the device is also used to: after receiving a task instruction generation request, input the target speech data, object identifiers corresponding to multiple conversation objects, and the context feature set of the speech conversation into the trained task instruction generation model to obtain the task instruction, wherein the context feature set is used to indicate the historical interaction records of the conversation objects in the speech conversation.

[0183] As an optional approach, the device is also used to: acquire a set of speech training data before receiving a task instruction generation request; input the set of speech training data into an initial task instruction generation model to obtain text recognition loss, task assignment trigger classification loss, segment start and end position loss, object type classification loss, and candidate object ranking loss; perform a weighted summation of the text recognition loss, task assignment trigger classification loss, segment start and end position loss, object type classification loss, and candidate object ranking loss to obtain a target loss; and iteratively train the initial task instruction generation model based on the target loss until the target loss meets a preset convergence condition to obtain a trained task instruction generation model.

[0184] For a description of the features in the embodiment corresponding to the task instruction generation device, please refer to the relevant description in the embodiment corresponding to the task instruction generation method, which will not be repeated here.

[0185] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described task instruction generation method embodiments.

[0186] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described task instruction generation method embodiments at runtime.

[0187] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0188] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described task instruction generation method embodiments.

[0189] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described task instruction generation method embodiments.

[0190] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0191] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] The foregoing has provided a detailed description of a task instruction generation method, apparatus, storage medium, and electronic device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for generating task instructions, characterized in that, include: Receive a voice task execution request, wherein the voice task execution request includes target voice data in the voice session and object identifiers of multiple session objects participating in the voice session; In the case of identifying the task assignment key field and the object indication key field in the voice task text converted from the target voice data, the target object is determined according to the association confidence of each of the multiple conversation objects. The object indication key field is used to indicate the object that performs the target task indicated by the task assignment key field. The association confidence is determined based on the historical interaction record of the conversation object in the voice conversation and is used to indicate the degree of association between the conversation object and the target task. The target text is obtained by replacing the object indicator key field in the voice task text with the object identifier of the target object. The task instructions for the target task are generated based on the target text.

2. The method according to claim 1, characterized in that, Before determining the target object based on the association confidence of each session object in the set of session objects, the process includes: Obtain multiple constraints, wherein the constraints are used to determine the candidate object identifier; Iterate through the multiple constraints and perform the following operations: If the historical interaction record or session configuration of the voice session matches the i-th constraint, at least one candidate object identifier is determined from the historical interaction record or session configuration according to the i-th constraint, where i is a natural number greater than or equal to 1, and the priority of the i-th constraint is greater than the priority of the (i+1)-th constraint. Determine the candidate type for each of at least one of the candidate object identifiers; Based on the at least one candidate object identifier and the candidate type of each candidate object identifier, the association confidence of each session object is determined.

3. The method according to claim 2, characterized in that, Based on the at least one candidate object identifier and the candidate type of each candidate object identifier, the association confidence of each session object is determined, including: Iterate through the at least one candidate object identifier and the plurality of session objects, and perform the following operations: If the j-th candidate object identifier matches the object identifier of the k-th session object, the first identifier coefficient is determined as the identifier matching score of the k-th session object, where j and k are natural numbers greater than or equal to 1; If the j-th candidate object identifier matches other identifiers of the k-th session object, the second identifier coefficient is determined as the identifier matching score of the k-th session object; If the candidate type of the j-th candidate object matches the object type of the k-th session object, the type coefficient is determined as the type matching score of the k-th session object. The association confidence of the k-th session object is obtained by weighted summation of the identifier matching score and the type matching score, wherein the association confidence of the k-th session object is used to indicate the probability that the k-th session object is the object indicated by the j-th candidate object identifier.

4. The method according to claim 3, characterized in that, Based on the association confidence score of each session object in the set of session objects, the target object is determined, including: A set of association confidence scores matching the j-th candidate object identifier is determined, wherein the set of association confidence scores includes multiple association confidence scores, and the number of association confidence scores in the set of association confidence scores is the same as the number of multiple session objects; If the difference between the first association confidence score and the second association confidence score in the association confidence score set is greater than or equal to a first preset threshold, and the first association confidence score is greater than the second preset threshold, then the session object corresponding to the first association confidence score is determined as the target object, wherein the first association confidence score is the highest value association confidence score in the association confidence score set, and the second association confidence score is the second highest value association confidence score in the association confidence score set; If the difference between the first association confidence score and the second association confidence score in the association confidence score set is less than a first preset threshold, and the first association confidence score is greater than a second preset threshold, a prompt message is sent to the user account. The prompt message includes session objects corresponding to the first association confidence score and the second association confidence score, respectively. The prompt message is used to prompt the user account to select a target object. The task instruction generation request is issued by the user account.

5. The method according to claim 1, characterized in that, Before determining the target object based on the association confidence of each session object in the set of session objects, the process includes: Iterate through each of the session objects in the collection of session objects and perform the following operations: Based on the matching status between the object type of the nth session object and the task assignment term, the semantic matching score of the nth session object is determined, where n is a natural number greater than or equal to 1; Based on the historical interaction records of the nth session object, the activity score of the nth session object is determined; Based on the historical selection record of the nth session object, a historical selection score for the nth session object is determined, wherein the historical selection score is used to indicate the number of times the nth session object has been identified as the target object; The association confidence of the nth session object is obtained by weighted summation of the semantic matching score, the activity score, and the historical selection score.

6. The method according to any one of claims 1-5, characterized in that, After receiving the task instruction generation request, it includes: The target speech data, the object identifiers corresponding to the multiple conversation objects, and the context feature set of the speech conversation are input into the trained task instruction generation model to obtain the task instruction, wherein the context feature set is used to indicate the historical interaction records of the conversation objects in the speech conversation.

7. The method according to claim 5, characterized in that, Before receiving the task instruction generation request, it includes: Obtain the speech training data set; The speech training dataset is input into the initial task instruction generation model to obtain text recognition loss, task assignment triggering classification loss, segment start and end position loss, object type classification loss and candidate object ranking loss. The target loss is obtained by weighted summing of the text recognition loss, task assignment triggering classification loss, fragment start and end position loss, object type classification loss, and candidate object ranking loss. The initial task instruction generation model is iteratively trained based on the target loss until the target loss meets the preset convergence condition, thus obtaining the trained task instruction generation model.

8. A task instruction generation device, characterized in that, include: A receiving unit is configured to receive a voice task execution request, wherein the voice task execution request includes target voice data in a voice session and object identifiers of multiple session objects participating in the voice session. The recognition unit is configured to, when recognizing a task assignment key field and an object indication key field in the voice task text converted from the target voice data, determine the target object based on the association confidence of each of the plurality of conversation objects. The object indication key field is used to indicate the object that performs the target task indicated by the task assignment key field, and the association confidence is determined based on the historical interaction records of the conversation object in the voice conversation, and is used to indicate the degree of association between the conversation object and the target task. The replacement unit is used to replace the object indicator key field in the speech task text with the object identifier of the target object to obtain the target text; The generation unit is used to generate task instructions for the target task based on the target text.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program is executed by an electronic device to perform the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.

Images