Multi-agent matching method and device, storage medium and electronic device

Abstract

The invention discloses a multi-agent matching method and device, a storage medium and an electronic device, and relates to the technical field of smart home / smart home, and the multi-agent matching method comprises the steps: obtaining user interaction data; when determining that the data input type of the user interaction data is voice input, obtaining a user identity and a wake-up word corresponding to the user identity according to the user interaction data; determining a target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity in combination with a preset agent database; wherein the preset agent database is firstly constructed based on different users, agents correspondingly bound to the users and preset wake-up words of the agents. According to the method, an agent scheduling mechanism based on identity and wake-up word dual verification is constructed, so that the target agent scheduled by the system is ensured not only to be bound with the identity of the user, but also to be activated through a correct wake-up word, and therefore, service agent matching with high individuation, high safety and high accuracy is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home, and in particular to a multi-agent matching method and device, a storage medium and an electronic device. BACKGROUND

[0002] With the development of large language model (LLM) and intelligent agent (Agent) technology, AI agent applications driven by LLM with environment perception and autonomous decision-making capabilities are becoming the core of human-computer interaction. These agents achieve highly personalized and proactive intelligent interaction loops by deeply understanding user intent and calling corresponding tools and services, providing technical possibilities for implementing multi-user differentiated services in home devices.

[0003] Currently, the television system has integrated basic AI functions such as voice recognition. The television system mostly provides fixed services such as general recommendation lists through a single user portrait, and performs user identity verification through the use of voiceprint or facial recognition technology to unlock the device.

[0004] However, the above television system cannot distinguish between multiple users, causing all users to share the same service strategy, which can easily result in inaccurate recommended content. Moreover, since only fixed services are provided, there is no need for dynamic agent matching, making the interaction response irrelevant to user preferences, resulting in a decline in user experience, such as high false wake-up rates and response delays. SUMMARY

[0005] The present application provides a multi-agent matching method, device, storage medium and electronic device to solve the problem of poor content recommendation accuracy due to the inability to dynamically match different agents according to user groups in the prior art, and to achieve highly personalized, high-security and high-accuracy agent service matching.

[0006] The present application provides a multi-agent matching method, comprising: obtaining user interaction data, the user interaction data being used to represent data input by a user based on a user intent; when the data input type of the user interaction data is determined to be voice input, obtaining a user identity and a wake-up word corresponding to the user identity according to the user interaction data; determining a target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity, in combination with a preset agent database; wherein the preset agent database is constructed based on different users, agents bound to each user and preset wake-up words of each agent.

[0007] According to the multi-agent matching method provided in the application, the target agent corresponding to the user identity is determined according to the user identity and the wake-up word corresponding to the user identity, in combination with the preset agent database, including: according to the user identity, searching the preset agent database to obtain a corresponding agent candidate result; traversing the agent candidate result to determine whether the preset wake-up word of each candidate agent matches the wake-up word corresponding to the user identity, to obtain an agent matching result; based on the agent matching result including one matching agent, taking the matching agent as the target agent; based on the agent matching result including at least two matching agents or a matching failure, respectively determining the first similarity of the preset wake-up word of each matching agent or each candidate agent to the wake-up word corresponding to the user identity, and selecting the matching agent or candidate agent corresponding to the highest first similarity as the target agent.

[0008] According to the multi-agent matching method provided in the application, the target agent corresponding to the user identity is determined according to the user identity and the wake-up word corresponding to the user identity, in combination with the preset agent database, including: according to the user identity, searching the preset agent database to obtain a corresponding agent candidate result; traversing the agent candidate result to determine whether the preset wake-up word of each candidate agent matches the wake-up word corresponding to the user identity, to obtain an agent matching result; based on the agent matching result including one matching agent, taking the matching agent as the target agent; based on the agent matching result including at least two matching agents or a matching failure, respectively determining the first similarity of the preset wake-up word of each matching agent or each candidate agent to the wake-up word corresponding to the user identity, and selecting the matching agent or candidate agent corresponding to the highest first similarity as the target agent.

[0009] According to the multi-agent matching method provided in the application, the user identity is obtained according to the user interaction data, including: performing voiceprint recognition on the user interaction data to obtain a voiceprint recognition result; wherein the voiceprint recognition result includes the confidence of at least one first user; obtaining a user image from the user interaction data, and performing face recognition on the user image to obtain a face recognition result; wherein the face recognition result includes the confidence of at least one second user; fusing the voiceprint recognition result and the face recognition result, and obtaining the corresponding user identity based on the user confidence after fusion being greater than a first preset threshold.

[0010] According to the multi-agent matching method provided in the application, before fusing the voiceprint recognition result and the face recognition result, including: determining the user with the highest confidence in the voiceprint recognition result to obtain a first target user; determining the user with the highest confidence in the face recognition result to obtain a second target user; when the first target user and the second target user are the same user, or the first target user and the second target user are different users, and the confidence difference between the first target user and / or the second target user and other users in the corresponding recognition result is less than a preset difference, fusing the voiceprint recognition result and the face recognition result.

[0011] According to the multi-agent matching method provided in the application, the user identity corresponding to the wake-up word is obtained according to the user interaction data, including: when the user interaction data includes at least one user identity, performing wake-up word detection on the user interaction data based on the user identity to locate the start and end time points of the wake-up word corresponding to the user identity from the user interaction data, and determining the wake-up word audio corresponding to the user identity; performing speech recognition on the wake-up word audio corresponding to the user identity, and performing standardization processing on the speech recognition result to obtain the wake-up word corresponding to the user identity; wherein the standardization processing includes at least one of removing punctuation, converting to lowercase, and processing homophonic characters; and / or, when the user interaction data includes one user identity, performing speech recognition on the user interaction data to obtain the speech recognition result corresponding to the user identity; performing standardization processing on each speech recognition result respectively, and performing word segmentation on the speech recognition result after standardization processing to obtain a plurality of segmented words; matching each segmented word with a preset wake-up word library respectively, and based on the matching success, determining that the corresponding segmented word is a wake-up word; wherein the preset wake-up word library is constructed based on the preset wake-up word corresponding to all agents.

[0012] According to the multi-agent matching method provided in the application, after the user interaction data is obtained, the method further comprises: when the data input type of the user interaction data is determined to be a non-voice input, obtaining a user image according to the user interaction data, and performing face recognition on the user image to obtain a face recognition result; wherein the face recognition result comprises a confidence level of at least one user; when the face recognition result indicates that the confidence level of a user is greater than a second preset threshold, selecting the user corresponding to the maximum confidence level to obtain a user identity; recognizing a user intent according to the user interaction data; and determining a target agent corresponding to the user identity by combining a preset agent database according to the user identity and the recognized user intent.

[0013] The application further provides a multi-agent matching device, comprising: a data acquisition module configured to acquire user interaction data, wherein the user interaction data is used to represent data input by a user based on a user intent; an identification module configured to, when the data input type of the user interaction data is determined to be a voice input, obtain a user identity and a wake-up word corresponding to the user identity according to the user interaction data; and a matching module configured to determine a target agent corresponding to the user identity by combining a preset agent database according to the user identity and the wake-up word corresponding to the user identity; wherein the preset agent database is constructed based on different users, agents corresponding to each user, and preset wake-up words of each agent.

[0014] The application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement any of the multi-agent matching methods described above.

[0015] The application further provides a computer-readable storage medium comprising a stored program, wherein the program is executed to implement any of the multi-agent matching methods described above.

[0016] The application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement any of the multi-agent matching methods described above.

[0017] The multi-agent matching method, device, storage medium and electronic device provided by the application establish a basic communication bridge between the system and the user by obtaining user interaction data, provide original and necessary data input for subsequent intelligent processing, and when the data input type of the user interaction data is voice input, obtain the user identity and the wake-up word corresponding to the user identity according to the user interaction data, and then determine the target agent corresponding to the user identity in combination with the preset agent database, so as to construct an agent scheduling mechanism based on double verification of identity and wake-up word, ensure that the target agent scheduled by the system is not only bound to the user identity, but also must be activated through the correct wake-up word, and thus realize highly personalized, highly secure and highly accurate service agent matching. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative labor.

[0020] Figure 1 Fig. 1 is a hardware environment schematic diagram of a multi-agent matching method according to an embodiment of the application; Figure 2 Fig. 2 is one of the flow schematic diagrams of the multi-agent matching method provided by the application; Figure 3 Fig. 3 is another of the flow schematic diagrams of the multi-agent matching method provided by the application; Figure 4 Fig. 4 is a structural schematic diagram of the multi-agent matching device provided by the application; Figure 5 Fig. 5 is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0021] In order to make the person skilled in the art better understand the application scheme, the technical solutions in the embodiments of the application will be clearly and completely described below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] According to an aspect of an embodiment of the present application, a multi-agent matching method is provided. The multi-agent matching method is widely applied to smart home, smart home, smart home device ecology, intelligence house ecology, and other whole-house intelligent digital control application scenarios. Optionally, in the present embodiment, the multi-agent matching method can be applied to the hardware environment composed of a terminal device 102 and a server 104 as shown in the figure. Figure 1 As shown in the figure, the server 104 is connected with the terminal device 102 through a network, which can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal, a database can be set on the server or independently of the server, which is used to provide data storage services for the server 104, cloud computing and / or edge computing services can be configured on the server or independently of the server, which is used to provide data operation services for the server 104. Figure 1 As shown in the figure, the server 104 is connected with the terminal device 102 through a network, which can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal, a database can be set on the server or independently of the server, which is used to provide data storage services for the server 104, cloud computing and / or edge computing services can be configured on the server or independently of the server, which is used to provide data operation services for the server 104.

[0024] The network can include but is not limited to at least one of the following: wired network, wireless network. The wired network can include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network. The wireless network can include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 can not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart oven, smart refrigerator, smart oven, smart oven, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projection equipment, smart television, smart clothesline, smart curtain, smart audio and video, smart socket, smart sound, smart sound box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart window cleaning robot, smart mopping robot, smart air purification equipment, smart steamer, smart microwave oven, smart kitchen treasure, smart purifier, smart water dispenser, smart door lock, etc.

[0025] Figure 2A flowchart of a multi-agent matching method is shown, the method comprises: S21, obtaining user interaction data, the user interaction data is used to represent data input by the user based on the user's intention; S22, when the data input type of the user interaction data is determined to be voice input, obtaining the user identity and the wake-up word corresponding to the user identity according to the user interaction data; S23, according to the user identity and the wake-up word corresponding to the user identity, combining the preset agent database, determining the target agent corresponding to the user identity; wherein the preset agent database is constructed based on different users, the agents corresponding to each user and the preset wake-up words of each agent.

[0026] It should be noted that the step number "S2N" in the present specification does not represent the order of the multi-agent matching method, and the following will be specifically combined Figure 3 The multi-agent matching method of the present application is described.

[0027] Step S21, obtaining user interaction data, the user interaction data is used to represent data input by the user based on the user's intention.

[0028] It should be noted that the user interaction data includes user voice input data or non-voice data, and the non-voice input data can be based on key input data or touch screen input data, which can be flexibly configured according to the actual involved scene, and is not limited further.

[0029] Step S22, when the data input type of the user interaction data is determined to be voice input, obtaining the user identity and the wake-up word corresponding to the user identity according to the user interaction data.

[0030] It should be noted that in the case of determining the user interaction data as voice, the user interaction data can be directly recognized by voice to obtain the corresponding user identity and the wake-up word corresponding to the user identity. However, the user identity and the wake-up word are output by the same model, which means that the decision logic of the model is coupled, and when the user's accent is heavy, etc., the model is prone to errors when identifying the user identity and the wake-up word.

[0031] Therefore, in the present embodiment, the user identity is obtained according to the user interaction data, comprising: performing voiceprint recognition on the user interaction data to obtain a voiceprint recognition result; wherein the voiceprint recognition result includes the confidence of at least one first user; obtaining a user image from the user interaction data and performing face recognition on the user image to obtain a face recognition result; wherein the face recognition result includes the confidence of at least one second user; fusing the voiceprint recognition result and the face recognition result, and based on the fused user confidence being greater than a first preset threshold, obtaining the corresponding user identity.

[0032] It should be noted that by performing voiceprint recognition on the user interaction data to identify the user, and introducing the user image for face recognition to effectively distinguish users with similar voices, and fusing the voiceprint recognition result and the face recognition result to complement the scene, the accuracy of identity recognition is effectively improved, and an attacker needs to fake the voiceprint and face of the user at the same time to deceive the system, which improves the security barrier of the system, and by respectively determining the user identity and the wake-up word, the decoupling of identity and intent is realized, and the robustness and security of the system are improved.

[0033] It should be noted that in the actual design process, there may be multiple sound sources in a piece of user interaction voice, therefore, the voiceprint recognition on the user interaction data to obtain the voiceprint recognition result includes: performing speech detection on the user interaction data to locate the start and end time points of the sound source in the user interaction data, and obtaining the sound source audio; performing voiceprint recognition on the sound source audio to obtain the corresponding voiceprint recognition result.

[0034] Specifically, the voiceprint recognition on the sound source audio to obtain the voiceprint recognition result includes: performing voiceprint extraction on the sound source audio to obtain voiceprint features; comparing the voiceprint features with a preset voiceprint database to obtain the corresponding voiceprint recognition result; wherein the preset voiceprint database is constructed in advance based on the voiceprint features of all users bound to the intelligent entity.

[0035] It should be supplemented that when there are at least two sound source audios, the voiceprint recognition results thereof need to be determined respectively, and the face recognition results obtained from the user images need to be fused respectively to determine the user identities corresponding to the sound source audios; when there are at least two same identities of the user corresponding to the sound source audios, the sound source audios corresponding to the same user identities are merged, and the final different user identities are obtained as the user identity of the user interaction data.

[0036] In addition, the face recognition on the user image to obtain the face recognition result includes: obtaining the user image from the user interaction data and performing face feature extraction, and comparing the extracted face features with a preset face database to obtain the face recognition result; wherein the preset face database is constructed in advance based on the face features of all users bound to the intelligent entity.

[0037] In addition, the fusion of the voiceprint recognition result and the face recognition result includes: for the same user, the confidence of the voiceprint recognition result and the confidence of the face recognition result are weighted and fused to obtain the corresponding fused user confidence.

[0038] In an optional embodiment, before fusing the voiceprint recognition result and the face recognition result, the following steps are included: determining the user with the highest confidence in the voiceprint recognition result to obtain a first target user; determining the user with the highest confidence in the face recognition result to obtain a second target user; when the first target user and the second target user are the same user, or when the first target user and the second target user are different users and there is a confidence difference between the first target user and / or the second target user and other users in the corresponding recognition result that is less than a preset difference value, fusing the voiceprint recognition result and the face recognition result.

[0039] It should be noted that, before fusing the voiceprint recognition result and the face recognition result, the device identifier is obtained, and the candidate user list is obtained by querying the preset device database according to the device identifier, so as to determine whether the user with the highest similarity in the candidate user list is the same user as the user with the highest confidence in the voiceprint recognition result and / or the face recognition result, so that when they are the same user, it is indicated that all evidence chains point to the same user, and at this time, more accurate confidence calculation is performed, which can ensure the accuracy of the identity while avoiding the waste of computing resources, or when there is a confidence difference between the corresponding target user and other users in the corresponding recognition result that is less than a preset difference value, the corresponding voiceprint recognition result or face recognition result is ambiguous, which means that any single-dimensional recognition result is not enough to make a high-confidence decision, at this time, more accurate confidence calculation is performed, which can actively identify identity confirmation problems that cannot be handled by a single modality, thereby greatly improving the recognition accuracy in complex scenarios.

[0040] In addition, after obtaining the first user and the second user, the following steps are further included: when the first user and the second user are different users and there is no confidence difference between the first target user and / or the second target user and other users in the corresponding recognition result that is less than a preset difference value, sending an identity inquiry notification to the user, and based on receiving the identity feedback of the user within a target time, determining the user identity, or entering a visitor mode.

[0041] It should be noted that, by requiring the user to perform more explicit identity verification, meaningless fusion calculation on low-quality biometric features is avoided, and the reliability of the system is improved. In addition, when entering the visitor mode, the smart agent can be rejected or a smart agent reserved in advance for visitors can be assigned. The specific allocation of smart agents for the visitor mode can be configured based on actual design requirements, and the corresponding smart agent allocation method can be referred to in the following description, which will not be repeated here.

[0042] In an optional embodiment, the corresponding wake-up word of the user identity is obtained according to the user interaction data, including: when it is determined that the user interaction data includes at least one user identity, performing wake-up word detection on the user interaction data based on the user identity, to locate the start and end time points of the wake-up word corresponding to the user identity from the user interaction data, and determine the wake-up word audio corresponding to the user identity; performing speech recognition on the wake-up word audio corresponding to the user identity, and performing standardized processing on the speech recognition result to obtain the wake-up word corresponding to the user identity; wherein the standardized processing includes at least one of removing punctuation, converting to lowercase, and processing homophonic variant words.

[0043] It should be noted that in the actual design process, there may be multiple sound sources in a segment of user interaction voice, therefore, performing wake-up word detection on the user interaction data based on the user identity includes: based on the user identity, obtaining corresponding sound source audio, and performing wake-up word detection on the corresponding sound source audio to locate the start and end time points of the wake-up word, so as to facilitate identifying the corresponding wake-up word for each user identity.

[0044] In addition, by performing wake-up word detection on the user interaction data based on the user identity, the multi-user concurrent voice scene is accurately isolated, so that the wake-up word segment said by the specific user is accurately cut out from the mixed voice stream, the mutual interference of different user voices is avoided, pure and high-quality audio data is provided for subsequent wake-up word recognition, and by converting the voice signal into processable text, structured and matchable wake-up word information is provided, so that the system can make accurate logical judgment and instruction routing based on the text content, and the voice interaction is improved from the signal processing level to the semantic understanding level, and the recognition accuracy is significantly improved.

[0045] In another optional embodiment, the corresponding wake-up word of the user identity is obtained according to the user interaction data, and further includes: when it is determined that the user interaction data includes one user identity, performing speech recognition on the user interaction data to obtain a speech recognition result corresponding to the user identity; performing standardized processing on each speech recognition result respectively, and performing word segmentation on the speech recognition result after the standardized processing to obtain a plurality of segmented words; matching each segmented word with a preset wake-up word library respectively, and based on a successful match, determining that the corresponding segmented word is a wake-up word; wherein the preset wake-up word library is constructed based on the preset wake-up words corresponding to all agents.

[0046] It should be noted that when the user interaction data only includes one user identity, the voice recognition is performed on the user interaction data to efficiently convert the complete voice instruction of the user into structured text, thereby providing basic data for subsequent semantic analysis, so that the system can process and understand the complete intention of the user at one time without needing to pre-suppose the position of the wake-up word, and the standardization processing and word segmentation are respectively performed on each voice recognition result to provide accurate and matchable word particles, and the pre-set wake-up word library is combined for matching to intelligently and imperceptibly identify the wake-up word from the natural and continuous voice stream of the user without the user needing to deliberately speak an independent wake-up instruction, so that the interaction process is more natural and smooth.

[0047] In step S23, the target agent corresponding to the user identity is determined according to the user identity and the wake-up word corresponding to the user identity in combination with the pre-set agent database; wherein the pre-set agent database is constructed in advance based on different users, the agents bound to each user and the pre-set wake-up words of each agent.

[0048] In this embodiment, the target agent corresponding to the user identity is determined according to the user identity and the wake-up word corresponding to the user identity in combination with the pre-set agent database, which includes: searching the pre-set agent database according to the user identity to obtain a corresponding agent candidate result; traversing the agent candidate result to determine whether the pre-set wake-up word of each candidate agent matches the wake-up word corresponding to the user identity to obtain an agent matching result; based on the agent matching result including one matching agent, the matching agent is taken as the target agent; based on the agent matching result including at least two matching agents or a matching failure, the first similarity of the pre-set wake-up word of each matching agent or each candidate agent to the wake-up word corresponding to the user identity is respectively determined, and the matching agent or candidate agent corresponding to the highest first similarity is selected as the target agent.

[0049] It should be noted that the agent database is pre-screened by the user identity to narrow the range of subsequent matching and calculation, thereby converting the global search problem into a local search problem, greatly improving the response speed and decision efficiency of the system, and the wake-up word is matched by traversing the agent candidate result to quickly screen the candidate agent to provide direct and accurate response for the user, and in the case of matching to one agent, the agent is directly taken as the target agent corresponding to the user identity to avoid unnecessary complex calculation and ensure the instant responsiveness of the system, and in the case of matching to at least two agents or a matching failure, the first similarity of the candidate agent of each user identity to the corresponding wake-up word is determined, and the candidate agent with the highest first similarity is selected as the target agent of the user identity, thereby effectively handling complex situations such as inaccurate pronunciation of the wake-up word, homophonic words or similar wake-up words of multiple agents, selecting the most possible intention through similarity calculation, and significantly improving the robustness of the system and the success rate of user interaction.

[0050] In addition, the first similarity can be determined based on a semantic similarity or a string similarity algorithm, and the specific algorithm can be selected according to actual design requirements. For example, the string similarity algorithm can be an edit Levenshtein distance, a Jaro-Winkler distance, and the like, which is not limited further herein.

[0051] In an optional embodiment, the user identities are at least two, and after determining the target agent corresponding to the user identity according to the user identity and the wake-up word in combination with the preset agent database, the method further includes: when the target agents corresponding to the user identities are determined to be different agents, waking up the corresponding target agent to analyze the corresponding user intent and execute; when the target agents corresponding to the user identities of at least two users are determined to be the same agent, taking the corresponding user as a target user; for each target user, determining a similarity between the preset wake-up word of the corresponding target agent and the corresponding wake-up word to obtain a wake-up word similarity of the corresponding target user; according to the wake-up word similarity of each target user, retaining the target agent of the target user with the highest similarity; when the target agent of at least one other target user is determined to be based on the corresponding agent matching result including one matching agent, deleting the retained target agent from the agent candidate result corresponding to the other target user, and determining a second similarity between the preset wake-up word of each candidate agent in the updated agent candidate result and the wake-up word of the corresponding other target user, and selecting the candidate agent corresponding to the highest second similarity as the target agent of the corresponding other target user; when the target agent of at least one other target user is determined to be based on the corresponding agent matching result including at least two matching agents or a matching failure, deleting the target agent from the corresponding agent matching result, and selecting the matching agent or candidate agent with the highest first similarity in the updated agent matching result as the target agent of the corresponding other target user; based on the updated target agent of each other target user, re-determining whether there are the same agents, and based on the existence of at least two same agents, re-updating the target agent of the corresponding user identity based on the above steps until the target agents corresponding to all user identities are not the same, and waking up the corresponding target agent to analyze the corresponding user intent and execute.

[0052] It should be noted that when the target agents corresponding to the user identities are different agents, the corresponding target agent is woken up to analyze the corresponding user intent and execute, so as to provide the most efficient parallel processing path for the ideal scenario of multi-user concurrent requests, and ensure that the intent of each user can be independently and conflict-free responded in real time by the exclusive agent of the user, thereby maximizing the system throughput.

[0053] When the target agents corresponding to the at least two user identities are the same agent, the corresponding users are taken as target users, and the similarity between the target agent of each target user and the wake-up word is determined respectively, so that the target agent of the target user with the highest similarity is retained by priority through optimal matching, the first-round agent attribution conflict is efficiently solved, and the user whose wake-up word recognition is the clearest and most accurate can obtain the expected agent service.

[0054] Further, when the target agent of the other target user is obtained based on the agent matching result including one matching agent, the occupied agent is excluded, and after the target agent is deleted, the corresponding matching agent does not exist in the updated agent candidate result, so that the target agent is re-matched in the matching failure mode, thereby providing a suboptimal selection mechanism for the other target user who loses the agent, ensuring that the user can still be allocated to the most suitable alternative agent, and improving the resource utilization rate and user satisfaction of the system.

[0055] In addition, when the target agent of the other target user is obtained based on the agent matching result including at least two matching agents or matching failure, since the agent matching result includes at least two, after the target agent is deleted, the suboptimal option in the original matching result can be directly enabled to quickly complete the re-allocation of the agent, avoid complex secondary calculation, and ensure the efficiency of conflict resolution.

[0056] In addition, by updating the target agent of each other target user, it is determined whether there is the same agent, and based on the existence of at least two same agents, the target agent of the corresponding same agent user is updated based on the above steps, to construct a closed-loop and iterative conflict resolution system, which ensures that the final allocation result of multiple user requests is absolutely conflict-free, and ensures that the system can provide an independent and executable agent service for each user, and realizes the system stability and robustness in a complex multi-user scenario.

[0057] In an optional embodiment, with reference to Figure 3 After obtaining the user interaction data, the method further includes: when the data input type of the user interaction data is a non-voice input, obtaining a user image according to the user interaction data, and performing face recognition on the user image to obtain a face recognition result; the face recognition result includes the confidence of at least one user; when the confidence of the user is greater than a second preset threshold, the user corresponding to the maximum confidence is selected as the user identity according to the face recognition result; the user intent is recognized according to the user interaction data; the target agent corresponding to the user identity is determined according to the user identity and the recognized user intent, in combination with a preset agent database.

[0058] It should be noted that when the data input type of the user interaction data is non-voice input, the corresponding user image is acquired to perform face recognition, so as to realize accurate identification of the user identity in a non-voice interaction scene, make up for the deficiency of voice recognition in a silent or noisy environment, thereby widening the human-computer interaction channel and applicable scene of the intelligent device, and determine the user identity when the confidence is greater than the second preset threshold, so as to effectively prevent identity misjudgment caused by image blur, poor light or similar faces, provide accuracy and reliability of the identity recognition result, and in combination with the intention recognition of the user interaction data, ensure that the system accurately understands the user intention, and clearly identifies the user identity, and then accurately schedules and matches the intelligent agent most suitable for the user, and provides a highly customized service experience.

[0059] It should be added that the user intention recognition can select a corresponding recognition mode based on the data input type of the user interaction data, for example, if the user interaction data is text information input by the user based on a touch screen, the user intention can be recognized by using natural language processing technology, and for another example, if the user interaction data is an operation behavior sequence input by the user based on a remote controller key, the user intention can be recognized by using behavior analysis and pattern recognition technology.

[0060] In addition, according to the user identity and the recognized user intention, in combination with the preset intelligent agent database, a target intelligent agent corresponding to the user identity is determined, including: according to the user identity, searching the preset intelligent agent database to obtain a corresponding intelligent agent candidate result; traversing the intelligent agent candidate result to determine the similarity between the function of each candidate intelligent agent and the user intention, and selecting the candidate intelligent agent with the maximum similarity as the target intelligent agent corresponding to the user identity.

[0061] In an optional embodiment, after the candidate intelligent agent with the maximum similarity is selected as the target intelligent agent corresponding to the user identity, including: waking up the target intelligent agent to analyze the user intention and execute.

[0062] In summary, the embodiments of the present application acquire user interaction data to establish a basic communication bridge between the system and the user, provide original and necessary data input for all subsequent intelligent processing, and when the data input type of the user interaction data is voice input, according to the user interaction data, the user identity and the wake-up word corresponding to the user identity are obtained, and then in combination with the preset intelligent agent database, the target intelligent agent corresponding to the user identity is determined, so as to construct an intelligent agent scheduling mechanism based on double verification of identity and wake-up word, ensure that the target intelligent agent scheduled by the system is not only bound to the user identity, but also must be activated by a correct wake-up word, so as to realize highly personalized, highly secure and highly accurate service intelligent agent matching.

[0063] The multi-agent matching device provided by the present application is described below, and the multi-agent matching device described below can be correspondingly referred to the multi-agent matching method described above.

[0064] Figure 4 A structural schematic diagram of a multi-agent matching device is shown. The device comprises: The data acquisition module 41 acquires user interaction data, and the user interaction data is used to represent data input by the user based on the user's intention. The recognition module 42 determines that the data input type of the user interaction data is voice input, and obtains the user identity and the wake-up word corresponding to the user identity according to the user interaction data. The matching module 43 determines the target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity, in combination with a preset agent database. The preset agent database is constructed in advance based on different users, agents bound to each user, and preset wake-up words of each agent.

[0065] In this embodiment, the recognition module 42 comprises: a voiceprint recognition unit that performs voiceprint recognition on the user interaction data to obtain a voiceprint recognition result; wherein the voiceprint recognition result comprises a confidence level of at least one first user; a face recognition unit that acquires a user image according to the user interaction data and performs face recognition on the user image to obtain a face recognition result; wherein the face recognition result comprises a confidence level of at least one second user; and a data fusion unit that fuses the voiceprint recognition result and the face recognition result, and obtains the corresponding user identity based on the fused user confidence level being greater than a first preset threshold.

[0066] Specifically, the voiceprint recognition unit comprises: an audio positioning sub-unit that performs voice detection on the user interaction data to locate the start and end time points of the sound source in the user interaction data, and obtains sound source audio; and a voiceprint recognition sub-unit that performs voiceprint recognition on the sound source audio to obtain the corresponding voiceprint recognition result.

[0067] Further, the voiceprint recognition sub-unit comprises: a voiceprint extraction sub-unit that performs voiceprint extraction on the sound source audio to obtain voiceprint features; and a voiceprint comparison sub-unit that compares the voiceprint features with a preset voiceprint database to obtain the corresponding voiceprint recognition result; wherein the preset voiceprint database is constructed in advance based on the voiceprint features of all users bound to the agent.

[0068] In addition, the face recognition unit comprises: a face extraction unit that acquires a user image according to the user interaction data and performs face feature extraction; and a face comparison unit that compares the extracted face features with a preset face database to obtain a face recognition result; wherein the preset face database is constructed in advance based on the face features of all users bound to the agent.

[0069] In addition, the data fusion unit is configured to: for the same user, weighting and fusing the confidence of the voiceprint recognition result and the confidence of the face recognition result to obtain a corresponding fused user confidence.

[0070] In an optional embodiment, the recognition module 42 further comprises: a first user determination unit configured to determine a user with the highest confidence in the voiceprint recognition result to obtain a first target user before fusing the voiceprint recognition result and the face recognition result; a second user determination unit configured to determine a user with the highest confidence in the face recognition result to obtain a second target user; and a verification unit configured to: when the first target user and the second target user are the same user, or when the first target user and the second target user are different users and there is a confidence difference between the first target user and / or the second target user and other users in the corresponding recognition result that is less than a preset difference value, the data fusion unit fuses the voiceprint recognition result and the face recognition result.

[0071] In addition, the recognition module 42 further comprises: an identity confirmation unit configured to: when the first user and the second user are different users and there is no confidence difference between the first target user and / or the second target user and other users in the corresponding recognition result that is less than a preset difference value, send an identity inquiry notification to the user, and determine the user identity based on receiving the identity feedback of the user within a target time, or a mode control unit configured to control entering a visitor mode.

[0072] In an optional embodiment, the recognition module 42 further comprises: a first detection unit configured to: when the user interaction data includes at least one user identity, perform wake-up word detection on the user interaction data based on the user identity to locate the start and end time points of the wake-up word corresponding to the user identity in the user interaction data, and determine the wake-up word audio corresponding to the user identity; and a wake-up word recognition unit configured to: perform speech recognition on the wake-up word audio corresponding to the user identity, and perform standardization processing on the speech recognition result to obtain the wake-up word corresponding to the user identity; wherein the standardization processing includes at least one of removing punctuation, converting to lowercase, and processing homophonic variant words.

[0073] In another optional embodiment, the recognition module 42 further comprises: a speech recognition unit configured to: when the user interaction data includes one user identity, perform speech recognition on the user interaction data to obtain a speech recognition result corresponding to the user identity; a word segmentation processing unit configured to: respectively perform standardization processing on each speech recognition result, and perform word segmentation on the speech recognition result after the standardization processing to obtain a plurality of segmented words; and a wake-up word matching unit configured to: match each segmented word with a preset wake-up word library, and based on a successful match, determine that the corresponding segmented word is a wake-up word; wherein the preset wake-up word library is constructed based on preset wake-up words corresponding to all agents.

[0074] The matching module 43 comprises: a searching unit configured to search a preset agent database according to the user identity to obtain a corresponding agent candidate result; a matching unit configured to traverse the agent candidate result to determine whether the preset wake-up word of each candidate agent matches the corresponding wake-up word of the user identity to obtain an agent matching result; and an agent determining unit configured to, based on the agent matching result comprising one matching agent, take the matching agent as a target agent; and based on the agent matching result comprising at least two matching agents or a matching failure, determine a first similarity between the preset wake-up word of each matching agent or each candidate agent and the corresponding wake-up word of the user identity, and select a matching agent or a candidate agent corresponding to the highest first similarity as the target agent.

[0075] In an optional embodiment, the user identity is at least two, and the device further comprises a wake-up execution module configured to: after determining the target agent corresponding to the user identity based on the user identity and the wake-up word in combination with the preset agent database, determine that the target agents corresponding to each user identity are different agents, and wake up the corresponding target agents to analyze the corresponding user intent and perform; determine that there are at least two target agents corresponding to the user identity that are the same agent, and take the corresponding user as a target user; determine the similarity between the preset wake-up word of the corresponding target agent and the corresponding wake-up word for each target user to obtain a wake-up word similarity of the target user; retain the target agent of the target user with the highest similarity based on the wake-up word similarity of each target user; determine that there is at least one target agent of another target user based on the corresponding agent matching result comprising one matching agent, delete the retained target agent from the agent candidate result corresponding to the other target user, and determine the second similarity between the preset wake-up word of each candidate agent in the updated agent candidate result and the wake-up word of the corresponding other target user, and select the candidate agent corresponding to the highest second similarity as the target agent of the corresponding other target user; determine that there is at least one target agent of another target user based on the corresponding agent matching result comprising at least two matching agents or a matching failure, delete the target agent from the corresponding agent matching result, and select the matching agent or the candidate agent with the highest first similarity in the updated agent matching result as the target agent of the corresponding other target user; re-determine whether there are the same agents based on the updated target agents of each other target user, and re-update the target agent of the corresponding user identity based on the above steps based on the existence of at least two same agents, until the target agents corresponding to all user identities are not the same, and wake up the corresponding target agents to analyze the corresponding user intent and perform.

[0076] In an optional embodiment, the device further comprises: a face recognition module, after obtaining the user interaction data, when the data input type of the user interaction data is determined to be a non-voice input, obtaining a user image according to the user interaction data, and performing face recognition on the user image to obtain a face recognition result; wherein the face recognition result comprises a confidence level of at least one user; an identity determination module, when the confidence level of the existing user is greater than a second preset threshold according to the face recognition result, selecting the user corresponding to the maximum confidence level to obtain the user identity; an intent recognition module, recognizing the user intent according to the user interaction data; and an agent matching module, determining the target agent corresponding to the user identity according to the user identity and the recognized user intent, and combining a preset agent database.

[0077] Specifically, the agent matching module comprises: a screening unit, which finds the corresponding agent candidate result from the preset agent database according to the user identity; and an agent matching unit, which traverses the agent candidate result, determines the similarity between the function of each candidate agent and the user intent, and selects the candidate agent with the maximum similarity as the target agent corresponding to the user identity.

[0078] In addition, the wake-up execution module is further configured to wake up the target agent to analyze the user intent and execute after selecting the candidate agent with the maximum similarity as the target agent corresponding to the user identity.

[0079] In summary, the embodiments of the present application obtain user interaction data through the data acquisition module to establish a basic communication bridge between the system and the user, providing original and necessary data input for all subsequent intelligent processing, and through the recognition module, when the data input type of the user interaction data is a voice input, obtaining the user identity and the wake-up word corresponding to the user identity according to the user interaction data, and then determining the target agent corresponding to the user identity through the matching module combined with the preset agent database, thereby constructing an intelligent agent scheduling mechanism based on double verification of identity and wake-up word, ensuring that the target agent scheduled by the system is not only bound to the user identity, but also must be activated through the correct wake-up word, thereby realizing highly personalized, highly secure and highly accurate service agent matching.

[0080] Figure 5 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 5As shown, the electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 complete mutual communication through the communications bus 540. The processor 510 can invoke a logical instruction in the memory 530 to execute a multi-agent matching method, which includes: obtaining user interaction data, the user interaction data being used to represent data input by a user based on a user intention; when a data input type of the user interaction data is determined to be voice input, obtaining a user identity and a wake-up word corresponding to the user identity according to the user interaction data; determining a target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity, in combination with a preset agent database; wherein the preset agent database is constructed in advance based on different users, agents bound to each user, and preset wake-up words of each agent.

[0081] In addition, the logical instruction in the memory 530 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0082] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a computer-readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the multi-agent matching method provided by the above-mentioned methods, which includes: obtaining user interaction data, the user interaction data being used to represent data input by a user based on a user intention; when a data input type of the user interaction data is determined to be voice input, obtaining a user identity and a wake-up word corresponding to the user identity according to the user interaction data; determining a target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity, in combination with a preset agent database; wherein the preset agent database is constructed in advance based on different users, agents bound to each user, and preset wake-up words of each agent.

[0083] In yet another aspect, the present application also provides a computer readable storage medium, which comprises a stored program, wherein the program performs the multi-agent matching method provided by the above method when running, and the method comprises: obtaining user interaction data, the user interaction data being used to represent data input by a user based on a user intention; when determining that a data input type of the user interaction data is voice input, obtaining a user identity and a wake-up word corresponding to the user identity according to the user interaction data; and determining a target agent corresponding to the user identity according to the user identity and the wake-up word corresponding to the user identity, in combination with a preset agent database; wherein the preset agent database is constructed in advance based on different users, agents bound to each user, and preset wake-up words of each agent.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0085] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0086] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-agent matching method, characterized in that, include: Acquire user interaction data, which is used to characterize data input by the user based on user intent; When the data input type of the user interaction data is determined to be voice input, the user identity and the wake word corresponding to the user identity are obtained based on the user interaction data. Based on the user identity and the wake word corresponding to the user identity, and in conjunction with a preset intelligent agent database, the target intelligent agent corresponding to the user identity is determined; wherein, the preset intelligent agent database is constructed in advance based on different users, the intelligent agents bound to each user, and the preset wake words of each intelligent agent.

2. The multi-agent matching method according to claim 1, characterized in that, Based on the user identity and its corresponding wake word, and in conjunction with a pre-defined agent database, the target agent corresponding to the user identity is determined, including: Based on the user's identity, the preset agent database is searched to obtain the corresponding agent candidate results; Traverse the candidate agent results and determine whether the preset wake word of each candidate agent matches the wake word corresponding to the user identity to obtain the agent matching result; Based on the agent matching result, a matching agent is included, and the matching agent is taken as the target agent; Based on the agent matching results, which include at least two matching agents or a failed match, the first similarity between the preset wake word of each matching agent or each candidate agent and the wake word corresponding to the user identity is determined, and the matching agent or candidate agent with the highest first similarity is selected as the target agent.

3. The multi-agent matching method according to claim 2, characterized in that, The user identity is at least two. After determining the target intelligent agent corresponding to the user identity based on the user identity and the wake word, combined with a preset intelligent agent database, the method further includes: When it is determined that the target intelligent agents corresponding to each user identity are different intelligent agents, the corresponding target intelligent agent is awakened to parse the corresponding user intent and execute it; When it is determined that there are at least two target agents corresponding to user identities that are the same agent, the corresponding user is taken as the target user; For each target user, the similarity between the preset wake word of the corresponding target agent and the corresponding wake word is determined to obtain the wake word similarity of the corresponding target user; Based on the similarity of the wake words of each target user, retain the target agent of the target user with the highest similarity; Determining that there is at least one target agent for another target user is based on the fact that the corresponding agent matching result includes a matching agent. When this is the case, the retained target agent is deleted from the agent candidate result corresponding to the other target user. The second similarity between the preset wake word of each candidate agent in the updated agent candidate result and the wake word of the corresponding other target user is determined. The candidate agent with the highest second similarity is selected as the target agent for the corresponding other target user. When determining that there is at least one target agent for another target user, if the corresponding agent matching result includes at least two matching agents or the matching fails, the target agent is deleted from the corresponding agent matching result, and the matching agent or candidate agent with the highest first similarity in the updated agent matching result is selected as the target agent for the corresponding other target user. Based on the updated target agents for each of the other target users, it is re-determined whether there are identical agents. Based on the existence of at least two identical agents, the target agents corresponding to the user identities are updated again based on the above steps until all the target agents corresponding to the user identities are different. Then, the corresponding target agents are awakened to parse the corresponding user intent and execute it.

4. The multi-agent matching method according to claim 1, characterized in that, Based on the user interaction data, the user identity is obtained, including: Voiceprint recognition is performed on the user interaction data to obtain voiceprint recognition results; wherein, the voiceprint recognition results include the confidence level of at least one first user; Based on the user interaction data, a user image is acquired, and facial recognition is performed on the user image to obtain a facial recognition result; wherein, the facial recognition result includes the confidence level of at least one second user; The voiceprint recognition result and the face recognition result are fused, and the corresponding user identity is obtained based on the user confidence score of the fused result being greater than a first preset threshold.

5. The multi-agent matching method according to claim 4, characterized in that, Before fusing the voiceprint recognition result and the face recognition result, the process includes: The user with the highest confidence level among the voiceprint recognition results is identified as the first target user. The user with the highest confidence level among the facial recognition results is identified as the second target user; When it is determined that the first target user and the second target user are the same user, or when the first target user and the second target user are different users, and there is a confidence difference between the first target user and / or the second target user and other users in the corresponding recognition results that is less than a preset difference, the voiceprint recognition result and the face recognition result are fused.

6. The multi-agent matching method according to claim 4, characterized in that, Based on the user interaction data, the wake-up word corresponding to the user's identity is obtained, including: When it is determined that the user interaction data includes at least one user identity, wake word detection is performed on the user interaction data based on the user identity to locate the start and end time points of the wake word corresponding to the user identity from the user interaction data and determine the wake word audio of the corresponding user identity. The wake-up word audio corresponding to the user's identity is subjected to speech recognition, and the speech recognition result is standardized to obtain the wake-up word corresponding to the user's identity; wherein, the standardization process includes at least one of removing punctuation, converting to lowercase, and handling homophones; and / or, When it is determined that the user interaction data includes a user identity, speech recognition is performed on the user interaction data to obtain the speech recognition result corresponding to the user identity. Each of the aforementioned speech recognition results is standardized, and the standardized speech recognition results are then segmented into words to obtain multiple word segments. Each of the aforementioned word segments is matched with a preset wake-up word library, and based on a successful match, the corresponding word segment is determined as a wake-up word; wherein, the preset wake-up word library is constructed based on the preset wake-up words corresponding to all intelligent agents.

7. The multi-agent matching method according to any one of claims 1-6, characterized in that, After acquiring user interaction data, the following is also included: When the data input type of the user interaction data is determined to be non-voice input, a user image is acquired based on the user interaction data, and face recognition is performed on the user image to obtain a face recognition result; wherein, the face recognition result includes the confidence level of at least one user; Based on the facial recognition results, when it is determined that a user's confidence level is greater than the second preset threshold, the user corresponding to the highest confidence level is selected to obtain the user's identity; Based on the user interaction data, identify the user's intent; Based on the user's identity and the identified user intent, and in conjunction with a pre-set intelligent agent database, the target intelligent agent corresponding to the user's identity is determined.

8. A multi-agent matching device, characterized in that, include: The data acquisition module acquires user interaction data, which is used to characterize data input by the user based on the user's intent. When the recognition module determines that the data input type of the user interaction data is voice input, it obtains the user identity and the wake word corresponding to the user identity based on the user interaction data. The matching module determines the target intelligent agent corresponding to the user identity based on the user identity and the wake word corresponding to the user identity, combined with a preset intelligent agent database; wherein, the preset intelligent agent database is constructed in advance based on different users, the intelligent agents bound to each user, and the preset wake words of each intelligent agent.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images