Intelligent agent-based anomaly detection method and device, electronic equipment and storage medium

Abstract

The invention provides an agent-based anomaly detection method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical field of artificial intelligence such as deep learning, large models and intelligent terminals. According to the specific scheme, firstly, to-be-processed data sent by home equipment is received, a first vector corresponding to the to-be-processed data is determined, then based on the first vector, a second vector with the similarity with the to-be-processed data larger than a first threshold value and risk description information corresponding to the second vector are retrieved from a reference database, and then based on the second vector and the risk description information, the risk description information corresponding to the second vector is obtained. And determining a risk level of the first vector, and finally returning first early warning information to the home device when the risk level of the first vector is greater than a second threshold.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology such as deep learning, large models, and smart terminals, and specifically to an anomaly detection method, device, electronic device, and storage medium based on an intelligent agent. Background Art

[0002] With the continuous development of large model technology, the application of home intelligent agents is becoming more and more extensive. However, in the existing technology, home intelligent agents usually lack the ability to adapt to changes and cannot provide users with reliable and accurate services in real time. How to provide accurate and reliable home intelligent agent services is an urgent problem to be solved. Summary of the invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0004] The first embodiment of the present disclosure proposes an agent-based anomaly detection method, comprising:

[0005] Receiving data to be processed sent by a household device, wherein the data to be processed is voice and / or image collected by the household device;

[0006] Determine a first vector corresponding to the data to be processed;

[0007] Based on the first vector, retrieve from a reference database a second vector whose similarity with the first vector is greater than a first threshold value, and risk description information corresponding to the second vector;

[0008] Determining a risk level of the first vector based on the second vector and the risk description information;

[0009] When the risk level of the first vector is greater than a second threshold, first warning information is returned to the home device.

[0010] The second aspect of the present disclosure provides an agent-based anomaly detection device, comprising:

[0011] A receiving module, used for receiving data to be processed sent by a household device, wherein the data to be processed is voice and / or image collected by the household device;

[0012] A first determining module, used to determine a first vector corresponding to the data to be processed;

[0013] A retrieval module, configured to retrieve, based on the first vector, from a reference database a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector;

[0014] A second determination module, configured to determine a risk level of the first vector based on the second vector and the risk description information;

[0015] A returning module is used to return first warning information to the home device when the risk level of the first vector is greater than a second threshold.

[0016] The third aspect embodiment of the present disclosure proposes a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the agent-based anomaly detection method proposed in the first aspect embodiment of the present disclosure is implemented.

[0017] The fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the agent-based anomaly detection method proposed in the first aspect embodiment of the present disclosure is implemented.

[0018] The fifth aspect embodiment of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the agent-based anomaly detection method proposed in the first aspect embodiment of the present disclosure.

[0019] The agent-based anomaly detection method, device, electronic device, and storage medium provided by the present disclosure have the following beneficial effects:

[0020] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household device, determines the first vector corresponding to the data to be processed, and then retrieves the second vector whose similarity with the first vector is greater than the first threshold value and the risk description information corresponding to the second vector from the reference database based on the first vector, and then determines the risk level of the first vector based on the second vector and the risk description information, and finally returns the first warning information to the household device when the risk level of the first vector is greater than the second threshold value. Thus, after receiving the collected data of the household device, the intelligent agent determines the vector of the collected data, retrieves the vector with high similarity with the vector of the collected data and the corresponding risk description information from the database, and determines the risk level of the vector of the collected data based on the retrieved vector and the risk description information, and sends the warning information to the household device when the risk level is greater than the threshold value, thereby ensuring that the household intelligent agent provides reliable and accurate services, and improving the efficiency and accuracy of the abnormality detection based on the intelligent agent.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 A flow chart of an agent-based anomaly detection method provided by an embodiment of the present disclosure;

[0024] Figure 2 A flow chart of an agent-based anomaly detection method provided by an embodiment of the present disclosure;

[0025] Figure 3 A flow chart of an agent-based anomaly detection method provided by an embodiment of the present disclosure;

[0026] Figure 4 A flow chart of an agent-based anomaly detection method provided by an embodiment of the present disclosure;

[0027] Figure 5 A flow chart of an agent-based anomaly detection method provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of the structure of the agent-based anomaly detection method provided by the present disclosure;

[0029] Figure 7 A schematic diagram of the structure of an abnormality detection device based on an intelligent agent provided in an embodiment of the present disclosure;

[0030] Figure 8 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] The present disclosure relates to the fields of artificial intelligence technologies such as deep learning, large models, and smart terminals.

[0033] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0034] Deep Learning (DL) is the process of learning the inherent laws and representation levels of sample data. The information obtained in the learning process is very helpful for interpreting data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds.

[0035] The large model can also be called the Foundation Model. The model extracts knowledge through billions of corpora or images, learns and then produces a large model with billions of parameters.

[0036] An intelligent terminal refers to a device with computing, storage and interactive capabilities that can provide a seamless connection between users and information and services. It usually adopts a variety of intelligent technologies, such as artificial intelligence, the Internet of Things, cloud computing, etc.

[0037] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0038] The following describes the agent-based anomaly detection method, device, electronic device, and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0039] Figure 1 A flowchart of an agent-based anomaly detection method provided in an embodiment of the present disclosure.

[0040] like Figure 1 As shown, the agent-based anomaly detection method may include the following steps:

[0041] Step 101: receiving data to be processed sent by a home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0042] The home appliance may be any device, for example, a smart doorbell, a computer, a mobile phone, etc., which is not limited in the present disclosure.

[0043] It should be noted that the data to be processed may be the real-time monitoring of the external environment by home appliances, and then the voice and / or images collected at a preset collection frequency, and the present disclosure does not limit this.

[0044] In the present disclosure, the intelligent agent can receive the data to be processed sent by the home appliances through access services, and obtain the voice and / or image collected by the home appliances, so that the intelligent agent can perform anomaly detection based on the received data to be processed, thereby improving the accuracy and reliability of anomaly detection based on the intelligent agent.

[0045] It should be noted that an intelligent agent is a computer system or entity that can act autonomously, perceive the environment, make decisions, and interact with the environment. It usually uses a large model as the core decision-making and processing unit, and has the ability to think independently and call tools to gradually complete a given goal. It can be any type of intelligent agent. For example, it can be a home security intelligent agent, etc., which is not limited in this disclosure.

[0046] Step 102: determine a first vector corresponding to the data to be processed.

[0047] In the present disclosure, after the intelligent agent receives the data to be processed sent by the home device, if the data to be processed is voice and / or image, the intelligent agent first converts the voice into text, and then determines the first vector corresponding to the text and / or image. The present disclosure does not limit this.

[0048] It should be noted that when converting speech into text and determining the first vector corresponding to the text and / or image, the preset vector embedding (Embedding) service can be called to determine the vector corresponding to the text, and the preset image embedding (Img2Vec) service can be called to determine the vector corresponding to the image. The present disclosure does not limit this.

[0049] Step 103: Based on the first vector, retrieve from a reference database a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector.

[0050] Among them, the reference database can be a database for storing and retrieving vectors, which can be pre-set, and can store risk dialogue text in the form of <vector, dialogue text, risk description information, risk level>, or can also store risk pictures in the form of <vector, picture, risk description information, risk level>, etc., and the present disclosure does not limit this.

[0051] The first threshold value may be a similarity critical value used to determine whether the data to be processed corresponding to the first vector is risk data, which may be preset. For example, the first threshold value may be 0.8, which is not limited in the present disclosure.

[0052] The second vector may be a vector corresponding to the risk data stored in the reference database.

[0053] It should be noted that the risk description information may be any type of risk description information. For example, the risk description information may be fraud risk description information, pyramid scheme risk description information, robbery risk description information, etc., which is not limited in the present disclosure.

[0054] In the present disclosure, after determining the first vector corresponding to the data to be processed, the intelligent agent can call the BES service, and based on the first vector, retrieve from the reference database a second vector whose similarity is greater than a first threshold, and risk description information corresponding to the second vector, thereby improving the efficiency and reliability of agent-based anomaly detection.

[0055] Among them, BES is the abbreviation of distributed storage system: Big Earth Storage, which is suitable for vector database scenarios.

[0056] Step 104: Determine the risk level of the first vector based on the second vector and the risk description information.

[0057] The risk level may be preset, and this disclosure does not limit this.

[0058] In the present disclosure, after retrieving a second vector whose similarity is greater than a first threshold value and the risk description information corresponding to the second vector from a reference database, since the similarity between the first vector and the second vector is high, it can be determined that there is a risk in the first vector. At this time, the risk level of the first vector can be determined based on the second vector and the risk description information, thereby improving the accuracy of anomaly detection.

[0059] It should be noted that the number of second vectors retrieved by the intelligent agent from the reference database can be multiple. When the number of second vectors is multiple, the intelligent agent can splice each second vector and the corresponding risk level and the first vector into prompt information, and send it to the preset model service. The model service outputs the risk level of the first vector based on the prompt information. The present disclosure does not limit this.

[0060] Step 105: When the risk level of the first vector is greater than the second threshold, first warning information is returned to the home device.

[0061] Among them, the second threshold value can be used to determine the risk level critical value when sending the first warning information to the home device. It can be pre-set and the present disclosure does not limit this.

[0062] The first warning information may be pre-set or generated according to the risk description information, which is not limited in the present disclosure.

[0063] In the present disclosure, after determining the risk level of the first vector, the intelligent agent can call a preset warning service when the risk level of the first vector is greater than a second threshold, and return the first warning information to the home appliance, so that the home appliance can implement corresponding warning measures, thereby improving the reliability and efficiency of anomaly detection on the basis of improving home safety.

[0064] It should be noted that the risk level of the first vector is different, and the intelligent agent can trigger different levels of warnings. For example, the higher the risk level of the first vector, the higher the warning level triggered, which is not limited in the present disclosure.

[0065] Optionally, when the risk level of the first vector is greater than a fourth threshold, a third warning message is sent to a user terminal associated with the home device, so that the user can confirm the third warning message, thereby improving the reliability of agent-based anomaly detection.

[0066] Among them, the fourth threshold value may be a risk level critical value used to determine whether to send the third warning information to the user terminal, which may be pre-set and is not limited in the present disclosure.

[0067] It should be noted that the fourth threshold may be the same as the second threshold, or may be different from the second threshold, and the present disclosure does not limit this.

[0068] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household device, determines the first vector corresponding to the data to be processed, and then retrieves the second vector whose similarity with the first vector is greater than the first threshold value and the risk description information corresponding to the second vector from the reference database based on the first vector, and then determines the risk level of the first vector based on the second vector and the risk description information, and finally returns the first warning information to the household device when the risk level of the first vector is greater than the second threshold value. Thus, after receiving the collected data of the household device, the intelligent agent determines the vector of the collected data, retrieves the vector with high similarity with the vector of the collected data and the corresponding risk description information from the database, and determines the risk level of the vector of the collected data based on the retrieved vector and the risk description information, and sends the warning information to the household device when the risk level is greater than the threshold value, thereby ensuring that the household intelligent agent provides reliable and accurate services, and improving the efficiency and accuracy of the abnormality detection based on the intelligent agent.

[0069] Figure 2 A flowchart of an agent-based anomaly detection method provided in one embodiment of the present disclosure.

[0070] like Figure 2 As shown, the agent-based anomaly detection method may include the following steps:

[0071] Step 201: receiving data to be processed sent by a home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0072] Step 202: determine a first vector corresponding to the data to be processed.

[0073] The specific implementation forms of step 201 to step 202 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0074] Step 203, determining the current working mode of the home appliance.

[0075] The working mode may be any mode. For example, the current working mode of the home appliance may be a security mode, or a message mode, etc., which is not limited in the present disclosure.

[0076] In the present disclosure, after determining the first vector corresponding to the data to be processed, the intelligent agent may first determine the current working mode of the home appliance in order to improve the accuracy of the service called to process the data to be processed.

[0077] Step 204 , when the current working mode of the home appliance is the first mode, based on the first vector, a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector are retrieved from a reference database.

[0078] Among them, the first mode can be a security mode of home equipment.

[0079] In the present disclosure, when the intelligent agent determines that the current working mode of the home appliance is the first mode, it can determine that the current home appliance needs to determine whether the first vector is a risk vector. At this time, the BES service can be called to retrieve a second vector with a similarity greater than a first threshold value and the risk description information corresponding to the second vector from a reference database based on the first vector, thereby providing conditions for determining whether the first vector is a risk vector. The present disclosure does not limit this.

[0080] Step 205: Determine the risk level of the first vector based on the second vector and the risk description information.

[0081] Step 206: When the risk level of the first vector is greater than the second threshold, first warning information is returned to the home device.

[0082] The specific implementation forms of step 205 to step 206 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0083] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the home appliance, determines the first vector corresponding to the data to be processed, then determines the current working mode of the home appliance, and when the current working mode of the home appliance is the first mode, based on the first vector, retrieves from the reference database a second vector with a similarity greater than a first threshold value, and the risk description information corresponding to the second vector, then determines the risk level of the first vector based on the second vector and the risk description information, and finally returns the first warning information to the home appliance when the risk level of the first vector is greater than the second threshold. Thus, after receiving the collected data of the home appliance, when the home appliance is currently in the security mode, the intelligent agent retrieves from the database a vector with a high similarity to the vector of the collected data and the corresponding risk description information, and determines the risk level of the vector of the collected data based on the vector and the risk description information, and sends a warning information to the home appliance when the risk level is greater than the threshold, so that the home service based on the intelligent agent can be applied to different scenarios, and the accuracy and efficiency of the abnormality detection based on the intelligent agent are improved.

[0084] Figure 3 A flowchart of an agent-based anomaly detection method provided in one embodiment of the present disclosure.

[0085] like Figure 3 As shown, the agent-based anomaly detection method may include the following steps:

[0086] Step 301: receiving data to be processed sent by a home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0087] Step 302: determine a first vector corresponding to the data to be processed.

[0088] Step 303: determine the current working mode of the home appliance.

[0089] The specific implementation forms of steps 301 to 303 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0090] Step 304: when the current working mode of the home appliance is the second mode, determine a response prompt template corresponding to the first vector.

[0091] The second mode may be a message mode for a home appliance. For example, when the home appliance is a smart doorbell, the smart doorbell may be in a message mode for communicating with outsiders in scenarios such as takeout, express delivery, and property management when no one is at home. This disclosure does not limit this.

[0092] Among them, the answer prompt template corresponding to the first vector can be configured according to user settings, or can also be automatically generated by the intelligent agent service, which is not limited in the present disclosure.

[0093] In the present disclosure, when the intelligent agent determines that the current working mode of the home appliance is the second mode, it can determine that the home appliance needs to obtain the response statement corresponding to the first vector. At this time, the intelligent agent can call the message service to determine the response prompt template corresponding to the first vector, thereby generating the response statement conditions corresponding to the first vector.

[0094] Optionally, when determining the response prompt template corresponding to the first vector, in order to improve the accuracy of the determined response prompt template, the first vector can be first matched with the reference vector associated with each candidate prompt template to obtain the candidate prompt template associated with the reference vector with the highest matching degree between the first vector. At this time, it can be determined that the first vector has a high similarity with the reference vector, and the candidate prompt template associated with the reference vector can be determined as the response prompt template corresponding to the first vector.

[0095] The candidate prompt template may be pre-set, and the present disclosure does not limit this.

[0096] It should be noted that different candidate prompt templates may have different associated reference vectors, which is not limited in the present disclosure.

[0097] Step 305: Generate a response sentence based on the response prompt template and the first vector.

[0098] It should be noted that if the response prompt template and the first vector are different, the generated response statements may be different, and the present disclosure does not limit this.

[0099] In the present disclosure, after determining the response prompt template corresponding to the first vector, the intelligent agent can generate a response sentence corresponding to the first vector based on the response prompt template and the first vector, thereby improving the reliability and accuracy of the generated response sentence.

[0100] It should be noted that when generating a response statement based on the response prompt template and the first vector, the message service can send the response prompt template and the first vector to the model service, and then the model service outputs the response statement. This disclosure does not limit this.

[0101] Step 306: Return a response statement to the home appliance.

[0102] In the present disclosure, after generating a response statement, the intelligent agent can return the response statement to the home appliance, so that the home appliance can respond based on the response statement in the second mode, thereby improving the accuracy and flexibility of the home appliance working in the second mode.

[0103] It should be noted that when returning a response statement to a home appliance, a voice conversion service may be called to convert the response statement into voice, which is then returned to the home appliance to play the voice, and the present disclosure does not limit this.

[0104] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household appliance, determines the first vector corresponding to the data to be processed, and then determines the current working mode of the household appliance. When the current working mode of the household appliance is the second mode, it determines the response prompt template corresponding to the first vector, and then generates a response statement based on the response prompt template and the first vector, and finally returns the response statement to the household appliance. Thus, after the intelligent agent receives the collected data of the household appliance and determines the corresponding vector, when it determines that the household appliance is currently in the message mode, it generates a response statement based on the response prompt template corresponding to the vector and the vector, and sends the response statement to the household appliance, thereby improving the flexibility and reliability of the operation of the household appliance, and improving the diversity and accuracy of abnormality detection based on the intelligent agent.

[0105] Figure 4 A flowchart of an agent-based anomaly detection method provided in one embodiment of the present disclosure.

[0106] like Figure 4 As shown, the agent-based anomaly detection method may include the following steps:

[0107] Step 401: receiving data to be processed sent by a home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0108] Step 402: determine a first vector corresponding to the data to be processed.

[0109] The specific implementation forms of step 401 to step 402 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0110] Step 403 : when no second vector having a similarity with the first vector greater than a first threshold value is retrieved in the reference database, all risk description information in the reference database is obtained.

[0111] In the present disclosure, after determining the first vector corresponding to the data to be processed, the intelligent agent can first search in the reference database to obtain a second vector whose similarity with the first vector is greater than a first threshold. If the second vector whose similarity with the first vector is greater than the first threshold is not retrieved, in order to determine whether the first vector is risky, the intelligent agent can obtain all risk description information in the reference database.

[0112] Step 404: determine the risk characteristics corresponding to each risk description information.

[0113] The risk feature may be any risk feature, for example, a person wearing clothes of a certain color or style, a person carrying dangerous goods, etc., which is not limited in the present disclosure.

[0114] It should be noted that the corresponding risk characteristics may be different depending on the risk description information, and this disclosure does not limit this.

[0115] In the present disclosure, after obtaining all risk description information in the reference database, the intelligent agent can determine the risk characteristics corresponding to each risk description information, thereby improving the accuracy of determining whether the first vector has a risk.

[0116] Step 405, matching each risk feature with the data to be processed, and determining the number of risk features contained in the data to be processed.

[0117] In the present disclosure, after determining the risk feature corresponding to each risk description information, the intelligent agent determines the number of risk features contained in the data to be processed by matching each wind direction feature with the data to be processed, thereby judging whether the first vector has risk.

[0118] Optionally, when determining the number of risk features contained in the data to be processed, each risk feature can be first matched with the data to be processed to determine the probability that the data to be processed contains it, and then, when the probability that the data to be processed contains any risk feature is greater than a probability threshold, it can be determined that the processed data contains any risk feature, thereby improving the accuracy of the determined number of risk features contained in the data to be processed.

[0119] The probability threshold may be a probability critical value used to determine whether the data to be processed contains any risk feature, which may be preset. For example, the probability threshold may be 50%, which is not limited in the present disclosure.

[0120] It should be noted that when determining the number of risk features contained in the data to be processed, the intelligent agent can splice the risk features of each risk description information into prompt information, and then send the prompt information and the data to be processed to the model service. The model service determines the number of risk features contained in the data to be processed based on the risk features in the prompt information. The present disclosure does not limit this.

[0121] Step 406: When the number of risk features included in the data to be processed is greater than the third threshold, return the second warning information to the home device.

[0122] Among them, the third threshold value can be a critical value of the number of risk features used to determine whether to send the second warning information to the home device. It can be pre-set and the present disclosure does not limit this.

[0123] In the present disclosure, after the intelligent agent determines the number of risk features contained in the data to be processed, if the number of risk features contained in the data to be processed is greater than a third threshold, it can be determined that the data to be processed is risk data. At this time, the intelligent agent can return a second warning information to the home device.

[0124] Optionally, before returning the second warning information to the home device, the intelligent agent may determine the second warning information based on the risk description information corresponding to the risk characteristics contained in the data to be processed, thereby improving the accuracy and reliability of the determined second warning information.

[0125] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household device, determines the first vector corresponding to the data to be processed, and then, if the second vector whose similarity with the first vector is greater than the first threshold value is not retrieved in the reference database, obtains all the risk description information in the reference database, and determines the risk features corresponding to each risk description information, then matches each risk feature with the data to be processed, determines the number of risk features contained in the data to be processed, and finally returns the second warning information to the household device if the number of risk features contained in the data to be processed is greater than the third threshold value. Thus, after the intelligent agent determines the vector corresponding to the collected data of the household device, if the intelligent agent does not retrieve a vector with a high similarity with the vector in the database, it matches each risk feature corresponding to all the risk description information in the database with the collected data, and sends the second warning information to the household device if the number of risk features contained in the collected data is greater than the threshold value, thereby improving the flexibility and reliability of the abnormality detection based on the intelligent agent.

[0126] Figure 5 A flowchart of an agent-based anomaly detection method provided in one embodiment of the present disclosure.

[0127] like Figure 5 As shown, the agent-based anomaly detection method may include the following steps:

[0128] Step 501: receiving data to be processed sent by a home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0129] Step 502: determine a first vector corresponding to the data to be processed.

[0130] Step 503: Based on the first vector, retrieve from a reference database a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector.

[0131] Step 504: determine the risk level of the first vector based on the second vector and the risk description information.

[0132] Step 505: When the risk level of the first vector is greater than the second threshold, first warning information is returned to the home device.

[0133] The specific implementation forms of steps 501 to 505 can refer to the detailed descriptions in other embodiments of the present disclosure, and will not be described in detail here.

[0134] Step 506: upon receiving a confirmation feedback for the first warning information, determining risk description information corresponding to the first vector according to the data to be processed and the first warning information.

[0135] In the present disclosure, when the intelligent agent receives confirmation feedback for the first warning information, it can determine the risk description information corresponding to the first vector based on the data to be processed and the first warning information, so that the intelligent agent can update the reference database and improve the real-time performance of the data contained in the database.

[0136] Step 507: associate the first vector, the risk description information and the data to be processed and store them in a reference database.

[0137] In the present disclosure, after determining the risk description information corresponding to the first vector, the intelligent agent can associate the first vector, the risk description information and the data to be processed and store them in a reference database, thereby improving the real-time performance of the data in the database and improving the reliability of anomaly detection based on the intelligent agent.

[0138] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household device, determines the first vector corresponding to the data to be processed, and retrieves the second vector with a similarity greater than the first threshold value and the risk description information corresponding to the second vector from the reference database based on the first vector, and then determines the risk level of the first vector based on the second vector and the risk description information, and returns the first warning information to the household device when the risk level of the first vector is greater than the second threshold value, and then, after receiving the confirmation feedback for the first warning information, determines the risk description information corresponding to the first vector according to the data to be processed and the first warning information, and associates the first vector, the risk description information and the data to be processed into the reference database. Thus, after determining the vector corresponding to the collected data of the household device, the intelligent agent obtains the vector with a high similarity to the vector and the corresponding risk description information from the database to determine the risk level of the vector, sends the warning information to the household device when the risk level is greater than the threshold value, and after receiving the confirmation feedback for the warning information, associates the collected data, the corresponding vector and the risk description information into the database, thereby improving the reliability of the abnormal detection based on the intelligent agent on the basis of ensuring the real-time performance of the data in the database.

[0139] Combine the following Figure 6 , an example is given to illustrate the agent-based anomaly detection method provided by the present disclosure, Figure 6 A schematic diagram of the structure of the agent-based anomaly detection method provided in the present invention.

[0140] like Figure 6 As shown, the agent-based anomaly detection method provided by the present disclosure can trigger a serverless service through an http trigger, wherein http is the abbreviation of Hypertext Transfer Protocol (HTTP), and the present disclosure does not limit this.

[0141] Among them, the access service can be an access service of the intelligent agent, which can call other services based on the user's request;

[0142] The speech conversion service can convert speech to text; the conversation risk monitoring service can detect whether the conversation text contains risks; the image risk detection service can detect whether the image contains risks;

[0143] Vector embedding service can convert text into vectors; image embedding Img2Vec service: can convert images into vectors;

[0144] The distributed storage system Bes service can store and retrieve vectors, store risk conversations including <vector, conversation text, risk description information, risk level>, store historical risk images including <vector, image, risk description identifier>, etc., which is not limited in this disclosure;

[0145] The database Mysql service can store image risk descriptions, including <risk description identifier, risk description information, risk level, risk characteristics>, and store warning information, including <warning information identifier, warning object, warning method, mobile phone number, email address, warning level>, etc. This disclosure does not limit this.

[0146] Early warning service: triggers an early warning for risky voice or images;

[0147] Model services can determine whether conversation text or images contain risks.

[0148] It should be noted that the agent-based anomaly detection method provided by the present disclosure may also include a message service, which is used to answer questions based on preset prompts in scenarios such as express delivery and property management when no one is at home. The present disclosure does not limit this.

[0149] It should be noted that the agent-based anomaly detection method provided by the present disclosure may also include a learning service. For the risk conversation text confirmed by the user, the Embedding service can be called to generate a vector and then stored in the Bes service. For the risk image confirmed by the user, the Img2Vec service can be called to generate a vector and stored in the Bes service, etc. The present disclosure does not limit this.

[0150] Among them, the smart terminal can be a home appliance such as a smart doorbell, or a computer, mobile phone or other device associated with the home appliance, and the present disclosure does not limit this.

[0151] Depend on Figure 6 It can be obtained that the smart terminal sends the collected voice and / or image to the access service of the intelligent body through an http trigger, and then the intelligent body first calls the voice conversion service through the access service to convert the voice into text, and then calls the dialogue risk monitoring service and the image risk detection service to determine whether there is a risk in the voice and / or image collected by the home device. If there is a risk, the corresponding risk level will be determined, and finally the early warning service will be called to send a warning message to the smart terminal. The early warning message can also be sent to the user through SMS, phone and email.

[0152] Furthermore, when the intelligent agent calls the conversation risk monitoring service and the image risk detection service to determine whether the voice and / or image collected by the smart terminal poses a risk, it can first call the Embedding service to convert the conversation text corresponding to the voice into a vector, call the Img2Vec service to convert the image into a vector, and then call the Bes service to retrieve the vectors with a similarity greater than a threshold and their corresponding risk description information in the reference database to determine the risk level. If no vectors with a similarity greater than a threshold are retrieved in the reference database, the model service can be called based on each risk feature corresponding to all the risk description information in the database and the collected voice and / or image to perform risk feature matching on the collected voice and / or image, thereby determining whether the collected voice and / or image poses a risk. The present disclosure does not limit this.

[0153] In order to implement the above embodiments, the present disclosure also proposes an agent-based anomaly detection device.

[0154] Figure 7 A schematic diagram of the structure of an anomaly detection device based on an intelligent agent provided in an embodiment of the present disclosure.

[0155] like Figure 7 As shown, the agent-based anomaly detection device 700 includes: a receiving module 701, a first determination module 702, a retrieval module 703, a second determination module 704, and a return module 705.

[0156] The receiving module 701 is used to receive the data to be processed sent by the home appliance, wherein the data to be processed is the voice and / or image collected by the home appliance;

[0157] A first determining module 702, used to determine a first vector corresponding to the data to be processed;

[0158] A retrieval module 703 is used to retrieve, based on the first vector, a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector from a reference database;

[0159] A second determination module 704, configured to determine a risk level of the first vector based on the second vector and the risk description information;

[0160] The return module 705 is used to return the first warning information to the home device when the risk level of the first vector is greater than the second threshold.

[0161] In a possible implementation of the present disclosure, the retrieval module 703 is specifically used to:

[0162] Determine the current operating mode of home devices;

[0163] When the current working mode of the home appliance is the first mode, based on the first vector, a second vector having a similarity greater than a first threshold is retrieved from a reference database.

[0164] In a possible implementation of the present disclosure, the retrieval module 703 is further used to:

[0165] When the current working mode of the home appliance is the second mode, determining a response prompt template corresponding to the first vector;

[0166] Generate a response sentence based on the response prompt template and the first vector;

[0167] Return a response statement to the home device.

[0168] In a possible implementation of the present disclosure, the retrieval module 703 is further used to:

[0169] Matching the first vector with a reference vector associated with each candidate prompt template;

[0170] The candidate prompt template associated with the reference vector with the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

[0171] In a possible implementation of the present disclosure, the first determining module 702 is further configured to:

[0172] When no second vector having a similarity with the first vector greater than a first threshold value is retrieved in the reference database, all risk description information in the reference database is obtained;

[0173] Determine the risk characteristics corresponding to each risk description information;

[0174] Match each risk feature with the data to be processed to determine the number of risk features contained in the data to be processed;

[0175] When the number of risk features contained in the data to be processed is greater than the third threshold, the second warning information is returned to the home device.

[0176] In a possible implementation of the present disclosure, the first determining module 702 is further configured to:

[0177] Match each risk feature with the data to be processed to determine the probability that the data to be processed contains it;

[0178] When the probability that the data to be processed contains any risk feature is greater than the probability threshold, it is determined that the processed data contains any risk feature.

[0179] In a possible implementation of the present disclosure, the first determining module 702 is further configured to:

[0180] The second warning information is determined based on the risk description information corresponding to the risk characteristics contained in the data to be processed.

[0181] In a possible implementation of the present disclosure, the returning module 705 is further configured to:

[0182] In the case of receiving a confirmation feedback for the first warning information, determining risk description information corresponding to the first vector according to the data to be processed and the first warning information;

[0183] The first vector, risk description information and data to be processed are associated and stored in a reference database.

[0184] In a possible implementation of the present disclosure, the second determining module 704 is further configured to:

[0185] When the risk level of the first vector is greater than a fourth threshold, third warning information is sent to a user terminal associated with the home device.

[0186] The functions and specific implementation principles of the above modules in the embodiments of the present disclosure can be referred to the above method embodiments, and will not be repeated here.

[0187] In the disclosed embodiment, the intelligent agent first receives the data to be processed sent by the household device, determines the first vector corresponding to the data to be processed, and then retrieves the second vector whose similarity with the first vector is greater than the first threshold value and the risk description information corresponding to the second vector from the reference database based on the first vector, and then determines the risk level of the first vector based on the second vector and the risk description information, and finally returns the first warning information to the household device when the risk level of the first vector is greater than the second threshold value. Thus, after receiving the collected data of the household device, the intelligent agent determines the vector of the collected data, retrieves the vector with high similarity with the vector of the collected data and the corresponding risk description information from the database, and determines the risk level of the vector of the collected data based on the retrieved vector and the risk description information, and sends the warning information to the household device when the risk level is greater than the threshold value, thereby ensuring that the household intelligent agent provides reliable and accurate services, and improving the efficiency and accuracy of the abnormality detection based on the intelligent agent.

[0188] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0189] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0190] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0191] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0192] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as an agent-based anomaly detection method. For example, in some embodiments, the agent-based anomaly detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the agent-based anomaly detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform an agent-based anomaly detection method in any other appropriate manner (e.g., by means of firmware).

[0193] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable agent-based anomaly detection device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0195] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0197] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0198] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0199] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0200] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the description of the present disclosure, the words "if" and "if" used can be interpreted as "at the time of" or "when" or "in response to determination" or "under the circumstances of".

[0201] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An agent-based anomaly detection method, comprising: Receiving data to be processed sent by a household device, wherein the data to be processed is voice and / or image collected by the household device; Determine a first vector corresponding to the data to be processed; Based on the first vector, retrieve from a reference database a second vector whose similarity with the first vector is greater than a first threshold value, and risk description information corresponding to the second vector; Determining a risk level of the first vector based on the second vector and the risk description information; When the risk level of the first vector is greater than a second threshold, first warning information is returned to the home device.

2. The method of claim 1, wherein: The retrieving, based on the first vector, from a reference database a second vector having a similarity greater than a first threshold value thereto, comprises: Determining a current operating mode of the home device; When the current working mode of the household device is the first mode, based on the first vector, a second vector having a similarity greater than a first threshold is retrieved from a reference database.

3. The method of claim 2, wherein: After determining the current working mode of the household device, the method further includes: When the current working mode of the household device is the second mode, determining a response prompt template corresponding to the first vector; Generate a response sentence based on the response prompt template and the first vector; The response statement is returned to the home device.

4. The method of claim 3, wherein: The determining the answer prompt template corresponding to the first vector includes: Matching the first vector with a reference vector associated with each candidate prompt template; The candidate prompt template associated with the reference vector with the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

5. The method according to any one of claims 1 to 4, wherein: After determining the first vector corresponding to the data to be processed, the method further includes: If no second vector having a similarity with the first vector greater than a first threshold value is retrieved from the reference database, acquiring all risk description information in the reference database; Determine the risk characteristics corresponding to each of the risk description information; Matching each of the risk features with the data to be processed to determine the number of risk features contained in the data to be processed; When the number of risk features included in the data to be processed is greater than a third threshold, second warning information is returned to the home device.

6. The method of claim 5, wherein: The step of matching each of the risk features with the data to be processed to determine the number of risk features contained in the data to be processed includes: Matching each risk feature with the data to be processed to determine the probability that the data to be processed contains the risk feature; When the probability that the data to be processed includes any risk feature is greater than a probability threshold, it is determined that the processed data includes any risk feature.

7. The method of claim 5, wherein: Before returning the second warning information to the household device, the method further includes: The second warning information is determined according to the risk description information corresponding to the risk characteristics contained in the data to be processed.

8. The method according to any one of claims 1 to 4, wherein: After returning the first warning information to the household device, the method further includes: In the case of receiving a confirmation feedback for the first warning information, determining risk description information corresponding to the first vector according to the data to be processed and the first warning information; The first vector, the risk description information and the data to be processed are associated and stored in the reference database.

9. The method according to any one of claims 1 to 4, wherein: After determining the risk level of the first vector, the method further includes: When the risk level of the first vector is greater than a fourth threshold, third warning information is sent to a user terminal associated with the home device.

10. An agent-based anomaly detection device, wherein: The device comprises: A receiving module, used for receiving data to be processed sent by a household device, wherein the data to be processed is voice and / or image collected by the household device; A first determining module, used to determine a first vector corresponding to the data to be processed; A retrieval module, configured to retrieve, based on the first vector, from a reference database a second vector having a similarity greater than a first threshold value and risk description information corresponding to the second vector; A second determination module, configured to determine a risk level of the first vector based on the second vector and the risk description information; A returning module is used to return first warning information to the home device when the risk level of the first vector is greater than a second threshold.

11. The device of claim 10, wherein: The retrieval module is specifically used for: Determining a current operating mode of the home device; When the current working mode of the household device is the first mode, based on the first vector, a second vector having a similarity greater than a first threshold is retrieved from a reference database.

12. The device of claim 11, wherein: The retrieval module is further used for: When the current working mode of the household device is the second mode, determining a response prompt template corresponding to the first vector; Generate a response sentence based on the response prompt template and the first vector; The response statement is returned to the home device.

13. The device of claim 12, wherein: The retrieval module is further used for: Matching the first vector with a reference vector associated with each candidate prompt template; The candidate prompt template associated with the reference vector with the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

14. The device according to claims 10-13, wherein: The first determining module is further used for: If no second vector having a similarity with the first vector greater than a first threshold value is retrieved from the reference database, acquiring all risk description information in the reference database; Determine the risk characteristics corresponding to each of the risk description information; Matching each of the risk features with the data to be processed to determine the number of risk features contained in the data to be processed; When the number of risk features included in the data to be processed is greater than a third threshold, second warning information is returned to the home device.

15. The device of claim 14, wherein: The first determining module is further used for: Matching each risk feature with the data to be processed to determine the probability that the data to be processed contains the risk feature; When the probability that the data to be processed includes any risk feature is greater than a probability threshold, it is determined that the processed data includes any risk feature.

16. The device of claim 14, wherein: The first determining module is further used for: The second warning information is determined according to the risk description information corresponding to the risk characteristics contained in the data to be processed.

17. The device according to any one of claims 10 to 13, wherein: The return module is also used for: In the case of receiving a confirmation feedback for the first warning information, determining risk description information corresponding to the first vector according to the data to be processed and the first warning information; The first vector, the risk description information and the data to be processed are associated and stored in the reference database.

18. The device according to any one of claims 10 to 13, wherein: The second determining module is further used for: When the risk level of the first vector is greater than a fourth threshold, third warning information is sent to a user terminal associated with the home device.

19. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that may be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images