Agent-based anomaly detection method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an agent-based anomaly detection method and device, electronic equipment and storage medium, relating to the technical field of computers, and particularly to the technical field of artificial intelligence such as deep learning, large models, and intelligent terminals. The specific scheme is as follows: first, receiving the to-be-processed data sent by the home device, determining the first vector corresponding to the to-be-processed data, then based on the first vector, retrieving from the reference database a second vector with a similarity greater than a first threshold and risk description information corresponding to the second vector, then based on the second vector and the risk description information, determining the risk level of the first vector, and finally, in the case where the risk level of the first vector is greater than a second threshold, returning the first warning information to the home device.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of artificial intelligence technology such as deep learning, large models, and smart terminals. Specifically, it relates to an anomaly detection method, device, electronic device, and storage medium based on intelligent agents. Background Technology

[0002] With the continuous development of large-scale model technology, the application of home intelligent agents is becoming increasingly widespread. However, existing home intelligent agents typically lack adaptability and cannot provide reliable and accurate services to users in real time. How to provide accurate and reliable home intelligent agent services is a problem that urgently needs to be solved. Summary of the Invention

[0003] This disclosure aims to at least partially address one of the technical problems in the related art.

[0004] The first aspect of this disclosure proposes an anomaly detection method based on an intelligent agent, including:

[0005] Receive data to be processed sent by home appliances, wherein the data to be processed is voice and / or images collected by the home appliances;

[0006] Determine the first vector corresponding to the data to be processed;

[0007] Based on the first vector, a second vector with a similarity greater than the first threshold is retrieved from the reference database, along with the risk description information corresponding to the second vector;

[0008] Based on the second vector and the risk description information, the risk level of the first vector is determined;

[0009] If the risk level of the first vector is greater than the second threshold, a first warning message is returned to the home appliance.

[0010] A second aspect of this disclosure provides an anomaly detection device based on an intelligent agent, comprising:

[0011] A receiving module is used to receive data to be processed sent by home appliances, wherein the data to be processed is voice and / or images collected by the home appliances;

[0012] The first determining module is used to determine the first vector corresponding to the data to be processed;

[0013] The retrieval module is used to retrieve a second vector with a similarity greater than a first threshold from a reference database based on the first vector, and the risk description information corresponding to the second vector;

[0014] The second determining module is used to determine the risk level of the first vector based on the second vector and the risk description information;

[0015] The return module is used to return a first warning message to the home device when the risk level of the first vector is greater than the second threshold.

[0016] A third aspect of this disclosure provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the agent-based anomaly detection method as proposed in the first aspect of this disclosure.

[0017] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the agent-based anomaly detection method as proposed in the first aspect of this disclosure.

[0018] The fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the agent-based anomaly detection method as proposed in the first aspect of this disclosure.

[0019] The agent-based anomaly detection method, apparatus, electronic device, and storage medium disclosed herein have the following beneficial effects:

[0020] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, and then, based on the first vector, retrieves a second vector with a similarity greater than a first threshold from a reference database, along with its corresponding risk description information. Next, based on the second vector and the risk description information, it determines the risk level of the first vector. Finally, if the risk level of the first vector is greater than the second threshold, it returns a first warning message to the home appliance. Thus, after receiving data collected from the home appliance, the intelligent agent determines the vector of the collected data, retrieves vectors with high similarity to the collected data vector from the database, along with their corresponding risk description information, and determines the risk level of the collected data vector based on the retrieved vectors and risk description information. If the risk level is greater than the threshold, it sends a warning message to the home appliance. This improves the efficiency and accuracy of anomaly detection based on the intelligent agent while ensuring reliable and accurate service.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure;

[0024] Figure 2 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure;

[0025] Figure 3 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure;

[0026] Figure 4 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure;

[0027] Figure 5 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure;

[0028] Figure 6 This is a schematic diagram of the structure of the agent-based anomaly detection method provided in this disclosure;

[0029] Figure 7 This is a schematic diagram of the structure of the agent-based anomaly detection device provided in the embodiments of this disclosure;

[0030] Figure 8 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] This disclosure relates to artificial intelligence technologies such as deep learning, large models, and smart terminals.

[0033] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence.

[0034] Deep learning (DL) learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. The ultimate goal of deep learning is to enable machines to possess analytical and learning capabilities similar to humans, allowing them to recognize data such as text, images, and sound.

[0035] Large models, also known as Foundation Models, are models that extract knowledge from hundreds of millions of corpora or images, learn, and then produce large models with hundreds of millions of parameters.

[0036] A smart terminal is a device with computing, storage, and interactive capabilities that provides seamless connectivity between users and information and services. It typically employs various intelligent technologies, such as artificial intelligence, the Internet of Things, and cloud computing.

[0037] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0038] The following description, with reference to the accompanying drawings, outlines an agent-based anomaly detection method, apparatus, electronic device, and storage medium according to embodiments of the present disclosure.

[0039] Figure 1 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure.

[0040] like Figure 1 As shown, the agent-based anomaly detection method may include the following steps:

[0041] Step 101: Receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or images collected by the home appliance.

[0042] Home appliances can be any device. For example, home appliances can be smart doorbells, computers, mobile phones, etc., and this disclosure does not limit them.

[0043] It should be noted that the data to be processed can be voice and / or images collected by home appliances in real time to monitor the external environment, and collected according to a preset collection frequency. This disclosure does not limit this.

[0044] In this disclosure, an intelligent agent can receive data to be processed sent by home devices through an access service, and obtain voice and / or images collected by the home devices. This enables the intelligent agent to perform anomaly detection based on the received data to be processed, thereby improving the accuracy and reliability of anomaly detection based on the intelligent agent.

[0045] It should be noted that an intelligent agent is a computer system or entity capable of autonomous action, environmental perception, decision-making, and interaction with the environment. It typically uses a large model as its core decision-making and processing unit, possessing the ability to think independently and invoke tools to gradually achieve a given goal. It can be any type of intelligent agent. For example, it could be a home security intelligent agent, etc., and this disclosure does not limit it in this regard.

[0046] Step 102: Determine the first vector corresponding to the data to be processed.

[0047] In this disclosure, after receiving data to be processed sent by a home device, if the data to be processed is speech and / or image, the intelligent agent first converts the speech into text, and then determines the first vector corresponding to the text and / or image. This disclosure does not limit the scope of the data.

[0048] It should be noted that when converting speech to text and determining the first vector corresponding to the text and / or image, a preset vector embedding service can be called to determine the vector corresponding to the text, and a preset image embedding service can be called to determine the vector corresponding to the image. This disclosure does not limit this.

[0049] Step 103: Based on the first vector, retrieve the second vector with a similarity greater than the first threshold from the reference database, and the risk description information corresponding to the second vector.

[0050] The reference database can be a database used to store and retrieve vectors. It can be pre-set and can store risk dialogue text in the form of <vector, dialogue text, risk description information, risk level>, or it can store risk images in the form of <vector, image, risk description information, risk level>, etc. This disclosure does not limit it in this way.

[0051] The first threshold can be a similarity threshold used to determine whether the data to be processed corresponding to the first vector is risky data, and it can be preset. For example, the first threshold can be 0.8, and this disclosure does not limit it.

[0052] The second vector can be the vector corresponding to the risk data stored in the reference database.

[0053] It should be noted that the risk description information can be of any type. For example, it can be a risk description of fraud, a risk description of pyramid schemes, a risk description of robbery, etc., and this disclosure does not limit it.

[0054] In this disclosure, after determining the first vector corresponding to the data to be processed, the intelligent agent can call the BES service to retrieve a second vector with a similarity greater than a first threshold from the reference database based on the first vector, as well as the risk description information corresponding to the second vector, thereby improving the efficiency and reliability of anomaly detection based on the intelligent agent.

[0055] BES stands for Big Earth Storage, a distributed storage system suitable for vector database scenarios.

[0056] Step 104: Determine the risk level of the first vector based on the second vector and the risk description information.

[0057] The risk level can be preset, and this disclosure does not limit it.

[0058] In this disclosure, after retrieving a second vector with a similarity greater than a first threshold from a reference database, and obtaining the risk description information corresponding to the second vector, it can be determined that the first vector has a risk because the first vector and the second vector have a high similarity. At this time, the risk level of the first vector can be determined based on the second vector and the risk description information, thereby improving the accuracy of anomaly detection.

[0059] It should be noted that the agent may retrieve multiple second vectors from the reference database. When there are multiple second vectors, the agent may concatenate each second vector and its corresponding risk level with the first vector to form a prompt message, which is then sent to the preset model service. The model service outputs the risk level of the first vector based on the prompt message. This disclosure does not limit this.

[0060] Step 105: If the risk level of the first vector is greater than the second threshold, return the first warning information to the home device.

[0061] The second threshold can be used to determine the risk level threshold when sending the first warning information to home devices. It can be preset, and this disclosure does not limit it.

[0062] The first warning information can be pre-set or generated based on risk description information; this disclosure does not limit this.

[0063] In this disclosure, after determining the risk level of the first vector, if the risk level of the first vector is greater than the second threshold, the intelligent agent can call a preset early warning service to return the first early warning information to the home device, thereby enabling the home device to implement corresponding early warning measures, improving home security and enhancing the reliability and efficiency of anomaly detection.

[0064] It should be noted that the agent can trigger different levels of warnings depending on the risk level of the first vector. For example, the higher the risk level of the first vector, the higher the warning level triggered; this disclosure does not limit this.

[0065] Optionally, if the risk level of the first vector is greater than the fourth threshold, a third warning message is sent to the user terminal associated with the home device, so that the user can confirm the third warning message and improve the reliability of anomaly detection based on intelligent agents.

[0066] The fourth threshold can be a risk level threshold used to determine whether to send a third warning message to the user terminal. It can be preset, and this disclosure does not limit it.

[0067] It should be noted that the fourth threshold may be the same as the second threshold, or it may be different from the second threshold; this disclosure does not limit this.

[0068] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, and then, based on the first vector, retrieves a second vector with a similarity greater than a first threshold from a reference database, along with its corresponding risk description information. Next, based on the second vector and the risk description information, it determines the risk level of the first vector. Finally, if the risk level of the first vector is greater than the second threshold, it returns a first warning message to the home appliance. Thus, after receiving data collected from the home appliance, the intelligent agent determines the vector of the collected data, retrieves vectors with high similarity to the collected data vector from the database, along with their corresponding risk description information, and determines the risk level of the collected data vector based on the retrieved vectors and risk description information. If the risk level is greater than the threshold, it sends a warning message to the home appliance. This improves the efficiency and accuracy of anomaly detection based on the intelligent agent while ensuring reliable and accurate service.

[0069] Figure 2 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure.

[0070] like Figure 2 As shown, the agent-based anomaly detection method may include the following steps:

[0071] Step 201: Receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or images collected by the home appliance.

[0072] Step 202: Determine the first vector corresponding to the data to be processed.

[0073] The specific implementation of steps 201 to 202 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.

[0074] Step 203: Determine the current operating mode of the home appliances.

[0075] The operating mode can be any mode. For example, the current operating mode of the home device can be security mode, or it can be message mode, etc. This disclosure does not limit it.

[0076] In this disclosure, after determining the first vector corresponding to the data to be processed, the intelligent agent can first determine the current working mode of the home device in order to improve the accuracy of the service invoked to process the data to be processed.

[0077] Step 204: If the current working mode of the home device is the first mode, based on the first vector, retrieve the second vector with a similarity greater than the first threshold from the reference database, and the risk description information corresponding to the second vector.

[0078] The first mode can be the security mode for home devices.

[0079] In this disclosure, when the intelligent agent determines that the current working mode of the home device is the first mode, it can determine whether the current home device needs to determine whether the first vector is a risk vector. At this time, it can call the BES service to retrieve a second vector with a similarity greater than the first threshold and the risk description information corresponding to the second vector from the reference database based on the first vector, thereby providing conditions for determining whether the first vector is a risk vector. This disclosure does not limit this.

[0080] Step 205: Determine the risk level of the first vector based on the second vector and the risk description information.

[0081] Step 206: If the risk level of the first vector is greater than the second threshold, return the first warning information to the home device.

[0082] The specific implementation of steps 205 to 206 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.

[0083] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, then determines the current operating mode of the home appliance, and if the current operating mode is a first mode, retrieves a second vector with a similarity greater than a first threshold from a reference database based on the first vector, along with the corresponding risk description information. Then, based on the second vector and the risk description information, the agent determines the risk level of the first vector. Finally, if the risk level of the first vector is greater than the second threshold, the agent returns a first warning message to the home appliance. Thus, after receiving the collected data from the home appliance, if the home appliance is currently in security mode, the agent retrieves vectors with high similarity to the collected data and their corresponding risk description information from the database, determines the risk level of the collected data vector based on these vectors and, if the risk level is greater than the threshold, sends a warning message to the home appliance. This allows the agent-based home service to be applicable to different scenarios, improving the accuracy and efficiency of agent-based anomaly detection.

[0084] Figure 3 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure.

[0085] like Figure 3 As shown, the agent-based anomaly detection method may include the following steps:

[0086] Step 301: Receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or images collected by the home appliance.

[0087] Step 302: Determine the first vector corresponding to the data to be processed.

[0088] Step 303: Determine the current operating mode of the home appliances.

[0089] The specific implementation of steps 301 to 303 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.

[0090] Step 304: If the current working mode of the home device is the second mode, determine the response prompt template corresponding to the first vector.

[0091] The second mode can be a message mode for home devices. For example, when the home device is a smart doorbell, it can be in message mode for situations such as food delivery, courier services, and property management when no one is home, allowing the smart doorbell to communicate with outsiders. This disclosure does not limit this.

[0092] The response prompt template corresponding to the first vector can be configured according to user settings, or it can be automatically generated by the intelligent agent service; this disclosure does not limit this.

[0093] In this disclosure, when the intelligent agent determines that the current working mode of the home device is the second mode, it can determine that the home device needs to obtain the response statement corresponding to the first vector. At this time, the intelligent agent can call the message service to determine the response prompt template corresponding to the first vector, thereby providing the conditions for generating the response statement corresponding to the first vector.

[0094] Optionally, when determining the response prompt template corresponding to the first vector, in order to improve the accuracy of the determined response prompt template, the first vector can first be matched with the reference vectors associated with each candidate prompt template to obtain the candidate prompt template associated with the reference vector with the highest matching degree between the first vector and the first vector. At this time, it can be determined that the first vector has a high similarity with the reference vector, and the candidate prompt template associated with the reference vector can be determined as the response prompt template corresponding to the first vector.

[0095] The candidate suggestion template can be pre-set, and this disclosure does not limit it.

[0096] It should be noted that different candidate suggestion templates may have different associated reference vectors, and this disclosure does not impose any restrictions on this.

[0097] Step 305: Generate a response statement based on the response prompt template and the first vector.

[0098] It should be noted that different response prompt templates and first vectors may result in different generated response statements, and this disclosure does not impose any limitations on this.

[0099] In this disclosure, after determining the response prompt template corresponding to the first vector, the intelligent agent can generate the response statement corresponding to the first vector based on the response prompt template and the first vector, thereby improving the reliability and accuracy of the generated response statement.

[0100] It should be noted that when generating a response statement based on the response prompt template and the first vector, the message service can send the response prompt template and the first vector to the model service, and then the model service can output the response statement. This disclosure does not limit this.

[0101] Step 306: Return a response statement to the home appliance.

[0102] In this disclosure, after generating a response statement, the intelligent agent can return a response statement to the home device, enabling the home device to respond based on the response statement in the second mode, thereby improving the accuracy and flexibility of the home device in the second mode.

[0103] It should be noted that when returning a response statement to a home appliance, a voice conversion service can be invoked to convert the response statement into speech, and then the speech can be returned to the home appliance for playback. This disclosure does not limit this.

[0104] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines the first vector corresponding to the data, then determines the current operating mode of the home appliance. If the current operating mode is a second mode, the agent determines the response prompt template corresponding to the first vector. Based on the response prompt template and the first vector, the agent generates a response statement and finally returns the response statement to the home appliance. Thus, after receiving the collected data from the home appliance and determining its corresponding vector, and if the agent determines that the home appliance is currently in message mode, it generates a response statement based on the response prompt template and the vector, and sends the response statement to the home appliance. This improves the flexibility and reliability of the home appliance's operation and enhances the diversity and accuracy of anomaly detection based on the intelligent agent.

[0105] Figure 4 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure.

[0106] like Figure 4 As shown, the agent-based anomaly detection method may include the following steps:

[0107] Step 401: Receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or image collected by the home appliance.

[0108] Step 402: Determine the first vector corresponding to the data to be processed.

[0109] The specific implementation of steps 401 to 402 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.

[0110] Step 403: If no second vector with a similarity greater than the first threshold is found in the reference database, obtain all risk description information in the reference database.

[0111] In this disclosure, after determining the first vector corresponding to the data to be processed, the agent can first search the reference database to obtain a second vector whose similarity to the first vector is greater than a first threshold. If no second vector with a similarity to the first vector greater than the first threshold is found, in order to determine whether the first vector is risky, the agent can obtain all risk description information in the reference database.

[0112] Step 404: Determine the risk characteristics corresponding to each risk description.

[0113] The risk characteristic can be any risk characteristic. For example, the risk characteristic can be a person wearing a certain color and style of clothing, a person carrying dangerous goods, etc. This disclosure does not limit it.

[0114] It should be noted that different risk descriptions may lead to different corresponding risk characteristics, and this disclosure does not impose any limitations on this.

[0115] In this disclosure, after obtaining all risk description information in the reference database, the intelligent agent can determine the risk characteristics corresponding to each risk description information, thereby improving the accuracy of determining whether the first vector has risk.

[0116] Step 405: Match each risk feature with the data to be processed to determine the number of risk features contained in the data to be processed.

[0117] In this disclosure, after determining the risk features corresponding to each risk description information, the intelligent agent determines the number of risk features contained in the data to be processed by matching each wind direction feature with the data to be processed, thereby determining whether the first vector has risk.

[0118] Optionally, when determining the number of risk features contained in the data to be processed, each risk feature can be matched with the data to be processed first to determine the probability that the data to be processed contains it. Then, if the probability that the data to be processed contains any risk feature is greater than the probability threshold, it can be determined that the data to be processed contains any risk feature, thereby improving the accuracy of determining the number of risk features contained in the data to be processed.

[0119] The probability threshold can be a critical value used to determine whether the data to be processed contains any risk feature, and it can be preset. For example, the probability threshold can be 50%, and this disclosure does not limit it.

[0120] It should be noted that when determining the number of risk features contained in the data to be processed, the agent can concatenate the risk features of each risk description information into a prompt message, and then send the prompt message and the data to be processed to the model service. The model service determines the number of risk features contained in the data to be processed based on the risk features in the prompt message. This disclosure does not limit this.

[0121] Step 406: If the number of risk features contained in the data to be processed is greater than the third threshold, return a second warning message to the home device.

[0122] The third threshold can be a critical value for the number of risk characteristics used to determine whether to send a second warning message to home appliances. It can be preset, and this disclosure does not limit it.

[0123] In this disclosure, after determining the number of risk features contained in the data to be processed, if the number of risk features contained in the data to be processed is greater than a third threshold, the agent can determine that the data to be processed is risky data. At this time, the agent can return a second warning message to the home device.

[0124] Optionally, before returning the second warning information to the home device, the intelligent agent can determine the second warning information based on the risk description information corresponding to the risk characteristics contained in the data to be processed, thereby improving the accuracy and reliability of the determined second warning information.

[0125] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, and then, if no second vector with a similarity greater than a first threshold is found in the reference database, acquires all risk description information from the reference database and determines the risk feature corresponding to each risk description information. Each risk feature is then matched with the data to be processed to determine the number of risk features contained in the data. Finally, if the number of risk features contained in the data to be processed exceeds a third threshold, a second warning message is returned to the home appliance. Thus, after determining the vector corresponding to the data collected by the home appliance, if no vector with a high similarity to that vector is found in the database, the intelligent agent matches each risk feature corresponding to all risk description information in the database with the collected data. If the number of risk features contained in the collected data exceeds a threshold, a second warning message is sent to the home appliance, thereby improving the flexibility and reliability of anomaly detection based on the intelligent agent.

[0126] Figure 5 This is a flowchart illustrating an anomaly detection method based on an intelligent agent provided in an embodiment of this disclosure.

[0127] like Figure 5 As shown, the agent-based anomaly detection method may include the following steps:

[0128] Step 501: Receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or images collected by the home appliance.

[0129] Step 502: Determine the first vector corresponding to the data to be processed.

[0130] Step 503: Based on the first vector, retrieve the second vector with a similarity greater than the first threshold from the reference database, and the risk description information corresponding to the second vector.

[0131] Step 504: Determine the risk level of the first vector based on the second vector and the risk description information.

[0132] Step 505: If the risk level of the first vector is greater than the second threshold, return the first warning information to the home device.

[0133] The specific implementation of steps 501 to 505 can be found in the detailed descriptions of other embodiments in this disclosure, and will not be repeated here.

[0134] Step 506: Upon receiving confirmation feedback regarding the first warning information, determine the risk description information corresponding to the first vector based on the data to be processed and the first warning information.

[0135] In this disclosure, when the intelligent agent receives confirmation feedback for the first warning information, it can determine the risk description information corresponding to the first vector based on the data to be processed and the first warning information, thereby enabling the intelligent agent to update the reference database and improve the real-time performance of the data contained in the database.

[0136] Step 507: The first vector, risk description information, and data to be processed are associated and stored in the reference database.

[0137] In this disclosure, after determining the risk description information corresponding to the first vector, the intelligent agent can associate and store the first vector, the risk description information, and the data to be processed into a reference database, thereby improving the real-time performance of the data in the database and enhancing the reliability of anomaly detection based on the intelligent agent.

[0138] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, and retrieves a second vector with a similarity greater than a first threshold and its corresponding risk description information from a reference database based on the first vector. Then, based on the second vector and the risk description information, the agent determines the risk level of the first vector. If the risk level of the first vector is greater than the second threshold, the agent returns a first warning message to the home appliance. Upon receiving confirmation feedback for the first warning message, the agent determines the risk description information corresponding to the first vector based on the data to be processed and the first warning message, and stores the first vector, the risk description information, and the data to be processed in the reference database. Thus, after determining the vector corresponding to the data collected by the home appliance, the agent obtains a vector with a high similarity to that vector and its corresponding risk description information from the database to determine the risk level of the vector. If the risk level is greater than the threshold, the agent sends a warning message to the home appliance. Upon receiving confirmation feedback for the warning message, the agent stores the collected data, its corresponding vector, and the risk description information in the database. This improves the reliability of anomaly detection based on the intelligent agent while ensuring the real-time nature of the data in the database.

[0139] The following is combined with Figure 6 The present disclosure provides an example of the agent-based anomaly detection method. Figure 6 This is a schematic diagram of the structure of the agent-based anomaly detection method provided in this disclosure.

[0140] like Figure 6 As shown, the agent-based anomaly detection method provided in this disclosure can trigger a serverless service through an HTTP trigger, where HTTP is short for Hypertext Transfer Protocol (HTTP), and this disclosure does not limit its usage.

[0141] Among them, the access service can be the access service for intelligent agents, which can call other services based on user requests;

[0142] The speech-to-text conversion service can convert speech into text; the dialogue risk detection service can detect whether there are risks in the dialogue text; and the image risk detection service can detect whether there are risks in the image.

[0143] The vector embedding service can convert text into vectors; the image embedding service (Img2Vec) can convert images into vectors.

[0144] The distributed storage system BES service can store and retrieve vectors, store risk dialogues including <vectors, dialogue text, risk description information, risk level>, store historical risk images including <vectors, images, risk description identifiers>, etc., and this disclosure does not limit them;

[0145] The MySQL database service can store image risk descriptions, including <risk description identifier, risk description information, risk level, risk characteristics>, and store warning information, including <warning information identifier, warning object, warning method, mobile phone number, email address, warning level>, etc. This disclosure does not limit these aspects.

[0146] The early warning service triggers alerts for potentially risky voice or image inputs.

[0147] The model service can determine whether there is a risk in the dialogue text or image.

[0148] It should be noted that the anomaly detection method based on intelligent agents provided in this disclosure may also include a message service, which can be used to answer questions based on preset prompts in scenarios such as delivery or property management when no one is at home. This disclosure does not limit this.

[0149] It should be noted that the agent-based anomaly detection method provided in this disclosure may also include a learning service. For user-confirmed risk dialogue text, the Embedding service can be called to generate vectors and then stored in the Bes service. For user-confirmed risk images, the Img2Vec service can be called to generate vectors and then stored in the Bes service, etc. This disclosure does not limit this.

[0150] The smart terminal can be a smart doorbell or other home appliance, or a computer, mobile phone or other device associated with the home appliance; this disclosure does not limit this.

[0151] Depend on Figure 6 The smart terminal sends the collected voice and / or images to the smart agent's access service via an HTTP trigger. The smart agent then calls the voice conversion service through the access service to convert the voice into text. Next, it calls the dialogue risk monitoring service and the image risk detection service to determine whether there is any risk in the voice and / or images collected by the home device. If there is a risk, the corresponding risk level will be determined. Finally, the warning service is called to send a warning message to the smart terminal. Warning messages can also be sent to users via SMS, telephone, and email.

[0152] Furthermore, when the intelligent agent invokes the dialogue risk monitoring service and the image risk detection service to determine whether there is a risk in the voice and / or images collected by the intelligent terminal, it can first invoke the Embedding service to convert the dialogue text corresponding to the voice into a vector, invoke the Img2Vec service to convert the image into a vector, and then invoke the Bes service to retrieve vectors with a similarity greater than a threshold and their corresponding risk description information from the reference database to determine the risk level. If no vector with a similarity greater than a threshold is found in the reference database, the agent can invoke the model service based on each risk feature corresponding to all risk description information in the database, the collected voice and / or images, to perform risk feature matching on the collected voice and / or images, thereby determining whether there is a risk in the collected voice and / or images. This disclosure does not limit this aspect.

[0153] To implement the above embodiments, this disclosure also proposes an anomaly detection device based on intelligent agents.

[0154] Figure 7 This is a schematic diagram of the structure of the agent-based anomaly detection device provided in an embodiment of this disclosure.

[0155] like Figure 7 As shown, the anomaly detection device 700 based on intelligent agents includes: a receiving module 701, a first determining module 702, a retrieving module 703, a second determining module 704, and a returning module 705.

[0156] The receiving module 701 is used to receive data to be processed sent by the home appliance, wherein the data to be processed is voice and / or images collected by the home appliance;

[0157] The first determining module 702 is used to determine the first vector corresponding to the data to be processed;

[0158] The retrieval module 703 is used to retrieve a second vector with a similarity greater than a first threshold from the reference database based on the first vector, and the risk description information corresponding to the second vector;

[0159] The second determining module 704 is used to determine the risk level of the first vector based on the second vector and the risk description information;

[0160] The return module 705 is used to return a first warning message to the home device when the risk level of the first vector is greater than the second threshold.

[0161] In one possible implementation of this disclosure, the retrieval module 703 is specifically used for:

[0162] Determine the current operating mode of the home appliances;

[0163] Given that the current working mode of the home appliance is the first mode, a second vector with a similarity greater than the first threshold is retrieved from the reference database based on the first vector.

[0164] In one possible implementation of this disclosure, the retrieval module 703 is further configured to:

[0165] If the home appliance is currently operating in the second mode, determine the response prompt template corresponding to the first vector;

[0166] Generate a response statement based on the response prompt template and the first vector;

[0167] Return a response statement to the home appliance.

[0168] In one possible implementation of this disclosure, the retrieval module 703 is further configured to:

[0169] Match the first vector with the reference vector associated with each candidate suggestion template;

[0170] The candidate prompt template associated with the reference vector that has the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

[0171] In one possible implementation of this disclosure, the first determining module 702 is further configured to:

[0172] If no second vector with a similarity greater than the first threshold is found in the reference database, all risk description information in the reference database is obtained.

[0173] Determine the risk characteristics corresponding to each risk description;

[0174] Each risk feature is matched with the data to be processed to determine the number of risk features contained in the data to be processed;

[0175] If the number of risk features contained in the data to be processed exceeds the third threshold, a second warning message is returned to the home device.

[0176] In one possible implementation of this disclosure, the first determining module 702 is further configured to:

[0177] Each risk feature is matched with the data to be processed to determine the probability that the data to be processed contains it;

[0178] If the probability that the data to be processed contains any risk feature is greater than the probability threshold, then the data to be processed is determined to contain any risk feature.

[0179] In one possible implementation of this disclosure, the first determining module 702 is further configured to:

[0180] Based on the risk description information corresponding to the risk characteristics contained in the data to be processed, the second early warning information is determined.

[0181] In one possible implementation of this disclosure, the return module 705 is further configured to:

[0182] Upon receiving confirmation feedback regarding the first warning information, the risk description information corresponding to the first vector is determined based on the data to be processed and the first warning information.

[0183] The first vector, risk description information, and data to be processed are associated and stored in the reference database.

[0184] In one possible implementation of this disclosure, the second determining module 704 is further configured to:

[0185] If the risk level of the first vector is greater than the fourth threshold, a third warning message is sent to the user terminal associated with the home device.

[0186] The functions and specific implementation principles of the modules described in this embodiment can be found in the above method embodiments, and will not be repeated here.

[0187] In this embodiment, the intelligent agent first receives data to be processed from a home appliance, determines a first vector corresponding to the data, and then, based on the first vector, retrieves a second vector with a similarity greater than a first threshold from a reference database, along with its corresponding risk description information. Next, based on the second vector and the risk description information, it determines the risk level of the first vector. Finally, if the risk level of the first vector is greater than the second threshold, it returns a first warning message to the home appliance. Thus, after receiving data collected from the home appliance, the intelligent agent determines the vector of the collected data, retrieves vectors with high similarity to the collected data vector from the database, along with their corresponding risk description information, and determines the risk level of the collected data vector based on the retrieved vectors and risk description information. If the risk level is greater than the threshold, it sends a warning message to the home appliance. This improves the efficiency and accuracy of anomaly detection based on the intelligent agent while ensuring reliable and accurate service.

[0188] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0189] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0190] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0191] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0192] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as agent-based anomaly detection methods. For example, in some embodiments, the agent-based anomaly detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the agent-based anomaly detection method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform agent-based anomaly detection methods by any other suitable means (e.g., by means of firmware).

[0193] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable agent-based anomaly detection device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0195] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0197] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0198] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0199] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0200] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. In the description of this disclosure, the words "if" and "suppose" as used may be interpreted as "when," "when," "in response to determination," or "in the circumstances."

[0201] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An anomaly detection method based on intelligent agents, comprising: Receive data to be processed sent by home appliances, wherein the data to be processed is voice and / or images collected by the home appliances; Determine the first vector corresponding to the data to be processed; Based on the first vector, a second vector with a similarity greater than the first threshold is retrieved from the reference database, along with the risk description information corresponding to the second vector; Based on the second vector and the risk description information, the risk level of the first vector is determined; If the risk level of the first vector is greater than the second threshold, a first warning message is returned to the home appliance. Upon receiving confirmation feedback regarding the first warning information, the risk description information corresponding to the first vector is determined based on the data to be processed and the first warning information. The first vector, the risk description information, and the data to be processed are associated and stored in the reference database; The intelligent agent is pre-configured with speech conversion service, vector embedding service, image embedding service, distributed storage system service, and model service. Determining the first vector corresponding to the data to be processed includes: If the data to be processed contains speech, then the speech conversion service is invoked to convert the speech into text, and the vector embedding service is invoked to generate a text vector; If the data to be processed contains an image, then the image embedding service is invoked to generate an image vector; The text vector and / or image vector are determined as the first vector corresponding to the data to be processed; The step of determining the risk level of the first vector based on the second vector and the risk description information includes: If multiple second vectors are retrieved, each second vector and its corresponding risk level are concatenated with the first vector to form a prompt message, which is then sent to the model service, and the model service outputs the risk level of the first vector. After determining the first vector corresponding to the data to be processed, the method further includes: If no second vector with a similarity greater than a first threshold to the first vector is found in the reference database, all risk description information in the reference database is obtained. Determine the risk characteristics corresponding to each of the aforementioned risk description information; Each risk feature is matched with the data to be processed to determine the probability that the data to be processed contains it; If the probability that the data to be processed contains any risk feature is greater than a probability threshold, then the data to be processed is determined to contain any risk feature. If the number of risk features contained in the data to be processed exceeds a third threshold, a second warning message is returned to the home appliance.

2. The method as described in claim 1, wherein, The step of retrieving a second vector from the reference database based on the first vector, with a similarity greater than a first threshold, includes: Determine the current operating mode of the home appliance; When the current working mode of the home appliance is the first mode, a second vector with a similarity greater than a first threshold is retrieved from the reference database based on the first vector.

3. The method as described in claim 2, wherein, After determining the current operating mode of the home appliance, the method further includes: If the current working mode of the home appliance is the second mode, determine the response prompt template corresponding to the first vector; Based on the response prompt template and the first vector, a response statement is generated; Return the response statement to the home appliance.

4. The method of claim 3, wherein, The step of determining the response prompt template corresponding to the first vector includes: The first vector is matched with the reference vector associated with each candidate prompt template; The candidate prompt template associated with the reference vector that has the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

5. The method of claim 1, wherein, Before returning the second warning information to the home appliance, the method further includes: The second early warning information is determined based on the risk description information corresponding to the risk characteristics contained in the data to be processed.

6. The method as described in any one of claims 1-4, wherein, After determining the risk level of the first vector, the method further includes: If the risk level of the first vector is greater than the fourth threshold, a third warning message is sent to the user terminal associated with the home device.

7. An anomaly detection device based on intelligent agents, wherein, The device includes: A receiving module is used to receive data to be processed sent by home appliances, wherein the data to be processed is voice and / or images collected by the home appliances; The first determining module is used to determine the first vector corresponding to the data to be processed; The retrieval module is used to retrieve a second vector with a similarity greater than a first threshold from a reference database based on the first vector, and the risk description information corresponding to the second vector; The second determining module is used to determine the risk level of the first vector based on the second vector and the risk description information; The return module is used to return a first warning message to the home device when the risk level of the first vector is greater than the second threshold. The intelligent agent is pre-configured with speech conversion service, vector embedding service, image embedding service, distributed storage system service, and model service. The first determining module is specifically used for: If the data to be processed contains speech, then the speech conversion service is invoked to convert the speech into text, and the vector embedding service is invoked to generate a text vector; If the data to be processed contains an image, then the image embedding service is invoked to generate an image vector; The text vector and / or image vector are determined as the first vector corresponding to the data to be processed; The second determining module is specifically used for: If multiple second vectors are retrieved, each second vector and its corresponding risk level are concatenated with the first vector to form a prompt message, which is then sent to the model service, and the model service outputs the risk level of the first vector. The first determining module is further configured to: If no second vector with a similarity greater than a first threshold to the first vector is found in the reference database, all risk description information in the reference database is obtained. Determine the risk characteristics corresponding to each of the aforementioned risk description information; Each risk feature is matched with the data to be processed to determine the probability that the data to be processed contains it; If the probability that the data to be processed contains any risk feature is greater than a probability threshold, then the data to be processed is determined to contain any risk feature. If the number of risk features contained in the data to be processed exceeds a third threshold, a second warning message is returned to the home appliance. The return module is also used for: Upon receiving confirmation feedback regarding the first warning information, the risk description information corresponding to the first vector is determined based on the data to be processed and the first warning information. The first vector, the risk description information, and the data to be processed are associated and stored in the reference database.

8. The apparatus of claim 7, wherein, The retrieval module is specifically used for: Determine the current operating mode of the home appliance; When the current working mode of the home appliance is the first mode, a second vector with a similarity greater than a first threshold is retrieved from the reference database based on the first vector.

9. The apparatus of claim 8, wherein, The retrieval module is also used for: If the current working mode of the home appliance is the second mode, determine the response prompt template corresponding to the first vector; Based on the response prompt template and the first vector, a response statement is generated; Return the response statement to the home appliance.

10. The apparatus of claim 9, wherein, The retrieval module is also used for: The first vector is matched with the reference vector associated with each candidate prompt template; The candidate prompt template associated with the reference vector that has the highest matching degree with the first vector is determined as the response prompt template corresponding to the first vector.

11. The apparatus of claim 7, wherein, The first determining module is further configured to: The second early warning information is determined based on the risk description information corresponding to the risk characteristics contained in the data to be processed.

12. The apparatus according to any one of claims 7-10, wherein, The second determining module is further configured to: If the risk level of the first vector is greater than the fourth threshold, a third warning message is sent to the user terminal associated with the home device.

13. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that may be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images