Interaction method based on artificial intelligence equipment, server and artificial intelligence equipment

Abstract

The embodiment of the invention discloses an interaction method based on artificial intelligence equipment, a server and the artificial intelligence equipment. The method comprises the following steps: determining a target interaction scene corresponding to user input information, responding to the target interaction scene as a monitoring scene, calling a target large language model according to an execution element to analyze the user input information, obtaining target control parameters corresponding to the execution element, combining a plurality of target control parameters to obtain a target control table, and displaying the target control table. And sending the target control table to the artificial intelligence device, so that the artificial intelligence device executes a monitoring operation according to the target control table. According to the embodiment of the invention, the switching requirements of various interaction scenes are supported, the target monitoring task can be split into a plurality of execution elements, and the target control parameters corresponding to the execution elements are sorted out from the information input by the user, so that the target monitoring task is split into the target control parameters which can be understood by the hardware of the artificial intelligence equipment, and the user experience is improved. Therefore, the artificial intelligence equipment can reliably and accurately execute the target monitoring task.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to an interaction method, server and artificial intelligence device based on artificial intelligence equipment. Background Art

[0002] An AI device is a physical device that incorporates AI technology to simulate human intelligent behaviors, such as learning, reasoning, decision-making, and interaction. In related technologies, AI devices can intelligently interact with users based on machine learning algorithms. For example, AI devices can chat with users, output the day's weather or request songs according to the user's instructions, or monitor according to the monitoring tasks set by the user. The interaction mode of AI devices provided by related technologies is relatively simple, and they usually interact with users in a single established interaction mode, which cannot match the user's needs for multiple interaction scenarios. Summary of the invention

[0003] One purpose of the embodiments of the present application is to provide an interaction method, a server and an artificial intelligence device based on an artificial intelligence device to solve the technical problem of a single interaction scenario of the artificial intelligence device.

[0004] In a first aspect, an embodiment of the present application provides an interaction method based on an artificial intelligence device, which is applied to a server, and includes: obtaining a user input request, the user input request carrying user input information, responding to the user input request, determining a target interaction scenario corresponding to the user input information, the target interaction scenario being used to instruct a pre-trained target large language model to interact with the artificial intelligence device using a target interaction mode, the monitoring task represented by the user input information being a target monitoring task, the target monitoring task being composed of multiple execution elements, responding to the target interaction scenario as a monitoring scenario, calling the target large language model according to the element type of the execution element to parse the user input information, obtaining a target control parameter corresponding to the execution element, combining multiple target control parameters to obtain a target control table, and sending the target control table to the artificial intelligence device so that the artificial intelligence device performs a monitoring operation according to the target control table.

[0005] First of all, based on user input information, the embodiment of the present application can match the target interaction scene corresponding to the user input information in multiple interaction scenes for the user, thereby achieving the purpose of switching between multiple interaction scenes, which can effectively meet the user's switching needs for multiple interaction scenes.

[0006] Secondly, under the premise that the target interaction scenario is a monitoring scenario, since the user input information can be in any form of expression and can express any monitoring requirements, the embodiment of the present application can automatically match the target monitoring task for the artificial intelligence device based on the target large language model based on the user input information arbitrarily input by the user. In addition, the target large language model is good at refining and summarizing language compared to conventional neural network models. Therefore, the embodiment of the present application can match the artificial intelligence device with an accurate and reliable target monitoring task based on the target large language model.

[0007] Finally, the embodiment of the present application splits the target monitoring task into multiple execution elements, and matches the target control parameters for each execution element based on the target large language model. This is conducive to the artificial intelligence device calling the corresponding hardware to perform monitoring operations according to the corresponding target control parameters based on various target control parameters. In this way, the embodiment of the present application is equivalent to splitting the target monitoring task into target control parameters that the hardware of the artificial intelligence device can understand, so that the artificial intelligence device can reliably and accurately perform the target monitoring task.

[0008] Optionally, responding to the user input request and determining a target interaction scenario corresponding to the user input information includes: responding to the user input request, calling the target large language model to parse the user input information, obtaining candidate interaction scenarios corresponding to the user input information, and determining the target interaction scenario according to the type of the candidate interaction scenario.

[0009] Optionally, the candidate interaction scenarios include chat scenarios or monitoring scenarios, and determining the target interaction scenario according to the type of the candidate interaction scenarios includes: in response to the type of the candidate interaction scenario being a chat scene, determining the target interaction scenario to be a chat scene; or, in response to the type of the candidate interaction scenario being a monitoring scene, performing a monitoring confirmation operation.

[0010] The embodiments of the present application can realize at least two interaction scenarios, and automatically select the corresponding interaction scenario based on the analysis of user input information by the target large language model, thereby enriching the interaction mode of the artificial intelligence device and solving the problem of fixed and single interaction scenarios of the artificial intelligence device.

[0011] Optionally, the execution of the monitoring confirmation operation includes: determining the interaction scene of the last input information, the last input information being input information that is before the user input information and adjacent to the user input information; in response to the interaction scene of the last input information being a monitoring scene, determining the target interaction scene to be a monitoring scene; or, in response to the interaction scene of the last input information being a chat scene, controlling the artificial intelligence device to present a mode confirmation page, the mode confirmation page being used to prompt the user whether he needs to enter the monitoring mode.

[0012] The embodiments of the present application can reduce the erroneous analysis of user input information that leads to direct entry into monitoring mode from chat mode, interrupting the fluency of user chats, avoiding misoperations and unnecessary interference, and improving the reliability and availability of artificial intelligence devices.

[0013] Optionally, calling the target large language model according to the element type of the execution element to parse the user input information to obtain the target control parameters corresponding to the execution element includes: acquiring a prompt template library, the prompt template library including multiple types of prompt word templates; traversing the prompt word template corresponding to the element type of the execution element in the prompt template library as the target prompt word template; inputting the target prompt word template and the user input information into the target large language model for parsing to obtain the target control parameters corresponding to the execution element.

[0014] Optionally, the execution element includes a monitoring object, the target prompt word template is an object prompt word template, and the inputting the target prompt word template and the user input information into the target large language model for parsing to obtain the target control parameter corresponding to the execution element includes: inputting the object prompt word template and the user input information into the target large language model for parsing to obtain the monitoring object pointed to by the user input information; obtaining a model library, the model library including multiple types of lightweight small models; searching in the model library whether there is a target lightweight small model corresponding to the monitoring object, the target lightweight small model being one type of lightweight small model among the multiple types of lightweight small models; If it exists, then generate download link information corresponding to the target lightweight small model, the download link information is the target control parameter, wherein the artificial intelligence device can download the target lightweight small model to the local for storage according to the download link information, and detect in real time locally whether the picture taken by the artificial intelligence device contains the monitored object based on the target lightweight small model; If not, cloud detection information is generated, and the cloud detection information is the target control parameter, wherein the artificial intelligence device responds to the cloud detection information and transmits the real-time captured picture to the server, so that the server detects whether the monitored object exists in the picture.

[0015] The embodiment of the present application does not require the artificial intelligence device to save various detection models corresponding to various monitoring tasks. The artificial intelligence device only needs to download the target lightweight small model corresponding to the target monitoring task from the server to the local for storage. During the real-time monitoring process, the artificial intelligence device can detect the real-time image based on the target lightweight small model. If the real-time image contains a monitoring object corresponding to the target monitoring task, the artificial intelligence device can transmit the real-time image to the server for further detection. Therefore, the artificial intelligence device provided in the embodiment of the present application does not need to allocate more memory and computing power to complete the target monitoring task, and performs preliminary screening of the real-time image. After passing the screening, it can be transmitted to the server for further detection. This can avoid frequently transmitting real-time images that contain or do not contain monitoring images to the server for detection, thereby improving the efficiency of task monitoring.

[0016] Optionally, combining the plurality of target control parameters to obtain a target control table comprises: combining all the target control parameters to obtain a candidate control table; detecting whether the candidate control table has recorded the target control parameters of all the execution elements; If so, controlling the artificial intelligence device to present summary content about the target monitoring task, and determining a target control table according to the content operation information returned by the artificial intelligence device; If not, parameter missing information is generated, and the parameter missing information is used to prompt the user to supplement the associated information with the missing element, and the missing element is the execution element that lacks the target control parameter.

[0017] Optionally, determining the target control table based on the content operation information returned by the artificial intelligence device includes: in response to the content operation information being information indicating that the summary content is correct, determining the candidate control table as the target control table; or, in response to the content operation information being information indicating that the summary content is incorrect, generating modification prompt information, the modification prompt information being used to prompt the user to re-provide user input information.

[0018] The embodiments of the present application can avoid situations such as erroneous monitoring results, unnecessary waste of resources or monitoring omissions that may be caused by executing monitoring tasks based on erroneous information, and improve the accuracy and reliability of artificial intelligence equipment.

[0019] Optionally, after controlling the artificial intelligence device to present the modification prompt information, it also includes: obtaining a parameter modification request transmitted by the artificial intelligence device, the parameter modification request carrying parameter modification information; responding to the parameter modification request, calling the target large language model to parse the parameter modification information, and obtaining a parameter modification object and modification parameters corresponding to the parameter modification object; traversing the target control parameters corresponding to the parameter modification object in the candidate control table as control parameters to be modified; updating the control parameters to be modified to modification parameters corresponding to the parameter modification object, and obtaining an updated candidate control table. In a monitoring scenario, the embodiment of the present application can flexibly and randomly modify the target monitoring task and the target control parameters corresponding to the target monitoring task based on the target large language model, so that any monitoring needs of the user can be met, which is conducive to improving the user experience.

[0020] In a second aspect, an embodiment of the present application provides a server comprising a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the server implements the above-mentioned interaction method based on artificial intelligence devices.

[0021] In the third aspect, an embodiment of the present application provides an artificial intelligence device for communicating with the server, wherein the artificial intelligence device includes multiple types of hardware modules and a controller, and the controller is electrically connected to the multiple types of hardware modules respectively, wherein the controller includes a memory and a processor, and the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the artificial intelligence device performs the following operations: parses the target control parameters corresponding to each of the execution elements from the target control table, and calls the hardware module matching the target control parameters to perform monitoring operations according to the target control parameters.

[0022] Optionally, the target control parameters include download link information, the hardware module includes a camera module and a wireless communication module, and the controller is configured to: control the wireless communication module to download the target lightweight model to the local for storage according to the download link information; and detect in real time locally whether there is a monitored object in the picture taken by the camera module based on the target lightweight model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 A schematic diagram of the system architecture of a human-computer interaction system provided in an embodiment of the present application; Figure 2 A schematic diagram of a voice interaction between a user and a human-computer interaction system provided in an embodiment of the present application; Figure 3 A schematic diagram of the system architecture of another human-computer interaction system provided in an embodiment of the present application; Figure 4 A flowchart of an interaction method based on an artificial intelligence device provided in an embodiment of the present application; Figure 5 A schematic diagram of interaction between a server and an artificial intelligence device based on a monitoring scenario provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of a mode confirmation page provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of a memory module provided in an embodiment of the present application; Figure 8 It is a structural schematic diagram of an artificial intelligence device provided in an embodiment of the present application; Fig. 9 It is a schematic diagram of an interactive device based on an artificial intelligence device provided in an embodiment of the present application; Fig.10 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0026] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0027] In addition to discovering the related technical problems of the above-mentioned related technologies, the inventors also discovered the following problems: 1. As mentioned above, the interaction modes of home devices provided by related technologies are relatively simple, either not supporting chat mode or monitoring mode, and cannot meet users' scenario requirements for multiple interaction modes.

[0028] 2. The home devices provided by the relevant technology can complete the monitoring task, wherein the monitoring task needs to be configured in the home devices in advance. Specifically, the home devices need to save object detection models and behavior detection models of various object categories, such as the object detection model of "cat", the object detection model of "dog", the object detection model of "person", the object detection model of "fire", the behavior detection model of "fighting", the behavior detection model of "drinking water", the behavior detection model of "sleeping", etc. Therefore, the home devices need to consume a large amount of memory and computing power to support the storage and operation of the object detection models and behavior detection models of the above-mentioned various object categories.

[0029] 3. In order to realize monitoring, the user needs to operate the home device in advance and select a monitoring task from various monitoring tasks to configure the home device, wherein the monitoring tasks here are fixed and constructed in advance. For example, the monitoring library of the home device stores a variety of monitoring tasks, and one of the monitoring tasks is "Cats fight, notify me". When the user needs to monitor whether the cats at home are fighting, the user needs to operate the home device and select the monitoring task "Cats fight, notify me". At this time, the home device monitors the cat according to the established mode corresponding to the monitoring task "Cats fight, notify me". Specifically, the home device calls the camera to take a picture and inputs the picture into the object detection model. If the object detection model outputs the object as "cat", the behavior detection model detects whether the behavior of the "cat" is a fighting behavior. If so, the home device calls the wireless communication module to notify the user of the fact that "cats are fighting".

[0030] It is understandable that since the monitoring task is a fixed task set in advance, the designer clearly knows which hardware needs to be called to perform which operations in order to complete the monitoring task. At the same time, the designer can design which hardware needs to be called to perform which operations when the home device performs the monitoring task according to the above rules.

[0031] The above monitoring method is relatively fixed. When the user's monitoring needs change, and there is no monitoring task matching the user's monitoring needs in the monitoring library, the home device cannot match the user's monitoring needs in a timely manner.

[0032] 4. Currently, large language models are widely used in various technical fields. However, when many related technologies use large language models, most of them use the image recognition interface or dialogue interface of the large language model, and the application scope is relatively narrow.

[0033] First, the embodiments of the present application support at least two interaction scenarios, such as chat scenarios and monitoring scenarios, so as to meet the user's switching needs for multiple interaction scenarios of artificial intelligence devices.

[0034] Secondly, the embodiment of the present application does not require the artificial intelligence device to save various detection models corresponding to various monitoring tasks. The artificial intelligence device only needs to download the target lightweight small model corresponding to the target monitoring task from the server to the local for storage. During the real-time monitoring process, the artificial intelligence device can detect the real-time image based on the target lightweight small model. If the real-time image contains a monitoring object corresponding to the target monitoring task, the artificial intelligence device can transmit the real-time image to the server for further detection. Therefore, the artificial intelligence device provided in the embodiment of the present application does not need to allocate more memory and computing power to complete the target monitoring task, and performs a preliminary screening of the real-time image. After the screening, it can be transmitted to the server for further detection. This can avoid frequently transmitting real-time images that contain or do not contain monitoring images to the server for detection, thereby improving the efficiency of task monitoring.

[0035] Again, the embodiment of the present application is based on the target large language model, which can sort out the target monitoring task from the user input information of any expression form and any monitoring requirement, effectively understand the user's monitoring intention, and thus automatically match the target monitoring task for the artificial intelligence device. In addition, the target large language model is good at refining and summarizing language compared to conventional neural network models. Therefore, the embodiment of the present application can match the artificial intelligence device with accurate and reliable target monitoring tasks based on the target large language model.

[0036] Finally, the embodiment of the present application splits the target monitoring task into multiple execution elements, uses a multi-level prompt word project to sort out the target control parameters corresponding to the execution elements from the user input information, and establishes a hardware control flow based on various target control parameters and the hardware of the artificial intelligence device, so that the artificial intelligence device calls the corresponding hardware based on various target control parameters to perform monitoring operations according to the corresponding target control parameters. In this way, the embodiment of the present application is equivalent to splitting the target monitoring task into target control parameters that the hardware of the artificial intelligence device can understand, so that the artificial intelligence device can reliably and accurately perform the target monitoring task.

[0037] The following is a human-computer interaction system provided by the present application. Figure 1 The human-computer interaction system 100 includes an artificial intelligence device 11 and a server 12, and the artificial intelligence device 11 and the server 12 are communicatively connected.

[0038] The artificial intelligence device 11 can obtain user input information, encapsulate the user input information into a user input request, and send the user input request to the server 12. The server 12 calls the target large language model 13 to parse the user input information to determine the interaction scenario and target control parameters, generates a target control table according to the interaction scenario and target control parameters, and sends the target control table to the artificial intelligence device 11, so that the artificial intelligence device 11 performs the target monitoring task according to the target control table.

[0039] In some embodiments, see Figure 2 The artificial intelligence device 11 is equipped with a microphone, which is used to collect voice information. For example, the user inputs a voice message of "cats fighting, notify me" to the artificial intelligence device 11, and the artificial intelligence device 11 encapsulates the voice message as user input information into a user input request, and sends the user input request to the server 12.

[0040] In other embodiments, the artificial intelligence device 11 converts voice information into user input information such as “cats are fighting, notify me” based on an STT algorithm (Speech to Text), then encapsulates the user input information into a user input request, and sends the user input request to the server 12.

[0041] In some embodiments, see Figure 3 The human-computer interaction system 100 further includes an intelligent mobile terminal 14, which is in communication with the server 12, wherein the intelligent mobile terminal 14 is bound to the artificial intelligence device 11, and the server 12 stores the binding relationship between the intelligent mobile terminal 14 and the artificial intelligence device 11. The intelligent mobile terminal 14 can obtain user input information, encapsulate the user input information into a user input request, and send the user input request to the server 12.

[0042] In other embodiments, the smart mobile terminal 14 is communicatively connected to the server 12 , and the smart mobile terminal 14 converts the voice information into user input information such as “cats are fighting, notify me” based on the STT algorithm (Speech to Text), then encapsulates the user input information into a user input request, and sends the user input request to the server 12 .

[0043] As another aspect of this embodiment, the following embodiment of the present application provides an interaction method based on artificial intelligence for use in a human-computer interaction system.

[0044] See also Figure 4 The artificial intelligence-based interaction method includes step S41, wherein step S41 is: obtaining a user input request, the user input request carries user input information.

[0045] In this step, the user input request is a request for triggering the server and the artificial intelligence device to perform interactive operations. The user input information is included in the user input request and is transmitted in the form of a network data packet after being encapsulated by the network protocol.

[0046] The user input information is the input information collected by the artificial intelligence device. As mentioned above, in some embodiments, the data type of the user input information is a voice type, for example, the artificial intelligence device collects the voice information input by the user and uses the voice information as the user input information. In other embodiments, the data type of the user input information is a text type, for example, the artificial intelligence device converts the voice information into text information and uses the text information as the user input information.

[0047] The artificial intelligence device is configured with a button and a speaker. When the user presses the button of the artificial intelligence device, the artificial intelligence device responds to the pressing operation received by the button and controls the speaker to broadcast a default voice message. For example, the default voice message is: "If you want to perform a monitoring task, you can say: 'Monitor the arrival of the courier.' Alternatively, the artificial intelligence device is configured with a display screen. The artificial intelligence device responds to the pressing operation received by the button and controls the display screen to present a default display message. For example, the default display message is: "If you want to perform a monitoring task, you can say: 'Monitor the arrival of the courier.'

[0048] The user speaks the voice information according to the instructions of the default voice information or the default display information, and the artificial intelligence device then collects the voice information.

[0049] It is understandable that user input information can also be collected and transmitted to the server through a smart mobile terminal, or forwarded to the server through an artificial intelligence device.

[0050] It can also be understood that in some embodiments, the user input information may be information collected when the artificial intelligence device enters the chat mode, and the server processes the user input information based on the target large language model and returns the chat response content to the artificial intelligence device, thereby realizing the dialogue between the artificial intelligence device and the user. In some embodiments, the user input information may be information collected when the artificial intelligence device enters the monitoring mode. At this time, the server determines the intention of the user input information based on the target large language model. If the intention requires the artificial intelligence device to continue monitoring, the server controls the artificial intelligence device to monitor according to the monitoring task corresponding to the user input information. If the intention requires the artificial intelligence device to chat, the server controls the artificial intelligence device to output the corresponding chat content.

[0051] It can also be understood that for the same monitoring task, the user input information can be expressed in various forms. For example, for the same monitoring task of "cats fighting, notify me", the user input information is "Please monitor whether my little good cats fight. If they fight, please notify me by text message", or the user input information is "If the little greedy cats fight, notify me."

[0052] It is also understandable that the user can output the user input information according to his own monitoring needs. For example, the user input information is "If there is a cat fight, notify me", or the user input information is "If there is someone walking in the house between 9:30 am and 7:30 pm, please notify me."

[0053] In some embodiments, obtaining a user input request includes the following steps: based on the UDP (User Datagram Protocol) protocol, monitoring whether the artificial intelligence device sends a user input request; if so, saving the user input request sent by the artificial intelligence device; if not, the server continues to monitor the user input request.

[0054] After obtaining the user input request in step S41, Figure 4 As shown, the embodiment of the present application also includes step S42, and step S42 is: responding to the user input request, determining the target interaction scene corresponding to the user input information.

[0055] The target interaction scene is an interaction scene that matches the user input information. In the embodiment of the present application, the interaction scene that matches the user input information is determined as the target interaction scene in at least two preset interaction scenes. As mentioned above, the interaction scene includes a chat scene, a monitoring scene, and the like.

[0056] The target interaction scenario is used to instruct the pre-trained target large language model to use the target interaction mode to interact with the artificial intelligence device. When the target interaction scenario is a chat scenario, the server instructs the target large language model to use the chat mode to interact with the artificial intelligence device. Specifically, the server responds to the user input information transmitted by the artificial intelligence device, calls the target large language model to process the user input information, obtains the chat content, returns the chat content to the artificial intelligence device, and the artificial intelligence device presents the chat content to the user.

[0057] When the target interaction scenario is a monitoring scenario, the server instructs the target large language model to interact with the artificial intelligence device in monitoring mode. Specifically, the server responds to the user input information transmitted by the artificial intelligence device and enters the monitoring mode. In monitoring mode, the server receives the pictures collected by the artificial intelligence device in real time, and calls the target large language model to parse the pictures to obtain the parsing results. If the parsing results match the target monitoring task corresponding to the monitoring mode, the server performs the corresponding operation according to the monitoring processing results of the target monitoring task. The embodiments of the present application support at least two interaction scenarios, such as chat scenarios and monitoring scenarios, so as to meet the user's switching needs for multiple interaction scenarios of artificial intelligence devices.

[0058] The target large language model is a model obtained by fine-tuning the large language model using the prompt word method. Specifically, the embodiment of the present application collects various types of training data to fine-tune the large language model, and the training data may include: collecting a large amount of video and audio materials from various monitoring devices (such as cameras, microphones). These materials can cover different scenes, such as indoor and outdoor environments, day and night, different weather conditions, etc., to ensure that the model can adapt to diverse monitoring scenarios. For example, collect surveillance videos in shopping malls, including customers' shopping behaviors, employees' work scenes, and videos of possible security incidents; collect conversation texts from various sources, including online customer service chat records, social media conversations, forum discussions, etc. These conversation data should contain communication content of different topics and styles, such as technical support conversations, emotional communication conversations, product recommendation conversations, etc. For example, obtain conversations about product consultation, returns and exchanges from customer service chat records of e-commerce platforms; collect questions and answers on knowledge question and answer platforms. These data can help the model learn how to answer various knowledge questions and improve the ability to provide knowledge in chat mode.

[0059] The monitoring task represented by the user input information is the target monitoring task, for example, the target monitoring task represented by the user input information is “notify me if cats fight”. It is understandable that, as mentioned above, for the same monitoring task, the user input information can be expressed in various forms.

[0060] The target monitoring task is composed of multiple execution elements. The execution elements are the elements that constitute the target monitoring task. The execution elements include monitoring objects, monitoring behaviors, monitoring time, monitoring frequency, alarm methods, etc. For example, monitoring objects include people, pets, cars, etc. Monitoring behaviors include fighting, stealing, sleeping, falling, etc. Monitoring time includes all day, 9 am to 10 am, etc. Monitoring frequency includes monitoring once every hour. Monitoring alarm methods include phone notification, SMS notification, voice notification, flashing light notification, etc.

[0061] For example, for the target monitoring task "Cats are fighting, please notify me by SMS", the execution elements that constitute the target monitoring task include: {monitoring object, cat}, {monitoring behavior, fighting} and {alarm method, SMS}.

[0062] It is understandable that the execution elements included in different monitoring tasks may be the same or different. For example, for the target monitoring task "If a fight occurs, please notify me by SMS", the execution elements that constitute the target monitoring task include: {monitoring behavior, fighting} and {alarm method, SMS}. For another example, for the target monitoring task "If a cat appears at home, please notify me by SMS", the execution elements that constitute the target monitoring task include: {monitoring object, cat} and {alarm method, SMS}. For another example, for the target monitoring task "If a cat appears at home, please use voice to drive it away", the execution elements that constitute the target monitoring task include: {monitoring object, cat} and {alarm method, voice}.

[0063] It can also be understood that a monitoring task may include all of the above-mentioned execution elements, or may include only some of the above-mentioned execution elements.

[0064] It can also be understood that the embodiments of the present application can define each monitoring task as having all the above-mentioned execution elements. When the server calls the target large language model and cannot parse all the execution elements from the user input information, that is, the target monitoring task does not have all the execution elements, the missing execution elements are configured to the target monitoring task by default. For example, for the target monitoring task "If a fight occurs, please notify me by SMS", the execution elements that constitute the target monitoring task include: {monitoring behavior, fighting} and {alarm method, SMS}. Therefore, the target monitoring task lacks execution elements such as monitoring time and monitoring frequency. The embodiments of the present application configure the execution elements such as monitoring time and monitoring frequency to the target monitoring task by default.

[0065] The embodiment of the present application is based on the target large language model, which can sort out the target monitoring task from the user input information of any expression form and any monitoring requirement, effectively understand the user's monitoring intention, and thus automatically match the target monitoring task for the artificial intelligence device. In addition, the target large language model is good at refining and summarizing language compared to the conventional neural network model. Therefore, the embodiment of the present application can match the artificial intelligence device with accurate and reliable target monitoring tasks based on the target large language model.

[0066] After the target interaction scene is determined in step S43, Figure 4 As shown, the embodiment of the present application also includes step S43, which is: in response to the target interaction scenario being a monitoring scenario, calling the target large language model according to the element type of the execution element to parse the user input information, and obtaining the target control parameters corresponding to the execution element.

[0067] The target control parameters are the hardware content that can be executed by various types of hardware of artificial intelligence devices. If the execution element is the monitoring object, the target control parameters are the camera + model link information (optional); if the execution element is the monitoring behavior, the target control parameters are the camera + wireless communication module + model access link + prompt parameter; if the execution element is the monitoring time or monitoring frequency, the target control parameters are the timer + hardware parameters; if the execution element is the alarm method, the target control parameters are the speaker or RGB light or display or wireless communication module.

[0068] In this step, the server calls the target large language model according to the element type of the execution element to parse the user input information and obtain the target control parameters corresponding to the execution element. For example, the server calls the target large language model according to the monitoring object to parse the user input information and obtain the target control parameters corresponding to the monitoring object. Similarly, the server calls the target large language model according to the monitoring behavior to parse the user input information and obtain the target control parameters corresponding to the monitoring behavior, and so on, which will not be repeated here.

[0069] For example, if the execution elements of a monitoring task include monitoring objects, monitoring behaviors, monitoring time, etc., when the target monitoring task corresponding to the user input information is "If I monitor a cat fight, please notify me", the server calls the target large language model and parses the user input information according to the element type of the execution element {monitoring object, monitoring behavior, monitoring time} to obtain the corresponding target control parameters.

[0070] It is understandable that when the server calls the target large language model and cannot parse the target control parameters corresponding to the relevant execution elements from the user input information, the embodiment of the present application matches the default control parameters for such execution elements and uses the default control parameters as the target control parameters corresponding to such execution elements. For example, when the target monitoring task corresponding to the user input information is "Monitoring cats fighting, please notify me", the embodiment of the present application calls the target large language model according to the execution element "monitoring time" and cannot parse the specific value of the monitoring time from the user input information. In this case, the embodiment of the present application matches the default control parameter "timer + all day" for the execution element "monitoring time", that is, the control parameter "timer + all day" is the target control parameter.

[0071] As mentioned above, the server calls the target large language model to automatically parse the user input information and automatically generates the target control parameters corresponding to each execution element.

[0072] In other embodiments, the target control parameters corresponding to the execution elements can be input in other ways, such as by APP task arrangement. The user can arrange the parameters of the monitoring task to be issued in the APP, combine the target control parameters corresponding to each execution element in the APP, and send the target control parameters corresponding to each execution element to the server for storage.

[0073] In other embodiments, the target control parameters corresponding to the execution elements can also be input by editing the business on the Node-Red page. Node-Red is a process-based visual programming tool, mainly used for the development of the Internet of Things and automated processes. Node-Red builds processes based on nodes.

[0074] After the target control parameters are obtained in step S44, Figure 4 As shown, the embodiment of the present application also includes step S44, and step S44 is: combining multiple target control parameters to obtain a target control table.

[0075] In this step, the server uses a json data structure to combine multiple target control parameters to obtain a target control table. The target control table is used to instruct the artificial intelligence device to call the corresponding hardware to perform monitoring operations according to specific hardware parameters. The embodiment of the present application combines the execution elements and the target control parameters to output the target control table, please refer to Table 1: Table 1

[0076] The embodiment of the present application splits the target monitoring task into a "task flow" composed of abstract hardware capabilities. The abstract hardware capabilities include: camera, microphone, AI inference engine, timer, wireless communication module, display, RGB light, speaker, etc. In this way, the artificial intelligence device can better understand the "target monitoring task" based on the target control table, and can accurately and reliably call the execution hardware to perform the target monitoring task.

[0077] After the target control table is obtained in step S45, Figure 4 As shown, the embodiment of the present application also includes step S45, which is: sending a target control table to the artificial intelligence device so that the artificial intelligence device performs a monitoring operation according to the target control table.

[0078] The embodiment of the present application splits the target monitoring task into multiple execution elements, uses a multi-level prompt word project to sort out the target control parameters corresponding to the execution elements from the user input information, and establishes a hardware control flow based on the various target control parameters and the hardware of the artificial intelligence device, so that the artificial intelligence device calls the corresponding hardware to perform monitoring operations according to the corresponding target control parameters based on the various target control parameters. In this way, the embodiment of the present application is equivalent to splitting the target monitoring task into target control parameters that the hardware of the artificial intelligence device can understand, so that the artificial intelligence device can reliably and accurately perform the target monitoring task.

[0079] In order to elaborate on the interaction method based on artificial intelligence equipment in the embodiment of the present application, the embodiment of the present application combines Figure 5 The interaction between the artificial intelligence device 11 and the server 12 based on the monitoring scenario is described in detail as follows. The interaction method based on the artificial intelligence device includes the following steps: S51: The artificial intelligence device 11 sends a user input request to the server 12.

[0080] S52: The server 12 responds to the user input request and determines that the candidate interaction scenario is a monitoring scenario.

[0081] S53: The server 12 sends a control command to the artificial intelligence device 11.

[0082] S54: The artificial intelligence device 11 responds to the control command and presents a mode confirmation page.

[0083] S55: The artificial intelligence device 11 sends a monitoring confirmation request to the server 12.

[0084] S56: The server 12 responds to the monitoring confirmation request, parses the user input information, obtains the target control parameters, combines the target control parameters, obtains a candidate target control table, and obtains summary content.

[0085] S57: The server 12 sends a control display command to the artificial intelligence device 11.

[0086] S58: The artificial intelligence device 11 responds to the control display command and presents summary content.

[0087] S59: The artificial intelligence device 11 sends content operation information to the server 12.

[0088] S510: The server 12 determines the target control table according to the content operation information.

[0089] S511: The server 12 sends a target control table to the artificial intelligence device 11.

[0090] S512: The artificial intelligence device 11 executes the target monitoring task according to the target control table.

[0091] The embodiment of the present application can parse the user input information through the target large language model, obtain the target control parameters corresponding to the execution elements of the monitoring task, combine multiple target control parameters to obtain the target control table, and send the target control table to the artificial intelligence device so that the artificial intelligence device performs the monitoring operation according to the target control table. On the one hand, the target control table contains some download link information of lightweight small models. The artificial intelligence device will download the corresponding lightweight small model to the local, so that the artificial intelligence device can detect the captured image locally and complete the corresponding monitoring task. There is no need to frequently access the server, which reduces the demand for computing power and reduces the operating cost. On the other hand, the embodiment of the present application can parse the user input information through the target large language model, split a monitoring task into multiple fine-grained tasks that can be executed by each hardware of the artificial intelligence device, and finally combine them into a complete monitoring task. It can be seen that the flexibility of the monitoring task setting of the embodiment of the present application is relatively high.

[0092] In some embodiments, determining the target interaction scenario includes the following steps: responding to a user input request, calling a target large language model to parse the user input information, obtaining a candidate interaction scenario corresponding to the user input information, and determining the target interaction scenario according to the type of the candidate interaction scenario.

[0093] For example, the target large language model can realize at least two interaction scenarios. The target large language model parses the user input information to obtain candidate interaction scenarios, such as a chat scenario and a monitoring scenario, and responds to one of the interaction scenarios and determines it as the target interaction scenario. If the type of the candidate interaction scenario is a chat scenario, the target interaction scenario is determined to be a chat scenario. If the type of the candidate interaction scenario is a monitoring scenario, a monitoring confirmation operation is performed.

[0094] The embodiments of the present application can realize at least two interaction scenarios, and automatically select the corresponding interaction scenario based on the analysis of user input information by the target large language model, thereby enriching the interaction mode of the artificial intelligence device and solving the problem of fixed and single interaction scenarios of the artificial intelligence device.

[0095] In some embodiments, performing a monitoring confirmation operation includes the following steps: determining the interaction scenario of the last input information, where the last input information is input information that is before the user input information and adjacent to the user input information; if the interaction scenario in response to the last input information is a monitoring scenario, then determining that the target interaction scenario is a monitoring scenario; if the interaction scenario in response to the last input information is a chat scene, controlling the artificial intelligence device to present a mode confirmation page, and the mode confirmation page is used to prompt the user whether he needs to enter the monitoring mode.

[0096] For example, when the type of the candidate interaction scenario is a monitoring scenario, the most recent input information that is consistent with the user's input information in the historical session record is called as the last input information. If the last input information is not a chat content, the monitoring scenario will be entered; if the last input information is a chat content, the artificial intelligence device will respond with a fixed prompt: "Do you want to enter monitoring mode? You can reply yes or no", or the display screen will present a mode confirmation page to prompt the user whether to enter monitoring mode.

[0097] The embodiments of the present application can reduce the erroneous analysis of user input information that leads to direct entry into monitoring mode from chat mode, interrupting the fluency of user chats, avoiding misoperations and unnecessary interference, and improving the reliability and availability of artificial intelligence devices.

[0098] In some embodiments, see Figure 6 The mode confirmation page 600 includes a confirmation button 61 and a cancel button 62. After the artificial intelligence device presents the mode confirmation page, the interaction method also includes the following steps: responding to a monitoring confirmation request sent by the artificial intelligence device, determining that the target interaction scene is a monitoring scene, wherein the monitoring confirmation request is generated by the artificial intelligence device in response to an input operation of the confirmation button 61; responding to a chat confirmation request sent by the artificial intelligence device, determining that the target interaction scene is a chat scene, wherein the chat confirmation request is generated by the artificial intelligence device in response to an input operation of the cancel button 62.

[0099] The embodiments of the present application can prompt the user to confirm the interaction scenario, so that the user clearly knows that the operation he is about to perform may change the settings of the artificial intelligence device, so that the user can better use the artificial intelligence device according to his needs, thereby enhancing the usability and ease of use of the artificial intelligence device.

[0100] In some embodiments, calling the target large language model to parse the user input information according to the element type of the execution element to obtain the target control parameter corresponding to the execution element includes the following steps: S431: Acquire a prompt template library, where the prompt template library includes multiple types of prompt word templates.

[0101] S432: Traverse the prompt word template corresponding to the element type of the execution element in the prompt template library as the target prompt word template.

[0102] S433: Input the target prompt word template and the user input information into the target large language model for parsing to obtain the target control parameters corresponding to the execution elements.

[0103] In S431, the prompt word template is a prompt file, which is used to guide the target large language model to generate expected output. The prompt word template can help users interact with the target large language model more effectively and improve the accuracy and relevance of the model generation results. The prompt template library organizes, classifies and stores prompt word templates of various types and uses so that the server can quickly find suitable prompt word templates in different application scenarios to meet its specific needs.

[0104] In S432, in some embodiments, the embodiment of the present application selects each execution element in order from all execution elements, and traverses the prompt word template corresponding to the element type of the execution element in the prompt template library as the target prompt word template.

[0105] In some other embodiments, the server obtains an element resolution request sent by the smart mobile terminal, the element resolution request carries at least one target element tag, the target element tag is used to indicate the element type of the execution element, the server determines the execution element corresponding to the target element tag as the target execution element, and traverses the prompt word template corresponding to the element type of the target execution element in the prompt template library as the target prompt word template. The element resolution request is generated by the user operating the smart mobile terminal.

[0106] In some embodiments, the APP of the smart mobile terminal provides a tag selection page, which includes multiple types of element tags. The user selects one or more element tags on the tag selection page, and the smart mobile terminal responds to the user's selection operation and transmits the selected element tag as the target element tag to the server. The embodiment of the present application can customize the execution elements that the server needs to parse, so as to improve the efficiency of the server in generating target control parameters.

[0107] In S433, the target large language model parses the user input information under the prompt of the target prompt word template to obtain the target control parameter corresponding to the execution element.

[0108] In some embodiments, the execution element includes a monitoring object, the target prompt word template is an object prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameters corresponding to the execution element includes the following steps: inputting the object prompt word template and the user input information into the target large language model for parsing, obtaining the monitoring object pointed to by the user input information, obtaining the model library, and searching in the model library whether there is a target lightweight small model corresponding to the monitoring object.

[0109] If there is a target lightweight model corresponding to the monitored object, a download link information corresponding to the target lightweight model is generated, and the download link information is a target control parameter, wherein the artificial intelligence device can download the target lightweight model to the local for storage according to the download link information, and detect whether there is a monitored object in the picture taken by the artificial intelligence device in real time based on the target lightweight model. If the artificial intelligence device detects in real time based on the target lightweight model that there is no monitored object in the picture taken by the artificial intelligence device, the artificial intelligence device does not need to send the picture taken by the artificial intelligence device to the server for detection. If the artificial intelligence device detects in real time based on the target lightweight model that there is a monitored object in the picture taken by the artificial intelligence device, the artificial intelligence device sends the picture taken by the artificial intelligence device to the server for detection.

[0110] If there is no target lightweight model corresponding to the monitored object, cloud detection information is generated, and the cloud detection information is the target control parameter. The artificial intelligence device responds to the cloud detection information and transmits the real-time captured picture to the server, so that the server detects whether there is a monitored object in the picture.

[0111] The model library is a collection of lightweight small models that store many different types and correspond to different objects. Compared with large language models, lightweight small models have smaller model sizes and lower computing resource requirements. Lightweight small models have lower requirements for hardware computing power (such as GPU memory and computing power) and memory during the reasoning (generating output) stage, which allows them to run on resource-constrained devices such as mobile devices (smartphones, tablets), edge computing devices, or low-performance computers. At runtime, lightweight small models take up less storage space and generate results relatively quickly, and can complete reasoning tasks in a shorter time. Unlike the general language processing capabilities of large language models, lightweight small models are designed to efficiently complete specific tasks, and are targeted and efficient.

[0112] The cloud detection information is used to trigger the artificial intelligence device to request the server to perform related operations. Please refer to Table 1. The cloud detection information includes model link information, which is the link information for the artificial intelligence device to request the server to call the target large language model. Specifically, after receiving the cloud detection information, the artificial intelligence device will transmit the pictures taken in real time to the server according to its instructions. The cloud detection information transfers the execution subject of the detection task from the local to the server. Because there is no corresponding lightweight small model to complete the detection task locally, the server calls the target large language model with the help of the server's capabilities to detect whether the monitored object of interest exists in the received picture.

[0113] For example, if the object prompt word template and user input information are input into the target large language model for parsing, and the monitored object is a cat, the model library will be searched to see if there is a lightweight small model specifically for cats. For example, this lightweight small model of cats may be specifically used to identify cats, cat behaviors, characteristics, etc. Compared with the large language model, it will be lighter and more focused on specific cat-related tasks, and its computing efficiency may be higher, making it more suitable for performing some cat-related monitoring and analysis work locally.

[0114] If there is a lightweight small model of a cat, the server will generate a download link information accordingly. This download link is a way to obtain the lightweight small model. Through it, the corresponding model file can be obtained from the storage location of the model library or the server. At this time, the photos taken by the artificial intelligence device can be detected locally. If there is no lightweight small model of a cat, cloud detection information will be generated. The artificial intelligence device can respond to the cloud detection information, so that the artificial intelligence device transmits the real-time captured picture to the server, and the server calls the target large language model to detect whether the picture contains the monitored object.

[0115] The artificial intelligence device provided in the embodiment of the present application can download the corresponding lightweight small model locally, so that the artificial intelligence device can detect the captured images locally and complete the corresponding monitoring tasks. There is no need to frequently access the server, which reduces the demand for computing power and reduces operating costs.

[0116] In some embodiments, the execution element includes monitoring behavior, the target prompt word template is a behavior prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameters corresponding to the execution element includes the following steps: inputting the behavior prompt word template and the user input information into the target large language model for parsing, obtaining the monitoring behavior pointed to by the user input information, generating model link information and behavior prompt words corresponding to the monitoring behavior, the model link information and the behavior prompt words are both target control parameters, the model link information is used to instruct the artificial intelligence device to call the target large language model, and the behavior prompt words are used to instruct the target large language model to detect whether there is monitoring behavior in the pictures taken by the artificial intelligence device.

[0117] The model link information is used to instruct the artificial intelligence device to request the server to call the target large language model and use the capabilities of the target large language model to complete the target monitoring task. It provides a "path" for the artificial intelligence device so that the artificial intelligence device can accurately request the server to start the target large language model to perform operations.

[0118] In some embodiments, the execution element includes monitoring time, the target prompt word template is a time prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameter corresponding to the execution element includes the following steps: the time prompt word template and the user input information are input into the target large language model for parsing, and the monitoring time range pointed to by the user input information is obtained, and the monitoring time range is the target control parameter.

[0119] In some embodiments, the execution element includes a monitoring frequency, the target prompt word template is a frequency prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameter corresponding to the execution element includes the following steps: the frequency prompt word template and the user input information are input into the target large language model for parsing, and the monitoring frequency pointed to by the user input information is obtained, and the monitoring frequency is the target control parameter.

[0120] In some embodiments, the execution element includes a local alarm mode, the target prompt word template is a local alarm prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameter corresponding to the execution element includes the following steps: the local alarm prompt word template and the user input information are input into the target large language model for parsing, and the local alarm mode pointed to by the user input information is obtained, and the local alarm mode is the target control parameter.

[0121] In the embodiments of the present application, there are multiple ways to implement local alarms. For example, in the case of a sound alarm, when an alarm condition is triggered, the artificial intelligence device may emit a buzzer sound or play the corresponding alarm information through a speaker; in the case of a light alarm, different alarm states are indicated by flashing lights of different colors and frequencies. For example, a fast flashing red indicator light may indicate that all contents of the monitoring task have been detected, while a slow flashing yellow indicator light may indicate that part of the monitoring task has been detected; in the case of a display alarm, the alarm text information is displayed on the display screen of the artificial intelligence device.

[0122] In some embodiments, the execution element includes a cloud alarm method, the target prompt word template is a cloud alarm prompt word template, the target prompt word template and the user input information are input into the target large language model for parsing, and obtaining the target control parameter corresponding to the execution element includes the following steps: the cloud alarm prompt word template and the user input information are input into the target large language model for parsing, and the cloud alarm method pointed to by the user input information is obtained, and the cloud alarm method is the target control parameter.

[0123] In the embodiment of the present application, cloud alarms can be implemented in a variety of ways, such as SMS notifications, email notifications, and smart mobile terminal app push notifications. When the alarm condition is triggered, the server sends an alarm notification to the user via the bound mobile phone number, email, and smart mobile terminal app.

[0124] As described above, the embodiment of the present application utilizes a multi-level prompt word project to sequentially call various prompt word templates to parse user input information, gradually narrowing down from larger and broader execution elements (such as monitoring objects) to smaller execution elements (such as monitoring frequencies), thereby being able to perform a smaller and more fine-grained analysis of the target monitoring task, and then obtain a comprehensive and reliable variety of target control parameters. In this way, the artificial intelligence device is based on a variety of target control parameters. When facing flexible, changeable and uncertain target monitoring tasks (as mentioned above, due to the variety of task requirements for user input information, the target monitoring tasks will also be diverse), it can effectively understand the "target monitoring task", thereby being able to accurately call various execution hardware to perform monitoring operations, making the artificial intelligence device more like an intelligent entity rather than a device that operates according to a preset fixed monitoring task.

[0125] In some embodiments, the target prompt word template and user input information are input into the target large language model for parsing to obtain the target control parameters corresponding to the execution elements, including the following steps: in response to the existence of historical input information corresponding to the historical monitoring task of the artificial intelligence device, the historical input information, the target prompt word template and the user input information are input into the target large language model for parsing to obtain the target control parameters corresponding to the execution elements; or, in response to the absence of historical input information corresponding to the historical monitoring task of the artificial intelligence device, the target prompt word template and the user input information are input into the target large language model for parsing to obtain the target control parameters corresponding to the execution elements.

[0126] Historical input information refers to the historical conversation content between the user and the artificial intelligence device before the user outputs the user input information. Historical input information can provide a comprehensive contextual background for parsing the target monitoring task. Historical input information may record common monitoring objects in a specific area, types of abnormal events that have occurred in the past, etc. When input into the target large language model together with the target prompt word template and user input information, the target large language model can better understand the overall picture of the entire monitoring scene, including past monitoring priorities and possible situations. Therefore, historical input information helps guide the target large language model to understand the user's expectations and intentions for monitoring tasks. The embodiments of the present application can improve the accuracy of the target large language model parsing, can accurately meet the user's requirements for monitoring tasks, and thus improve the performance of artificial intelligence devices.

[0127] In some embodiments, combining multiple target control parameters to obtain a target control table includes the following steps: S441: Combine all target control parameters to obtain a candidate control table.

[0128] S442: Check whether the candidate control table has recorded the target control parameters of all execution elements.

[0129] S443: If so, control the artificial intelligence device to present summary content about the target monitoring task, and determine the target control table according to the content operation information returned by the artificial intelligence device.

[0130] S444: If not, generate parameter missing information.

[0131] In S441, the candidate control table is a control table that has not been confirmed by the user. When the user determines that the candidate control table is correct, the embodiment of the present application updates the candidate control table to the target control table. When the user determines that there is a problem with the candidate control table and the user modifies and confirms it, the embodiment of the present application updates the candidate control table to the target control table.

[0132] In S442, in some embodiments, the embodiment of the present application uses each execution element of all execution elements as an index to sequentially traverse the candidate control table to see if the target control parameters of all execution elements have been recorded. In some embodiments, the embodiment of the present application detects whether the candidate control table has recorded the target control parameters of all target execution elements.

[0133] In S443, the summary content is a summary of the target monitoring task, and the summary content explains who the monitored object is, what the monitored behavior is, what the monitoring time range is, what the monitoring frequency is, and what the alarm method is.

[0134] In some embodiments, the embodiment of the present application presents the summary content through a display screen. In some embodiments, the embodiment of the present application presents the summary content in the form of voice broadcast.

[0135] Content operation information is information output by the artificial intelligence device in response to the user's confirmation operation on the summary content. Content operation information includes content confirmation information and content modification information. Content confirmation information is information indicating that the summary content is correct, and content modification information is information indicating that the summary content is wrong.

[0136] The artificial intelligence device collects content feedback information input by the user, encapsulates the content feedback information into a content confirmation request, and sends the content confirmation request to the server. The server parses the content confirmation request, obtains the content feedback information, and calls the target large language model to process the content feedback information. If the content feedback information indicates that the user confirms that the summary content is correct, the server generates content confirmation information. If the content feedback information indicates that the user needs to modify the summary content, the server generates content modification information.

[0137] In some embodiments, determining a target control table based on content operation information returned by an artificial intelligence device includes the following steps: in response to the content operation information being information indicating that the summary content is correct, determining a candidate control table as a target control table, or in response to the content operation information being information indicating that the summary content is incorrect, controlling the artificial intelligence device to present modification prompt information, the modification prompt information being used to prompt the user to re-provide user input information.

[0138] When user feedback indicates that the summary content is wrong, the server controls the artificial intelligence device to present modification prompt information, guiding the user to review and correct the user input information previously provided, so that the artificial intelligence device can perform the target monitoring task that matches the user's actual monitoring needs, or avoid erroneous monitoring results, unnecessary waste of resources or monitoring omissions that may result from executing monitoring tasks based on erroneous information, thereby improving the accuracy and reliability of the artificial intelligence device.

[0139] After controlling the artificial intelligence device to present the modification prompt information, the method also includes the following steps: obtaining a parameter modification request transmitted by the artificial intelligence device, the parameter modification request carrying the parameter modification information, responding to the parameter modification request, calling the target large language model to parse the parameter modification information, obtaining a parameter modification object and modification parameters corresponding to the parameter modification object, traversing the target control parameters corresponding to the parameter modification object in the candidate control table as the control parameters to be modified, updating the control parameters to be modified into the modification parameters corresponding to the parameter modification object, and obtaining an updated candidate control table.

[0140] The AI ​​device responds to the user's pressing of the button input, controls the microphone to collect the parameter modification information input by the user, and encapsulates the parameter modification information into a parameter modification request, and sends the parameter modification request to the server. For example, the previous user input information is: Cats are fighting, please notify me by SMS, and the server controls the AI ​​device to present the summary content: Hello, Xiaodi monitors cats to see if they are fighting all day long. If a cat fights, Xiaodi will promptly notify the owner by SMS. However, the user suddenly wants to monitor whether the cat drinks water, so the user operates the AI ​​device, causing the AI ​​device to return content modification information indicating that the summary content is wrong. The server responds to the content modification information, controls the AI ​​device to present the modification prompt information, and both the server and the AI ​​device enter the content modification state.

[0141] The user presses a button on the artificial intelligence device, and the artificial intelligence device responds to the button pressing operation, controls the microphone to collect the parameter modification information input by the user, "I want to monitor whether the cat drinks water", and encapsulates the parameter modification information into a parameter modification request and sends it to the server. The server responds to the parameter modification request, calls the target large language model to parse the parameter modification information, and obtains the parameter modification object and the modification parameters corresponding to the parameter modification object. For example, the parameter modification object is "monitoring behavior", and the previous target control parameter corresponding to the parameter modification object is "fighting". At this time, the modification parameter corresponding to the parameter modification object is "drinking water". The server traverses the target control parameter corresponding to the parameter modification object in the candidate control table as the control parameter to be modified, that is, "fighting" is the control parameter to be modified. The server updates the control parameter to be modified to the modification parameter corresponding to the parameter modification object, and obtains the updated candidate control table, that is, "fighting" is replaced by "drinking water".

[0142] In a monitoring scenario, the embodiment of the present application can flexibly and randomly modify the target monitoring task and the target control parameters corresponding to the target monitoring task based on the target large language model, thereby meeting any monitoring needs of the user and helping to improve the user experience.

[0143] In S444, the missing parameter information is used to prompt the user to supplement the information associated with the missing element, where the missing element is the execution element that lacks the target control parameter. The server can control the artificial intelligence device to present the missing parameter information, prompt the user to supplement the information associated with the missing element, clearly explain to the user which execution element-related parameters are missing, and guide the user to provide the corresponding information so that the candidate control table can be improved and complete in the future, and then the target control table can be determined to ensure the smooth execution of the monitoring task.

[0144] In some embodiments, the server implements a memory module 700 for storing historical input information of the user. Figure 7 Chat_history complete conversation history 71 is used to store the historical input information of the chat scene, Task_history monitoring task summary conversation history 72 is used to store the historical input information of the monitoring task control parameters of the monitoring scene, and Session_status conversation status 73 is used to store the status of the interaction scene of the current user input information.

[0145] The server converts the user input information into {session_id, prompt} format for storage. First, the user's current interaction scene is determined by checking the dialogue state 73. If it is 0, the chat scene is entered; if it is 1, the monitoring scene is entered; if it is 2, the interaction scene is switched, and the user is prompted whether to switch the interaction scene.

[0146] If it is 0, the chat scene is entered. After entering the chat scene, the target large language model will be used to determine the intention of the prompt word, and the user's input information will be analyzed to determine whether it is chat content or a monitoring task. If it is determined to be chat content, the target large language model will check the complete conversation history 71, reply to the chat content based on the historical input information, and update the complete conversation history 71; if it is determined to be setting a monitoring task, the status of the current interaction scene will be updated to 2, and the mode confirmation page will be entered to prompt the user whether to switch the interaction mode.

[0147] If it is 1, it will enter the monitoring scene. After entering the monitoring scene, it will first determine whether there is a monitoring task summary dialogue history 72. When there is no monitoring task summary dialogue history 72, the server will parse the user input information to obtain the target control parameters; when there is a monitoring task summary dialogue history 72, the server will parse the user input information in combination with the monitoring task summary dialogue history 72 to obtain the target control parameters. The server will combine the obtained target control parameters to generate a candidate control table, and determine whether the target control parameters of all execution elements recorded in the current candidate control table are complete. If the target control parameters are complete, the artificial intelligence device will be controlled to present the summary content of the target monitoring task, and the target control table will be determined according to the content operation information returned by the artificial intelligence device; if the target control parameters are incomplete, parameter missing information will be generated, and the parameter missing information is used to prompt the user to supplement the associated information with the missing elements.

[0148] Among them, when the content operation information is information indicating that the summary content is correct, the candidate control table is determined to be the target control table; or, when the content operation information is information indicating that the summary content is wrong, a modification prompt information is generated to prompt the user to give the user input information again. The embodiment of the present application can improve the accuracy of the target large language model analysis, can accurately meet the user's requirements for monitoring tasks, and thus improve the performance of artificial intelligence equipment.

[0149] See also Figure 8 As another aspect of the embodiment of the present application, the embodiment of the present application provides an artificial intelligence device 800 for communicating with the server, the artificial intelligence device includes a controller 81 and multiple types of hardware modules, the controller 81 is electrically connected to the multiple types of hardware modules, wherein the multiple types of hardware modules are a camera 82, a microphone 83, a speaker 84, a wireless communication module 85, a timer 86 and a display screen 87, the controller 81 includes a memory and a processor, the memory is connected to the processor, the processor is used to execute one or more computer programs stored in the memory, when the processor executes the one or more computer programs, the artificial intelligence device implements the following operations: parse the target control parameters corresponding to each of the execution elements from the target control table, call the hardware module matching the target control parameters to perform the monitoring operation according to the target control parameters.

[0150] Wherein, the target control parameter includes download link information, the hardware module includes a camera 82 and a wireless communication module 85, the controller 81 controls the wireless communication module 85 to download the target lightweight model to the local for storage according to the download link information, and detects whether there is a monitoring object in the picture taken by the camera 82 in real time based on the target lightweight model. If the artificial intelligence device detects in real time based on the target lightweight model that there is no monitoring object in the picture taken by the artificial intelligence device, the artificial intelligence device does not need to send the picture taken by the artificial intelligence device to the server for detection. If the artificial intelligence device detects in real time based on the target lightweight model that there is a monitoring object in the picture taken by the artificial intelligence device, the artificial intelligence device sends the picture taken by the artificial intelligence device to the server for detection.

[0151] The server receives the target image taken by the artificial intelligence device, performs a duplicate check on the target image, and determines whether the target image meets the duplicate check conditions. If the target image meets the duplicate check conditions, the server calls the target large language model to process the target image. If the target image does not meet the duplicate check conditions, the server does not process the target image.

[0152] Determining whether the target image meets the duplicate checking conditions includes the following steps: obtaining historical images with timestamps before the target image, calculating the degree of difference between the target image and the historical images, if the degree of difference is greater than or equal to a preset difference threshold, determining that the target image does not meet the duplicate checking conditions; if the degree of difference is less than the preset difference threshold, determining that the target image meets the duplicate checking conditions.

[0153] The embodiment of the present application performs a duplicate check on the target image. If the difference between the target image and the historical image is not large, the server does not need to process it, thus saving computing power.

[0154] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0155] As another aspect of the embodiments of the present application, the embodiments of the present application provide an interactive device based on an artificial intelligence device. The interactive device based on an artificial intelligence device can be a software module, which includes several instructions stored in a memory, and the processor can access the memory and call the instructions for execution to complete the interactive method based on the artificial intelligence device described in the above various embodiments.

[0156] In some embodiments, the interactive device based on artificial intelligence equipment can also be built by hardware devices. For example, the interactive device based on artificial intelligence equipment can be built by one or more chips, and each chip can work in coordination with each other to complete the interactive method based on artificial intelligence equipment described in the above embodiments. For another example, the interactive device based on artificial intelligence equipment can also be built by various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), single-chip microcomputers, ARM (AcornRISC Machine) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or any combination of these components.

[0157] See also Fig. 9 The interactive device 900 based on artificial intelligence equipment includes a request acquisition module 91, a scene confirmation module 92, a parameter acquisition module 93, a parameter combination module 94 and a sending execution module 95.

[0158] The request acquisition module 91 is used to obtain the user's input request, and the input request carries the user input information. The scene confirmation module 92 is used to respond to the user's input request and determine the target interaction scene corresponding to the user's input information. The parameter acquisition module 93 is used to respond to the target interaction scene as a monitoring scene, call the target large language model according to the element type of the execution element to parse the user input information, and obtain the target control parameter corresponding to the execution element. The parameter combination module 94 is used to combine multiple target control parameters to obtain a target control table. The sending execution module 95 is used to send the target control table to the artificial intelligence device so that the artificial intelligence device performs the monitoring operation according to the target control table.

[0159] In some embodiments, the scene confirmation module 92 is specifically used to: respond to user input requests, call the target large language model to parse the user input information, obtain candidate interaction scenes corresponding to the user input information, and determine the target interaction scene according to the type of the candidate interaction scene.

[0160] In some embodiments, the candidate interaction scenarios include chat scenarios or monitoring scenarios, and the scenario confirmation module 92 is further specifically used to: respond to the type of the candidate interaction scenario being a chat scenario, determine the target interaction scenario to be a chat scenario, or respond to the type of the candidate interaction scenario being a monitoring scenario, perform a monitoring confirmation operation.

[0161] In some embodiments, the scene confirmation module 92 is also specifically used to: determine the interaction scene of the last input information, the last input information is input information that is before the user input information and adjacent to the user input information, the interaction scene in response to the last input information is a monitoring scene, determine the target interaction scene as a monitoring scene, or, the interaction scene in response to the last input information is a chat scene, control the artificial intelligence device to present a mode confirmation page, and the mode confirmation page is used to prompt the user whether he needs to enter the monitoring mode.

[0162] In some embodiments, the parameter acquisition module 93 is specifically used to: obtain a prompt template library, the prompt template library includes multiple types of prompt word templates, traverse the prompt word template corresponding to the element type of the execution element in the prompt template library as the target prompt word template, input the target prompt word template and user input information into the target large language model for parsing, and obtain the target control parameter corresponding to the execution element.

[0163] In some embodiments, the parameter acquisition module 93 is also specifically used to: input the object prompt word template and the user input information into the target large language model for parsing, obtain the monitoring object pointed to by the user input information, obtain the model library, the model library includes multiple types of lightweight small models, and search in the model library whether there is a target lightweight small model corresponding to the monitoring object, the target lightweight small model is a type of lightweight small model among the multiple types of lightweight small models. If it exists, then generate download link information corresponding to the target lightweight small model, the download link information is the target control parameter, wherein the artificial intelligence device can download the target lightweight small model to the local according to the download link information for local storage, and based on the target lightweight small model, detect in real time whether there is a monitoring object in the picture taken by the artificial intelligence device locally, if not, then generate cloud detection information, the cloud detection information is the target control parameter, wherein the artificial intelligence device responds to the cloud detection information and transmits the real-time taken picture to the server, so that the server detects whether the picture contains the monitoring object.

[0164] In some embodiments, the parameter combination module 94 is specifically used to: combine all target control parameters to obtain a candidate control table, detect whether the candidate control table has recorded the target control parameters of all execution elements, if so, control the artificial intelligence device to present summary content about the target monitoring task, and determine the target control table based on the content operation information returned by the artificial intelligence device; if not, generate parameter missing information, and the parameter missing information is used to prompt the user that the user needs to supplement the associated information with the missing element, and the missing element is the execution element that lacks the target control parameter.

[0165] In some embodiments, the parameter combination module 94 is also specifically used to: respond to the content operation information as information indicating that the summary content is correct, determine the candidate control table as the target control table, or, respond to the content operation information as information indicating that the summary content is wrong, control the artificial intelligence device to present modification prompt information, and the modification prompt information is used to prompt the user to give the user input information again.

[0166] In some embodiments, the parameter combination module 94 is also specifically used to: obtain a parameter modification request transmitted by an artificial intelligence device, the parameter modification request carries parameter modification information, respond to the parameter modification request, call the target large language model to parse the parameter modification information, obtain a parameter modification object and modification parameters corresponding to the parameter modification object, traverse the target control parameters corresponding to the parameter modification object in the candidate control table as control parameters to be modified, update the control parameters to be modified to the modification parameters corresponding to the parameter modification object, and obtain an updated candidate control table.

[0167] It should be noted that the above-mentioned interactive device based on artificial intelligence device can execute the interactive method based on artificial intelligence device provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in the embodiment of the interactive device based on artificial intelligence device, please refer to the interactive method based on artificial intelligence device provided in the embodiment of the present application.

[0168] See also Fig.10 , Fig.10 1 is a schematic diagram of the structure of a server 1000 provided in an embodiment of the present application. The server includes one or more processors 101 and a memory 102. The memory 102 is connected to the one or more processors 101, for example, connected to the processor 101 through a bus.

[0169] The processor 101 is configured to support the server to perform the corresponding functions in the method in the above method embodiment. The processor can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0170] The memory 102 is used to store program codes, etc. The memory may include a volatile memory (VM), such as a random access memory (RAM); the memory may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory may also include a combination of the above types of memory.

[0171] The memory 102 can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the interactive method based on artificial intelligence devices in the embodiments of the present application. The processor executes various functional applications and data processing of the interactive method based on artificial intelligence devices and the interactive device based on artificial intelligence devices by running the non-volatile software programs, instructions and modules stored in the memory, that is, realizes the functions of each module or unit of the interactive method based on artificial intelligence devices and the interactive device based on artificial intelligence devices provided in the above method embodiments.

[0172] The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. The data storage area may store data created based on the use of an interactive device based on an artificial intelligence device, etc. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the interactive device based on the artificial intelligence device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The one or more modules are stored in the memory, and when executed by the one or more processors, the interaction method based on the artificial intelligence device in any of the above method embodiments is executed, for example, the method steps described in the above method embodiments are executed to realize the functions of the modules described in the above device embodiments.

[0174] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0175] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0176] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An interactive method based on artificial intelligence equipment, applied to a server, characterized in that: include: Obtaining a user input request, where the user input request carries user input information; In response to the user input request, determine a target interaction scenario corresponding to the user input information, the target interaction scenario is used to instruct the pre-trained target large language model to interact with the artificial intelligence device using a target interaction mode, the monitoring task represented by the user input information is a target monitoring task, and the target monitoring task is composed of multiple execution elements; In response to the target interaction scenario being a monitoring scenario, calling the target large language model according to the element type of the execution element to parse the user input information to obtain a target control parameter corresponding to the execution element; Combining a plurality of the target control parameters to obtain a target control table; The target control table is sent to the artificial intelligence device so that the artificial intelligence device performs a monitoring operation according to the target control table.

2. The interactive method according to claim 1, characterized in that: The step of responding to the user input request and determining a target interaction scenario corresponding to the user input information includes: In response to the user input request, calling the target large language model to parse the user input information to obtain a candidate interaction scenario corresponding to the user input information; A target interaction scenario is determined according to the type of the candidate interaction scenario.

3. The interactive method according to claim 2, characterized in that: The candidate interaction scenarios include a chat scenario or a monitoring scenario, and determining a target interaction scenario according to the type of the candidate interaction scenarios includes: In response to the type of the candidate interaction scene being a chat scene, determining that the target interaction scene is a chat scene; or, In response to the type of the candidate interaction scenario being a monitoring scenario, a monitoring confirmation operation is performed.

4. The interactive method according to claim 3, characterized in that: The execution monitoring confirmation operation includes: Determine the interaction scenario of the last input information, where the last input information is input information that is before the user input information and adjacent to the user input information; In response to the interaction scene of the last input information being a monitoring scene, determining that the target interaction scene is a monitoring scene; or, In response to the interaction scenario of the last input information being a chat scenario, the artificial intelligence device is controlled to present a mode confirmation page, and the mode confirmation page is used to prompt the user whether to enter the monitoring mode.

5. The interactive method according to any one of claims 1 to 4, characterized in that: The step of calling the target large language model according to the element type of the execution element to parse the user input information and obtaining the target control parameter corresponding to the execution element includes: Acquire a prompt template library, wherein the prompt template library includes multiple types of prompt word templates; Traversing the prompt word template corresponding to the element type of the execution element in the prompt template library as a target prompt word template; The target prompt word template and the user input information are input into the target large language model for parsing to obtain the target control parameters corresponding to the execution elements.

6. The interactive method according to claim 5, characterized in that: The execution element includes a monitoring object, the target prompt word template is an object prompt word template, and the target prompt word template and the user input information are input into the target large language model for parsing to obtain the target control parameters corresponding to the execution element. Inputting the object prompt word template and the user input information into the target large language model for parsing to obtain the monitoring object pointed to by the user input information; Obtain a model library, wherein the model library includes multiple types of lightweight small models; Searching in the model library whether there is a target lightweight small model corresponding to the monitored object, the target lightweight small model being one type of lightweight small model among the multiple types of lightweight small models; If it exists, then generate download link information corresponding to the target lightweight small model, the download link information is the target control parameter, wherein the artificial intelligence device can download the target lightweight small model to the local for storage according to the download link information, and detect in real time locally whether the picture taken by the artificial intelligence device contains the monitored object based on the target lightweight small model; If not, cloud detection information is generated, and the cloud detection information is the target control parameter, wherein the artificial intelligence device responds to the cloud detection information and transmits the real-time captured picture to the server, so that the server detects whether the monitored object exists in the picture.

7. The interactive method according to any one of claims 1 to 4, characterized in that: The combining of the plurality of target control parameters to obtain a target control table comprises: Combining all of the target control parameters to obtain a candidate control table; Detecting whether the candidate control table has recorded the target control parameters of all the execution elements; If so, controlling the artificial intelligence device to present summary content about the target monitoring task, and determining a target control table according to the content operation information returned by the artificial intelligence device; If not, parameter missing information is generated, and the parameter missing information is used to prompt the user to supplement the associated information with the missing element, and the missing element is the execution element that lacks the target control parameter.

8. The interactive method according to claim 7, characterized in that: Determining the target control table according to the content operation information returned by the artificial intelligence device includes: In response to the content operation information being information indicating that the summary content is correct, determining the candidate control table as the target control table; or, In response to the content operation information being information indicating that the summary content is wrong, the artificial intelligence device is controlled to present modification prompt information, wherein the modification prompt information is used to prompt the user to give the user input information again.

9. The interactive method according to claim 8, characterized in that: After controlling the artificial intelligence device to present the modification prompt information, the method further includes: Obtaining a parameter modification request transmitted by the artificial intelligence device, wherein the parameter modification request carries parameter modification information; In response to the parameter modification request, calling the target large language model to parse the parameter modification information to obtain a parameter modification object and a modification parameter corresponding to the parameter modification object; Traversing the target control parameter corresponding to the parameter modification object in the candidate control table as the control parameter to be modified; The control parameters to be modified are updated into modification parameters corresponding to the parameter modification objects, and an updated candidate control table is obtained.

10. A server, characterized in that: It includes a memory and a processor, the memory is connected to the processor, the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the server implements the interaction method based on the artificial intelligence device as described in any one of claims 1 to 9.

11. An artificial intelligence device, characterized in that: The artificial intelligence device is used to communicate with the server according to claim 10, wherein the artificial intelligence device includes multiple types of hardware modules and a controller, and the controller is electrically connected to the multiple types of hardware modules respectively, wherein the controller includes a memory and a processor, the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the artificial intelligence device implements the following operations: Parsing the target control parameter corresponding to each of the execution elements from the target control table; A hardware module matching the target control parameter is called to perform a monitoring operation according to the target control parameter.

12. The artificial intelligence device according to claim 11, characterized in that The target control parameter includes download link information, the hardware module includes a camera module and a wireless communication module, and the controller is configured as follows: Control the wireless communication module to download the target lightweight model to local storage according to the download link information; In response to the target lightweight model detecting in real time locally that a monitored object exists in the picture taken by the camera module, the picture taken by the camera module is transmitted to the server; In response to the target lightweight model, the local real-time detection is performed to determine whether the monitored object exists in the picture taken by the camera module, and based on the target lightweight model, the real-time detection is performed to determine whether the monitored object exists in the picture taken by the camera module.

Images