Large model-based man-machine interaction method and device and man-machine interaction equipment

Abstract

The invention relates to a man-machine interaction method and device based on a large model and man-machine interaction device.The man-machine interaction method based on the large model comprises the steps that user input information is obtained, and a trained target large model is used for analyzing the user input information to obtain user intention information; obtaining a preset function configuration knowledge graph; under the condition that the user intention information indicates a function configuration operation for the industrial equipment, performing query processing on the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation; inputting the target function information into the target large model, and outputting configuration structured parameters; generating guidance information based on the configuration structured parameters; and obtaining function configuration information input to the target large model by the user based on the guide information, and performing function configuration on the industrial equipment based on the function configuration information. According to the invention, the problem that the man-machine interaction process is tedious is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a human-computer interaction method, apparatus, and human-computer interaction device based on a large model. Background Art

[0002] Human-Computer Interaction (HCI) is a discipline that studies how people and computers interact effectively, efficiently, and pleasantly; its goal is to design and develop user-friendly, easy-to-use, and powerful interactive systems to improve user satisfaction and efficiency. When faced with the configuration of professional industrial equipment, ordinary users face great obstacles in configuring related functions due to lack of relevant knowledge, complex operating interfaces, and difficult to understand specialized terms in help manuals, resulting in a cumbersome human-computer interaction process.

[0003] Currently, no effective solution has been proposed for the cumbersome problem of human-computer interaction process in related technologies. Summary of the invention

[0004] The embodiments of the present application provide a large-model-based human-computer interaction method, apparatus, and human-computer interaction device to at least solve the problem of cumbersome human-computer interaction process in related technologies.

[0005] In a first aspect, the embodiments of the present application provide a human-computer interaction method, apparatus, and human-computer interaction device based on a large model, the method comprising: Get user intention information input by the user; Acquire a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, query the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation; Inputting the target function information into the target macro model and outputting configuration structured parameters; generating guidance information based on the configuration structured parameters; The function configuration information input by the user to the target large model based on the guidance information is obtained, and the function configuration of the industrial equipment is performed based on the function configuration information.

[0006] In some embodiments, after parsing the user input information to obtain the user intention information, the method further includes: Obtain preset industrial equipment document information; Input the industrial equipment document information into the target big model and construct a question-answer knowledge graph; In the case where it is detected that the user intention information indicates a knowledge question and answer operation for the industrial equipment, query processing is performed on the question and answer knowledge graph according to the user input information to obtain target question and answer information corresponding to the knowledge question and answer operation; The target question and answer information is input into the target macro model, and a target answer result is generated.

[0007] In some embodiments, the inputting the target function information into the target macro model and outputting the configuration structured parameters includes: Input the target function information into the target macro model, and output the operation steps, parameter list and parameter description information corresponding to the target function information; The configuration structured parameters are generated according to the operation steps, the parameter list and the parameter description information.

[0008] In some embodiments, generating the guidance information based on the configuration structured parameters includes: Based on each of the operation steps in the configuration structured parameters, generating first guidance information; Determine a parameter name corresponding to the operation step from the parameter list, and determine target description information corresponding to the parameter name from each parameter description information; Based on the parameter name and the target description information, second guidance information is generated; wherein the guidance information includes the first guidance information and the second guidance information.

[0009] In some embodiments, the step of obtaining the function configuration information input by the user to the target macro model based on the guidance information includes: Traversing each of the operation steps in the configuration structured parameters; For the currently traversed operation step, the first guidance information and the second guidance information corresponding to the first guidance information are output to the human-computer interaction interface by using the target large model for display, and the current step configuration information input by the user to the target large model via the human-computer interaction interface based on the second guidance information is obtained; For the next operation step traversed, the interaction process between the user and the target large model is repeated until all the operation steps are traversed, the function configuration information generated based on the configuration information of each current step is obtained, and the function configuration information is input into the target large model.

[0010] In some embodiments, the obtaining of the current step configuration information input by the user to the target large model via the human-computer interaction interface based on the second guide information includes: When it is detected that the configuration information verification of the current step fails, the target large model is used to output third guidance information to the human-computer interaction interface for display; the third guidance information is used to prompt the user to input information again; Acquire new step configuration information that the user re-enters into the target large model via the human-computer interaction interface based on the second guidance information and the third guidance information.

[0011] In some embodiments, the function configuration of the industrial device based on the function configuration information includes: In the case where the function configuration operation is an intelligent algorithm configuration operation, obtaining a target category to be analyzed in the function configuration information; The target category to be analyzed is sent to the visual language model applied to the industrial equipment to set the detection target, and complete the functional configuration of the industrial equipment.

[0012] In some embodiments, obtaining a preset function configuration knowledge graph includes: Get preset device usage text data; Inputting the device usage text data into the target macro model; parsing the device usage text data using the target macro model to obtain a plurality of text blocks, and identifying corresponding functional entity information and functional relationship information based on each of the text blocks; The function configuration knowledge graph is constructed according to the function entity information and the function relationship information.

[0013] In some embodiments, the method further comprises: Acquire preset security knowledge data; fine-tune the initial large model according to the security knowledge data, and generate the target large model.

[0014] In a second aspect, an embodiment of the present application provides a human-computer interaction device based on a large model, the device comprising: An input module, used to input the acquired user input information into the trained target large model, and use the target large model to parse the user input information to obtain user intention information; A graph construction module is used to obtain a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, query the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation; A structured parameter module, used to input the target function information into the target macro model and output configuration structured parameters; A guidance module, used for generating guidance information based on the configuration structured parameters; The configuration interaction module is used to obtain the function configuration information input by the user to the target large model based on the guidance information, and perform function configuration on the industrial equipment based on the function configuration information.

[0015] In a third aspect, an embodiment of the present application provides a human-computer interaction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the human-computer interaction method based on a large model as described in the first aspect above is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the human-computer interaction method based on a large model as described in the first aspect above is implemented.

[0017] Compared with the related art, the human-computer interaction method, device and human-computer interaction device based on the big model provided in the embodiment of the present application obtain user input information, and use the trained target big model to parse the user input information to obtain user intention information; obtain a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, query and process the function configuration knowledge graph according to the user input information to obtain the target function information corresponding to the function configuration operation; input the target function information into the target big model, and output the configuration structured parameters; generate guidance information based on the configuration structured parameters; obtain the function configuration information input by the user to the target big model based on the guidance information, and perform function configuration on the industrial equipment based on the function configuration information. Based on this, the flexible configuration of the functions of industrial equipment is realized, avoiding the phenomenon that ordinary users have difficulty in configuring related functions due to lack of relevant knowledge, complex operation interface, and difficult to understand special terms in the help manual when facing professional equipment configuration, and effectively solves the problem of cumbersome human-computer interaction process.

[0018] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a hardware structure block diagram of a terminal of a human-computer interaction method based on a large model according to an embodiment of the present application; Figure 2is a flow chart of a human-computer interaction method based on a large model according to an embodiment of the present application; Figure 3 is a flow chart of another large model-based human-computer interaction method according to an embodiment of the present application; Figure 4 It is a structural block diagram of a large model-based human-computer interaction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0021] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0022] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0023] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 1 is a hardware structure block diagram of a terminal of a human-computer interaction method based on a large model according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0024] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the human-computer interaction method based on the large model in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0025] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0026] This embodiment provides a human-computer interaction method based on a large model. Figure 2 is a flow chart of a human-computer interaction method based on a large model according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps: Step S210, obtain user input information, and use the trained target large model to parse the user input information to obtain user intention information.

[0027] Among them, the target large model refers to a deep learning model with a large number of parameters (usually ranging from hundreds of millions to hundreds of billions); these models are able to generate high-quality natural language text by training on large-scale text data and perform well in various natural language processing tasks.

[0028] In order to better apply it to the security field, in an optional embodiment, the training process for the target large model may also include the following steps: obtaining preset security knowledge data; fine-tuning the initial large model according to the security knowledge data, and generating a target large model. It should be understood that the security knowledge data may be a large amount of pre-collected text data in the security field. The large language model is fine-tuned using the industry knowledge in the security field, so that the fine-tuned target large model can better adapt to the tasks and requirements of the specific field of security, and the target large model can better understand and process specific terms and concepts in the security field.

[0029] In this step, during the interaction between the user and the large model, the user can first input information to the trained target large model, and the target large model parses the user input information and analyzes the user's intention to determine what operation the user needs to perform.

[0030] Step S220, obtaining a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, querying the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation.

[0031] The above-mentioned functional configuration knowledge graph refers to a knowledge graph that displays various functions of industrial equipment and their configuration information in a graphical way; the industrial equipment includes but is not limited to various cameras, industrial robots or other professional equipment used in industry. Among them, the knowledge network can be constructed through nodes (representing various functions or configuration items of industrial equipment) and edges (representing the association relationship between various function nodes or configuration item nodes). Specifically, the construction process of the functional configuration knowledge graph can be: extracting the relationship between functional entities, such as dependency relationships, mutually exclusive relationships, configuration parameters, etc. from the text through methods such as rule matching, machine learning or deep learning models; designing the node (entity) and edge (relationship) structure of the knowledge graph based on the identified entities and relationships, and using a graph database or graph processing framework to build a knowledge graph, and importing entities and relationships into the graph.

[0032] Alternatively, in order to improve the completeness and accuracy of the knowledge graph construction, it can also be constructed based on the big model. That is, in an optional embodiment, the above-mentioned acquisition of the preset functional configuration knowledge graph can also include the following steps: acquiring preset device usage text data; inputting the device usage text data into the target big model; parsing the device usage text data using the target big model, dividing it into multiple text blocks, and based on each text block, identifying the corresponding functional entity information and functional relationship information; and constructing the functional configuration knowledge graph based on the functional entity information and functional relationship information.

[0033] Among them, the above-mentioned equipment usage text data refers to text data used to explain the specific functions, configurations and usage methods of industrial equipment. The data source of the equipment usage text data can be product manuals, user guides, online help documents, technical support forums, etc. Before building the knowledge graph, the acquired equipment usage text data can be cleaned as necessary, such as removing redundant information, unifying the format, etc., to facilitate subsequent processing. Next, the processed equipment usage text data is input into the target large model; the target large model divides it into multiple logically relatively independent text blocks by understanding the text. Each text block can be a paragraph or a description of a functional module. Then, the graph-based knowledge retrieval augmented generation (graphRAG) method is used to extract instances of modules, functions, steps, parameters, etc. contained in each text block, so as to identify entities related to equipment functions in the text block (that is, the above-mentioned functional entity information), and further analyze the corresponding instance relationships and relationship descriptions in the text block (that is, the above-mentioned functional relationship information). In the knowledge graph, functional entities are used as nodes and functional relationships are used as edges. Each node can contain detailed information of the entity (such as name, description, etc.), and each edge can contain the type and direction of the relationship; based on the parsed functional entity information and functional relationship information, the structure of the knowledge graph is gradually constructed. Ensure that the nodes and edges in the graph can accurately reflect the functional configuration and mutual relationship of the device, and finally construct a functional configuration knowledge graph.

[0034] After the function configuration knowledge graph is constructed, if the above-analyzed user intention information indicates that the user needs to perform function configuration operations on the industrial equipment, the constructed function configuration instruction graph can be queried according to the user information to obtain detailed information on configuring the function, that is, the above-mentioned target function information. For example, the target function information may include module name, function description, operation steps, required parameters, etc.

[0035] Step S230, input the target function information into the target macro model, and output the configuration structured parameters.

[0036] Specifically, the target function information containing the function configuration details obtained in the above steps is input into the target big model, and the operation steps required for the function, as well as the parameter list and parameter description information that the user needs to send for each step are extracted to obtain the configuration structured data of step-parameter name-parameter description.

[0037] Step S240: Generate guidance information based on the configuration structured parameters.

[0038] Among them, the prompt information can drive the large language model to prompt the user to perform the operation in the current step, the specific parameters that the user needs to enter, and the explanation of the function of the parameters. Specifically, the configuration structured parameters can be filled into the pre-configured prompt information template to generate specific prompt information; this step is usually completed automatically by the program to ensure the accuracy and consistency of the information.

[0039] Step S250, obtaining the function configuration information input by the user to the target large model based on the guidance information, and performing function configuration on the industrial equipment based on the function configuration information.

[0040] In this step, the generated guide information is displayed to the user to guide the user to enter the required function configuration information according to the steps. The user enters the value of each function configuration parameter in turn according to the prompts of the guide information. These inputs will be collected and used in the subsequent configuration process. After that, the function configuration information entered by the user is converted into a command format acceptable to the industrial equipment and sent to the industrial equipment through an appropriate interface (such as USB, Wi-Fi, Bluetooth, etc.), and the industrial equipment completes the corresponding function configuration. Finally, feedback from the industrial equipment is received to confirm whether the function configuration has been successfully applied. If the configuration fails, the guide information can be redisplayed and the user can be prompted to check whether the input is correct.

[0041] Through the above steps, the target large language model and the configuration structured parameters can be used to generate guidance information to guide users to correctly configure the functions of industrial equipment; this method not only improves the accuracy of configuration, but also reduces the difficulty of user operation.

[0042] It is also necessary to add that in the relevant technology, ordinary users have the following difficulties when configuring professional industrial equipment such as camera equipment: (1) A certain technical background is required to correctly configure and use professional industrial equipment, and ordinary users lack relevant knowledge; (2) The operation interface design is complex and difficult to quickly get started; (3) The user manual contains a large number of professional terms that are difficult for ordinary users to understand. Therefore, users have a certain degree of difficulty in configuring and using professional industrial equipment, resulting in low user satisfaction and efficiency in the use of industrial equipment.

[0043] Compared with the related art, the embodiment of the present application uses the above-mentioned human-computer interaction method based on the big model. When it is parsed that the user intention information indicates the function configuration operation of the industrial equipment, the pre-built function configuration knowledge graph is queried according to the user input information, and the detailed steps and required parameters of the user's target configuration items are parsed. The user input is guided by the big language model to complete the conversation, thereby realizing flexible configuration of the functions of the industrial equipment, avoiding the phenomenon that ordinary users have difficulty in configuring related functions due to lack of relevant knowledge, complex operating interface, and difficult to understand special terms in the help manual when facing professional equipment configuration, and effectively solves the problem of cumbersome human-computer interaction process.

[0044] In some embodiments, after parsing the user input information to obtain the user intent information, the method further includes: Obtain preset industrial equipment document information; input the industrial equipment document information into the target big model, and construct a question-and-answer knowledge graph; when it is detected that the user intention information indicates a knowledge question-and-answer operation for industrial equipment, query and process the question-and-answer knowledge graph according to the user input information to obtain target question-and-answer information corresponding to the knowledge question-and-answer operation; input the target question-and-answer information into the target big model, and generate a target reply result.

[0045] Among them, the above-mentioned industrial equipment document information refers to the document information used to describe the product description of industrial equipment; the data source of the industrial equipment document information is usually all documents related to industrial equipment, including but not limited to user manuals, installation guides, troubleshooting, FAQs, Application Programming Interface (API) documents, update logs, etc. It should also be understood that the method of constructing a question-and-answer knowledge graph based on industrial equipment document information can be similar to the method of constructing a functional configuration knowledge graph mentioned above; that is, the industrial equipment document information after data preprocessing is input into the target large model; the target large model divides the document information into multiple text blocks through text understanding. Then, the graphRAG method is used to extract the instances contained in each text block, as well as the relationships and relationship descriptions between the instances, so as to generate a question-and-answer knowledge graph.

[0046] After the above knowledge graph is constructed, if the user intent information is analyzed to indicate a knowledge question-and-answer operation for industrial equipment, it means that the user has questions related to industrial equipment that need to be answered. In this process, the question-and-answer knowledge graph is queried, and the most relevant query content obtained is used as the context of the user's question and input into the target large model to obtain the corresponding output, that is, the above target answer result is obtained.

[0047] Through the above embodiments, the question-and-answer knowledge graph is combined with the target big model, and the structured knowledge in the knowledge graph is used to provide users with an accurate and reliable information basis. Combined with the semantic understanding ability of the big model, accurate matching and answers to user questions are achieved, which can not only improve the accuracy and efficiency of question-and-answer, enhance user experience and satisfaction, but also expand application scenarios and functions, improve the intelligence level and explainability of the system, and promote the updating and maintenance of knowledge.

[0048] In some embodiments, the step of inputting the target function information into the target macro model and outputting the configuration structured parameters may further include the following steps: The target function information is input into the target large model, and the operation steps, parameter list and parameter description information corresponding to the target function information are output; according to the operation steps, parameter list and parameter description information, the configuration structured parameters are generated.

[0049] Among them, the target large model receives the input target function information, and uses its internal knowledge structure and semantic understanding ability to parse it, identifies the operation steps related to the target function to be configured, the required parameter list and the description information of each parameter (such as the parameter type, purpose, possible value range or default value, etc.), and finally integrates and generates configuration structured parameters.

[0050] Taking the configuration of the night vision function of the camera of industrial equipment as an example, the configuration structured parameters can be shown in the following Table 1: Table 1 Example of configuration structured parameters

[0051] Through the above-mentioned embodiments, by automatically parsing and generating configuration structured parameters, they are easy to store and retrieve, which helps to share and reuse knowledge within a system or across systems, reduces the need for manual intervention, and improves the degree of automation of function configuration; configuration structured parameters also enable the function configuration process to easily adapt to different functional requirements and user preferences, thereby improving the flexibility of function configuration.

[0052] In some embodiments, the above-mentioned generating guidance information based on the configuration structured parameters may also include the following steps: Based on each operation step in the configuration structured parameters, generate the first guidance information; determine the parameter name corresponding to the operation step from the parameter list, and determine the target description information corresponding to the parameter name from each parameter description information; based on the parameter name and the target description information, generate the second guidance information; wherein the guidance information includes the first guidance information and the second guidance information.

[0053] Specifically, according to each operation step in the configuration structured parameters, the first guidance information is written or generated. The first guidance information may include the sequence number of the step, a brief description, and necessary operation prompts. These guidance information usually describe the operation process of each step in a user-friendly language, aiming to help users understand and perform corresponding actions.

[0054] Next, from the parameter list of the configuration structured parameters, identify the parameter name that is directly related to the current operation step. For each parameter name, find the corresponding target description information from the parameter description information. The target description information should clearly and accurately explain the function of the parameter, possible values, or how the user should set the parameter. Based on the parameter name and target description information, write or generate the second guidance information. The second guidance information can be presented in the form of a parameter table, information box, or a point-by-point explanation so that the user can easily find and understand it. This information should explain in detail the importance of each parameter, how to set it, and the possible consequences.

[0055] The first guide information and the second guide information are integrated to form a complete guide information. This ensures that the user can obtain relevant parameter setting guidance and instructions when performing each operation step. The integrated guide information can be provided to the user in the form of a step-by-step guide, an interactive interface or a document.

[0056] Through the above embodiment, clear guidance information is constructed through operation steps, parameter names and parameter description information, so that users can more easily understand the purpose and importance of each operation step, and how to correctly set relevant parameters, thereby improving the stability and reliability of the function configuration process.

[0057] In some embodiments, the obtaining of the function configuration information input by the user to the target macro model based on the guidance information may further include the following steps: Traverse each operation step in the configuration structured parameters; for the currently traversed operation step, use the target large model to output the first guidance information to the human-computer interaction interface, and display the second guidance information corresponding to the first guidance information, and obtain the current step configuration information input by the user to the target large model through the human-computer interaction interface based on the second guidance information; for the next traversed operation step, repeat the interaction process between the user and the target large model until all operation steps are traversed, obtain the function configuration information generated based on the configuration information of each current step, and input the function configuration information to the target large model.

[0058] In this step, the user is interacted with through the guidance information, and the function configuration information is gradually constructed based on the user's input in the human-computer interaction interface. The human-computer interaction interface can be a web page deployed on the terminal device, and the user can access and interact with it through a browser. The human-computer interaction interface can also be a graphical user interface (GUI) of the application, and the user can interact by clicking, dragging or inputting. In some professional or advanced application scenarios, the human-computer interaction interface can also be a command line interface (CLI), and the user controls the operation process by entering commands. The human-computer interaction process is described in detail below.

[0059] First, all the operation steps defined in the configuration structured parameters are traversed; these steps are a series of ordered actions required to perform a specific function configuration operation. For the currently traversed operation step, the target large model is used to generate and output the first guidance information. The first guidance information is usually a brief description or prompt of the operation step; for example, the first guidance information such as "Step 1, enter the camera setting interface" generated by the target large model can be displayed on the human-computer interaction interface. At the same time, the target large model also outputs the second guidance information corresponding to the first guidance information, that is, the parameter name and parameter description information related to the current operation step. This information helps the user understand the parameters that need to be set and their importance. These guidance information are displayed to the user through a human-computer interaction interface (such as a web page, application interface, or command line interface).

[0060] Based on the displayed second guidance information, the user enters the configuration information required for the current step on the human-computer interaction interface; this may include selecting parameter values, entering text or numbers, etc. The system receives the current step configuration information entered by the user through the human-computer interaction interface and stores it in the memory or a temporary file for subsequent processing.

[0061] Next, repeat the interaction process. The system continues to traverse the next operation step and repeats the above interaction process between the user and the target large model, including outputting guidance information and obtaining configuration information input by the user. This process continues until all operation steps have been traversed. After all operation steps have been processed, the system generates complete function configuration information based on the current step configuration information input by the user. This function configuration information contains all the steps and parameter settings required to perform a specific function. Finally, the system inputs this function configuration information into the target large model for subsequent processing or execution.

[0062] Through the above embodiment, by gradually guiding the user to input configuration information, the system can ensure that the user fully understands the requirements of each step, thereby improving the user's participation and accuracy. In addition, the user-friendly interactive interface and clear guidance information also help to improve the user's overall satisfaction with the system.

[0063] In some embodiments, the obtaining of the current step configuration information input by the user to the target large model via the human-computer interaction interface based on the second guidance information may further include the following steps: When it is detected that the verification of the current step configuration information fails, the target large model is used to output the third guidance information to the human-computer interaction interface for display; the third guidance information is used to prompt the user to re-enter the information; and the new step configuration information that the user re-enters into the target large model via the human-computer interaction interface based on the second guidance information and the third guidance information is obtained.

[0064] When the configuration information of the current step input by the user fails to be verified, the system outputs the third guidance information to the human-computer interaction interface through the target large model to prompt the user to re-enter the information. More specifically, after the user enters the configuration information of the current step, the system will verify the information. The verification may include checking the format, range, validity or compatibility with the previous step of the data. If the system detects that the configuration information input by the user does not meet the requirements (i.e., the verification fails), the error handling process is entered. At this time, the third guidance information is generated using the target large model and displayed to the user through the human-computer interaction interface. The third guidance information is usually an error prompt or warning, explaining the reason for the verification failure (for example, the user input information is invalid or the parameter information input is incomplete), and prompting the user to re-enter the information. The human-computer interaction interface is responsible for displaying the third guidance information to ensure that the user can clearly see the error prompt and the requirement to re-enter.

[0065] Afterwards, based on the second guide information (i.e., the parameter name and parameter description information related to the current operation step) and the third guide information (i.e., the error prompt), the user re-enters new step configuration information through the human-computer interaction interface. The human-computer interaction interface receives the information re-entered by the user and passes it to the system for processing. It should be understood that the system can re-verify the newly entered configuration information. If the verification passes, the next operation step will continue to be processed; if the verification still fails, the above error handling process can be repeated until the user enters the correct configuration information.

[0066] Through the above embodiment, when it is detected that the configuration information verification of the current step fails, the system outputs the third guidance information through the target large model and prompts the user to re-enter the information, which can ensure the accuracy and consistency of the data and reduce potential errors and problems.

[0067] In some embodiments, the above-mentioned function configuration of the industrial device based on the function configuration information may also include the following steps: When the function configuration operation is an intelligent algorithm configuration operation, the target category to be analyzed in the function configuration information is obtained; the target category to be analyzed is sent to the visual language model applied to the industrial equipment for detection target setting, and the function configuration of the industrial equipment is completed.

[0068] It should be noted that, in this embodiment, the function configuration for industrial equipment can be divided into different types: intelligent algorithm configuration function, camera setting function, event management function, system management function and other auxiliary functions. Among them, the intelligent algorithm configuration function needs to use the acquired structured data to interact with the visual language model.

[0069] Based on this, when the function configuration operation that the current user needs to perform is an intelligent algorithm configuration operation, the target category to be analyzed needs to be extracted from the function configuration information first. These target categories are used to indicate the type of object that the industrial equipment needs to detect, identify or analyze, such as faces, vehicles, animals, etc.

[0070] Once the target category to be analyzed is determined, this information, rule configuration parameters, and other related parameters will be sent to the visual language model applied to industrial equipment. The visual language model is a model that combines visual processing and natural language understanding capabilities. It can set the corresponding detection target according to the input target category. After receiving the target category, the visual language model will be configured accordingly based on this information so that the industrial equipment can efficiently and accurately detect or identify the specified target category. This process generally involves adjusting the parameters of the model, optimizing the detection algorithm, etc. After that, in the intelligent algorithm of this configuration, the development vocabulary target detection method (YOLO-World) is used to set the visual language model to detect specific categories of targets according to the received custom target category, and then follow up on such targets. Track, identify, logically process, etc., extract various attributes of such targets, whether they meet relevant alarm conditions, etc., and report intelligent events.

[0071] Through the above embodiments, by sending the target category to be analyzed to the visual language model applied to industrial equipment for detection target setting, the industrial equipment can flexibly configure the target category to be analyzed, thereby realizing flexible, accurate and convenient intelligent algorithm automatic configuration.

[0072] The present application is described in detail below in conjunction with specific embodiments. Taking industrial equipment as a camera device as an example, Figure 3 is a flowchart of another large model-based human-computer interaction method according to an embodiment of the present application. Figure 3 As shown, the process includes the following steps: Step S301, start the process; fine-tune the large language model. In this process, a large amount of text data in the field of security (i.e., the security knowledge data mentioned above) is collected, and the large language model is fine-tuned using the industry knowledge in the field of security, and a target large model is generated. Through this step, the model can better adapt to the tasks and needs of a specific field, and thus the large model can better understand and process specific terms and concepts in the field.

[0073] Step S302: Construct a functional configuration knowledge graph. Specifically, the target large model is used to parse the product manual (i.e., the above-mentioned device usage text data), and the product manual is divided into multiple text blocks, each of which can be a paragraph or a description of a functional module. The graphRAG method is used to extract the instances of modules, functions, steps, parameters, etc. contained in each chunk, as well as the corresponding instance relationships and relationship descriptions, to construct a functional configuration knowledge graph. Config Used for function configuration.

[0074] Step S303: Construct a common question and answer knowledge graph. Similar to the above step S302, all documents related to the product (i.e. the above industrial equipment document information) are parsed using the target big model to construct a question and answer knowledge graph about common industrial equipment questions. Q_A Used for knowledge quizzes.

[0075] Step S304, obtaining input information input by the user, and analyzing to obtain user intention information.

[0076] Step S305, return a response result for the knowledge question and answer operation process. Specifically, determine that the user intention information indicates a knowledge question and answer operation for industrial equipment; use the target big model to perform knowledge question and answer using the question and answer knowledge graph constructed above, and return a response result; end the process.

[0077] Step S306 : determining whether the user intention information indicates a function configuration operation for the industrial device.

[0078] Step S307, generating configuration structured parameters; wherein the configuration structured parameters include structured steps, parameters and other information; the configuration structured parameters are generated by querying the functional configuration knowledge graph constructed as described above.

[0079] Step S308: Generate guidance information based on the configuration structured parameters to guide the user to perform relevant actions and input the required function configuration information.

[0080] Step S309: calling a preset interface. Specifically, the configuration parameters (ie, the above-mentioned function configuration information) are sent through the preset interface.

[0081] If the function currently required to be configured is an intelligent algorithm, configure the target to be analyzed. After obtaining the target category to be analyzed, rule configuration parameters and other related parameters through the above steps, call the intelligent algorithm interface function to send it to the intelligent algorithm; in this intelligent algorithm, use YOLO-World to set the visual language model to detect targets of a specific category according to the received custom target category, and complete the intelligent algorithm function configuration; the configured industrial equipment can then perform subsequent tracking, identification, logical processing, etc. on such targets, and extract various attributes of such targets, whether they meet relevant alarm conditions, etc., and report intelligent events.

[0082] If the function currently required to be configured is other function configuration, the function configuration information input by the user is sent to the industrial equipment for corresponding function configuration.

[0083] Step S310, determine whether the human-computer interaction is finished; if not, return to the above step S307 to continue the loop execution; if so, end the process.

[0084] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0085] This embodiment also provides a human-computer interaction device based on a large model, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0086] Figure 4 is a structural block diagram of a human-computer interaction device based on a large model according to an embodiment of the present application, such as Figure 4 As shown, the device includes: an input module, a graph construction module, a structured parameter module, a guide module and a configuration interaction module; wherein: An input module 41 is used to input the acquired user input information into the trained target big model, and use the target big model to parse the user input information to obtain user intention information; a graph construction module 42 is used to obtain a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, the function configuration knowledge graph is queried and processed according to the user input information to obtain the target function information corresponding to the function configuration operation; a structured parameter module 43 is used to input the target function information into the target big model and output the configuration structured parameters; a guidance module 44 is used to generate guidance information based on the configuration structured parameters; a configuration interaction module 45 is used to obtain the function configuration information input by the user into the target big model based on the guidance information, and perform function configuration on the industrial equipment based on the function configuration information.

[0087] In some of the embodiments, the graph construction module 42 is also used to obtain preset camera equipment document information; the graph construction module 42 is also used to input the camera equipment document information into the target big model and construct a question and answer knowledge graph; the big model-based human-computer interaction device also includes a question and answer interaction module; the question and answer interaction module is used to query the question and answer knowledge graph according to the user input information when it is detected that the user intention information indicates a knowledge question and answer operation for the camera equipment, and obtain the target question and answer information corresponding to the knowledge question and answer operation; the question and answer interaction module is also used to input the target question and answer information into the target big model and generate a target reply result.

[0088] In some of the embodiments, the above-mentioned human-computer interaction device based on the large model also includes a model fine-tuning module; the model fine-tuning module is used to obtain preset security knowledge data; according to the security knowledge data, the initial large model is fine-tuned and a target large model is generated.

[0089] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination. For specific examples in this embodiment, reference can be made to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0090] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0091] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0092] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S1, obtaining user intention information input by the user.

[0093] S2, obtaining a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, querying and processing the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation.

[0094] S3, input the target function information into the target large model, and output the configuration structured parameters.

[0095] S4, generating guidance information based on the configuration structured parameters; S5, obtaining the function configuration information input by the user to the target large model based on the guidance information, and performing function configuration on the industrial equipment based on the function configuration information.

[0096] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0097] In addition, in combination with the large model-based human-computer interaction method in the above embodiments, the present application embodiment can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the large model-based human-computer interaction methods in the above embodiments is implemented.

[0098] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0100] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A human-computer interaction method based on a large model, characterized in that: The method comprises: Obtain user input information, and use the trained target large model to parse the user input information to obtain user intent information; Acquire a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, query the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation; Inputting the target function information into the target macro model and outputting configuration structured parameters; generating guidance information based on the configuration structured parameters; The function configuration information input by the user to the target large model based on the guidance information is obtained, and the function configuration of the industrial equipment is performed based on the function configuration information.

2. The human-computer interaction method according to claim 1, characterized in that: After parsing the user input information to obtain user intention information, the method further includes: Obtain preset industrial equipment document information; Input the industrial equipment document information into the target big model and construct a question-answer knowledge graph; In the case where it is detected that the user intention information indicates a knowledge question and answer operation for the industrial equipment, query processing is performed on the question and answer knowledge graph according to the user input information to obtain target question and answer information corresponding to the knowledge question and answer operation; The target question and answer information is input into the target macro model, and a target answer result is generated.

3. The human-computer interaction method according to claim 1, characterized in that: The step of inputting the target function information into the target macro model and outputting configuration structured parameters comprises: Input the target function information into the target macro model, and output the operation steps, parameter list and parameter description information corresponding to the target function information; The configuration structured parameters are generated according to the operation steps, the parameter list and the parameter description information.

4. The human-computer interaction method according to claim 3, characterized in that: The generating of the guidance information based on the configuration structured parameters comprises: Based on each of the operation steps in the configuration structured parameters, generating first guidance information; Determine a parameter name corresponding to the operation step from the parameter list, and determine target description information corresponding to the parameter name from each parameter description information; Based on the parameter name and the target description information, second guidance information is generated; wherein the guidance information includes the first guidance information and the second guidance information.

5. The human-computer interaction method according to claim 4, characterized in that: The obtaining of the function configuration information input by the user to the target macro model based on the guidance information includes: Traversing each of the operation steps in the configuration structured parameters; For the currently traversed operation step, the first guidance information and the second guidance information corresponding to the first guidance information are output to the human-computer interaction interface by using the target large model for display, and the current step configuration information input by the user to the target large model via the human-computer interaction interface based on the second guidance information is obtained; For the next operation step traversed, the interaction process between the user and the target large model is repeated until all the operation steps are traversed, the function configuration information generated based on the configuration information of each current step is obtained, and the function configuration information is input into the target large model.

6. The human-computer interaction method according to claim 5, characterized in that: The obtaining of the current step configuration information input by the user to the target large model via the human-computer interaction interface based on the second guide information includes: When it is detected that the configuration information verification of the current step fails, the target large model is used to output third guidance information to the human-computer interaction interface for display; the third guidance information is used to prompt the user to input information again; Acquire new step configuration information that the user re-enters into the target large model via the human-computer interaction interface based on the second guidance information and the third guidance information.

7. The human-computer interaction method according to claim 1, characterized in that: The step of performing functional configuration on the industrial device based on the functional configuration information includes: In the case where the function configuration operation is an intelligent algorithm configuration operation, obtaining a target category to be analyzed in the function configuration information; The target category to be analyzed is sent to the visual language model applied to the industrial equipment to set the detection target, and complete the functional configuration of the industrial equipment.

8. The human-computer interaction method according to claim 1, characterized in that: The obtaining of a preset function configuration knowledge graph includes: Get preset device usage text data; Inputting the device usage text data into the target macro model; parsing the device usage text data using the target macro model to obtain a plurality of text blocks, and identifying corresponding functional entity information and functional relationship information based on each of the text blocks; The function configuration knowledge graph is constructed according to the function entity information and the function relationship information.

9. The human-computer interaction method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquire preset security knowledge data; fine-tune the initial large model according to the security knowledge data, and generate the target large model.

10. A human-computer interaction device based on a large model, characterized in that: The device comprises: An input module, used to input the acquired user input information into the trained target large model, and use the target large model to parse the user input information to obtain user intention information; A graph construction module is used to obtain a preset function configuration knowledge graph; when the user intention information indicates a function configuration operation for industrial equipment, query the function configuration knowledge graph according to the user input information to obtain target function information corresponding to the function configuration operation; A structured parameter module, used to input the target function information into the target macro model and output configuration structured parameters; A guidance module, used for generating guidance information based on the configuration structured parameters; The configuration interaction module is used to obtain the function configuration information input by the user to the target large model based on the guidance information, and perform function configuration on the industrial equipment based on the function configuration information.

11. A human-computer interaction device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the large model-based human-computer interaction method according to any one of claims 1 to 9.

12. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the large model-based human-computer interaction method according to any one of claims 1 to 9 when running.

Images