Human-machine collaboration method based on large model

Abstract

The embodiment of the present application provides a human-machine collaboration method based on a large model, which belongs to the field of artificial intelligence technology. The method includes: starting a pre-stored large model; inputting user interaction information into the pre-stored large model; performing intent analysis on the user interaction information using the pre-stored large model; collecting production operation data in real time; performing graph query processing based on the intent analysis results and the pre-stored knowledge graph to obtain graph analysis data; collecting target production data from the visualized production information based on the intent analysis results, and recalling the target production data from the historical database to obtain integrated data; cleaning and analyzing the integrated data to generate retrieval analysis data; performing intelligent summary processing on the retrieval analysis data and / or graph analysis data, and outputting the information to be displayed; visualizing the information to be displayed for users to view. The method is used to achieve the effect of improving the scheduling efficiency and data analysis accuracy during the production operation process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a human-computer collaboration method based on a large model. Background Art

[0002] As an important part of the traditional energy industry, operators in the coal preparation industry need to actively collect data and monitor the equipment at each node in the production system in order to complete production operation scheduling based on the actively collected data and monitoring images.

[0003] The inventors found that in the prior art, due to changes and demands in the coal industry, operators only rely on collected data and monitoring images and their own experience analysis to complete production operation scheduling, which results in low scheduling efficiency and data analysis accuracy.

[0004] Therefore, there is an urgent need for a human-machine collaborative method based on large models to improve scheduling efficiency and data analysis accuracy during production operations. Summary of the invention

[0005] The embodiment of the present application provides a large model-based human-machine collaboration method to achieve the effect of improving scheduling efficiency and data analysis accuracy during production operations.

[0006] The first aspect is the human-machine collaboration method based on large models, including:

[0007] In response to a user's wake-up operation, starting the pre-stored large model;

[0008] Acquiring user interaction information, and inputting the user interaction information into the pre-stored large model;

[0009] Performing intent analysis on the user interaction information using the pre-stored large model to obtain an intent analysis result;

[0010] Collect production operation data in real time;

[0011] When it is detected that the intention analysis result contains a knowledge graph element, a graph query process is performed according to the intention analysis result and the pre-stored knowledge graph to obtain graph analysis data;

[0012] When it is detected that the intention analysis result contains business demand information, visual production information is called out according to the intention analysis result;

[0013] According to the intention analysis result, target production data is collected from the visualized production information, and according to the target production data, recall processing is performed from a historical database to obtain integrated data;

[0014] Performing cleaning and analysis processing on the integrated data to generate retrieval analysis data;

[0015] Intelligently summarizing the search analysis data and / or the graph analysis data, and outputting information to be displayed;

[0016] The information to be displayed is visualized for viewing by users.

[0017] In a possible implementation, the starting the pre-stored large model in response to the user's wake-up operation includes: generating a wake-up instruction in response to the user's wake-up operation; and starting the pre-stored large model according to the wake-up instruction.

[0018] In a possible implementation, the using the pre-stored large model to perform intent analysis on the user interaction information to obtain an intent analysis result includes: using the pre-stored large model to perform semantic understanding on the user interaction information to obtain a semantic analysis result; performing intent recognition based on the semantic analysis result to obtain an intent analysis result.

[0019] In a possible implementation, the using the pre-stored large model to perform semantic understanding processing on the user interaction information to obtain a semantic analysis result includes: using the pre-stored large model to perform format recognition processing on the user interaction information to detect the data format of the user interaction information; when it is detected that the user interaction information is in a text data format, performing semantic understanding processing on the user interaction information to obtain a semantic analysis result; when it is detected that the user interaction information is in a non-text data format, performing text conversion processing on the user interaction information to obtain text interaction information, and performing semantic understanding processing on the text interaction information to obtain a semantic analysis result.

[0020] In a possible implementation, the graph query processing is performed according to the intention analysis result and the pre-stored knowledge graph to obtain graph analysis data, including: analyzing the upstream and downstream relationships of the data related to production operations in the intention analysis result in the pre-stored knowledge graph based on the digital twin system in combination with the pre-stored knowledge graph according to the intention analysis result to obtain entity link relationships; querying the pre-stored knowledge graph according to the entity link relationship to obtain graph analysis data, wherein the graph analysis data includes production anomaly prediction information, production anomaly causes and recommended solutions.

[0021] In a possible implementation, calling out visualized production information according to the intention analysis result includes: determining a device to be called according to the intention analysis result; and calling out visualized production information from the device to be called.

[0022] In a possible implementation, the target production data is collected from the visualized production information according to the intention analysis result, and recalled from a historical database according to the target production data to obtain integrated data, including: collecting data related to the business needs as target production data from the visualized production information according to the intention analysis result; selecting industry production knowledge and database metadata from a historical database according to the target production data to determine as data to be integrated; and recalling and integrating the data to be integrated according to preset business logic to obtain integrated data.

[0023] In a possible implementation, the cleaning and analyzing processing of the integrated data to generate retrieval analysis data includes: performing data cleaning processing based on the integrated data to obtain integrated data to be analyzed; performing analysis and extraction processing based on the integrated data to be analyzed to extract key indicators and production trend information in the integrated data to be analyzed to generate retrieval analysis data.

[0024] In a possible implementation, the intelligent summarizing and processing of the retrieval analysis data and / or the graph analysis data and outputting the information to be displayed includes: intelligently summarizing and processing the retrieval analysis data by format conversion, and outputting the information to be displayed in at least one format of text, voice, table, chart, or analysis report document; or intelligently summarizing and processing the graph analysis data by format conversion, and outputting the information to be displayed in at least one format of text, voice, table, chart, or document; or intelligently summarizing and processing both the retrieval analysis data and the graph analysis data by format conversion, and outputting the information to be displayed in at least one format of text, voice, table, chart, or document.

[0025] In a possible implementation, it also includes: acquiring historical data of the coal preparation industry, and constructing a knowledge graph for the coal preparation industry based on the historical data of the coal preparation industry; training an initial large model based on the historical data of the coal preparation industry to obtain an intermediate large model applied to the coal preparation industry; integrating the knowledge graph of the coal preparation industry into the intermediate large model to obtain a pre-stored large model.

[0026] In a second aspect, an embodiment of the present application provides a large model-based human-machine collaboration device, including:

[0027] A human-computer interaction module, used to start the pre-stored large model in response to a user's wake-up operation;

[0028] The human-computer interaction module is further used to obtain user interaction information and input the user interaction information into the pre-stored large model;

[0029] An intention analysis module, used to perform intention analysis on the user interaction information using the pre-stored large model to obtain an intention analysis result;

[0030] The human-computer interaction module is also used to collect production operation data in real time;

[0031] A knowledge graph analysis module, which is used to perform graph query processing according to the intention analysis result and the pre-stored knowledge graph to obtain graph analysis data when it is detected that the intention analysis result contains a knowledge graph element;

[0032] A scheduling module, configured to call out visualized production information according to the intention analysis result when it is detected that the intention analysis result contains business demand information;

[0033] A data processing module, used for collecting target production data from the visualized production information according to the intention analysis result, and performing recall processing from a historical database according to the target production data to obtain integrated data;

[0034] The data processing module is also used to clean and analyze the integrated data to generate retrieval analysis data;

[0035] A summary output module, used to perform intelligent summary processing on the search analysis data and / or the graph analysis data, and output information to be displayed;

[0036] The summary output module is also used to visualize the information to be displayed for users to view.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0038] The memory stores computer-executable instructions;

[0039] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0041] The large-model-based human-machine collaborative device method provided in the embodiment of the present application first awakens the pre-stored large model through human-machine interaction, and then automatically realizes the functions of real-time collection of production operation data, calling up visual production information, generating retrieval analysis data and graph analysis data based on user interaction information and the predicted large model, and visualizes the final information to be displayed for user viewing, thereby optimizing the communication efficiency between the user and the terminal, and improving the user experience through a more intuitive interaction method. While improving the interaction efficiency, it also combines the knowledge graph and automatically executes the corresponding analysis or scheduling operations to improve the scheduling efficiency and data analysis accuracy during the production operation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] Figure 1 A schematic diagram of a scenario of a large-model-based human-machine collaboration method provided in this application;

[0044] Figure 2 A schematic diagram of the process of the human-machine collaboration method based on a large model provided for this application;

[0045] Figure 3 A schematic diagram of the human-computer interaction process of the human-computer collaboration method based on a large model provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of the structure of a large-scale human-machine collaborative device provided in this application;

[0047] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0048] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0049] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] The inventors found that in the prior art, there are also some semi-automatic operating systems that reduce the workload of operators. Operators manually collect monitoring data and view and analyze the collected monitoring data, and then use their own experience to write production analysis reports, as well as perform troubleshooting and maintenance. This is affected by human factors, especially because of the continuous changes and increased demand in the coal industry, which further leads to the problems of low scheduling efficiency and data analysis accuracy in the production and operation of the coal industry.

[0051] In response to the above technical problems, the inventors proposed the following technical ideas: Based on artificial intelligence AI big model calculation, combined with the existing semi-automatic operating system or intelligent system, the AI ​​big model is used through human-computer interaction to realize the intelligent scheduling of coal preparation plants, production situation reporting, equipment abnormality analysis, safety management and other capabilities. It can also receive instructions input by operators, and then combine knowledge graphs and automatically perform corresponding analysis or scheduling operations to improve scheduling efficiency and data analysis accuracy during production operations.

[0052] Figure 1 A schematic diagram of a scenario of a large-scale model-based human-machine collaboration method provided in this application, such as Figure 1 As shown, the specific application scenario of the present application includes a user 101 and a terminal 102 .

[0053] Among them, user 101 is an operator or dispatcher, and terminal 102 can be a computer, tablet computer or other hardware device that can deploy a large model. In the scenario of the human-machine collaborative method based on the large model, after the large model in terminal 102 is started and awakened by user 101, user 101 inputs interactive information to terminal 102, and the interactive information is input into the large model, and the functions of intelligent dispatching, production situation reporting, equipment abnormality analysis, safety management, etc. of the coal preparation plant are automatically realized. And the results can be output in various forms, including text, voice, statements, charts and reports.

[0054] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0055] Figure 2 This is a flow chart of the human-machine collaboration method based on a large model provided in this application. The execution subject of this embodiment can be Figure 1 The terminal 102 in the illustrated embodiment may also be other computer-related devices, which will not be described in detail in this embodiment.

[0056] like Figure 2 As shown, the human-machine collaboration method based on the large model includes the following steps:

[0057] S201: In response to a user's wake-up operation, start a pre-stored large model.

[0058] In this embodiment, the user's wake-up operation can be a simple voice control operation, gesture operation, or input operation. Based on these wake-up operations, the pre-stored large model is started.

[0059] Specifically, in an optional embodiment of the present application, step S201 includes:

[0060] S201a: In response to a user's wake-up operation, generate a wake-up instruction.

[0061] S201b: Start the pre-stored large model according to the wake-up instruction.

[0062] In this embodiment, the wake-up instruction refers to a control instruction generated in response to the user's wake-up operation. The wake-up instruction can automatically control the start of the pre-stored large model to complete the opening of the human-computer interaction window. Among them, the human-computer interaction window allows the user to input interaction information to the pre-stored large model in the form of voice input or gesture input.

[0063] S202: Obtain user interaction information, and input the user interaction information into a pre-stored large model.

[0064] In this embodiment, the user interaction information can be acquired through the peripheral hardware of the terminal, for example, it is used to operate the mouse, keyboard or directly issue voice commands and is acquired by the microphone installed on the terminal. The user interaction information can be information related to the production process, such as scheduling requirements and business requirements proposed by the user. For example, the user inputs the user interaction information in the form of text, voice or file, and enters it into the human-computer interaction window in the pre-stored large model.

[0065] S203: Use the pre-stored large model to perform intent analysis on the user interaction information to obtain intent analysis results.

[0066] In this embodiment, the pre-stored large model can perform steps such as keyword extraction and text recognition on the input user interaction information to complete intent analysis processing, so as to obtain intent analysis results that can express the user's actual needs.

[0067] In an optional embodiment of the present application, step S203 includes:

[0068] S203a: Use the pre-stored large model to perform semantic understanding processing on the user interaction information to obtain a semantic analysis result.

[0069] In this embodiment, the semantic understanding process can be a process of obtaining a semantic analysis result by performing semantic analysis on user interaction information using a natural language processing model in a pre-stored large model.

[0070] Further, in an optional embodiment of the present application, step S203a specifically includes:

[0071] a1: Use the pre-stored large model to perform format recognition processing on the user interaction information to detect the data format of the user interaction information.

[0072] a2: When it is detected that the user interaction information is in a text data format, semantic understanding processing is performed on the user interaction information to obtain a semantic analysis result.

[0073] a3: When it is detected that the user interaction information is in a non-text data format, the user interaction information is converted into text to obtain text interaction information, and the text interaction information is semantically understood to obtain a semantic analysis result.

[0074] In this embodiment, when the user interaction information input by the user into the pre-stored large model is information in the text data format of text information, the natural language processing model can be directly used for semantic understanding, and the text content related to the production process in the user interaction information can be analyzed and extracted as the semantic analysis result. When the user interaction information input by the user into the pre-stored large model is information in a non-text data format such as a file or voice information, the voice content must first be converted into text content to obtain text interaction information, and then the text interaction information must be processed for voice understanding to obtain a semantic analysis result for the subsequent steps of determining the user's intention.

[0075] S203b: Perform intent recognition processing based on the semantic analysis result to obtain the intent analysis result.

[0076] In this embodiment, the intention recognition process can be a process of analyzing the semantic analysis results to determine the user's real intention at this time. For example, when the user is a dispatcher of a coal preparation plant, the user interaction information input is "I want to see the production situation of the coal screening workshop". Then the intention analysis result obtained after step S203 is "call the monitoring screen of the production equipment of the coal screening workshop".

[0077] S204: Collect production operation data in real time.

[0078] In this embodiment, the production operation data may be real-time production data, equipment operation data, and operation logs, so that the subsequent pre-stored large model can analyze the data changes before and after the production anomaly occurs based on these data and quickly locate the cause of the production anomaly.

[0079] S205: When it is detected that the intent analysis result contains knowledge graph elements, graph query processing is performed based on the intent analysis result and the pre-stored knowledge graph to obtain graph analysis data.

[0080] In this embodiment, the knowledge graph elements may be entities or relationships in a pre-stored knowledge graph. Graph query includes the process of obtaining data that can be used for entity linking after natural language query to complex language query NL2CQL, entity recognition and relationship recognition. In this embodiment, the graph analysis data may be based on the pre-stored knowledge graph to analyze the relationship between production operation data and upstream and downstream, infer possible production anomalies, and the causes and solutions to the production anomalies.

[0081] In an optional embodiment of the present application, step S205 includes:

[0082] S205a: According to the intention analysis results, based on the digital twin system and the pre-stored knowledge graph, the upstream and downstream relationships of the data related to production operations in the intention analysis results in the pre-stored knowledge graph are analyzed to obtain the entity link relationship.

[0083] In this embodiment, the digital twin system analyzes the upstream and downstream relationships of the data related to production operation in the pre-stored knowledge graph in the intent analysis results in combination with the pre-stored knowledge graph. The process includes: NL2CQL, entity recognition and relationship recognition. Among them, NL2CQL is to convert natural language queries into complex language queries, and use complex query languages ​​to operate inside the pre-stored knowledge graph. The NL2CQL technology can convert natural language queries into query statements that can be understood by the knowledge graph, thereby realizing user-friendly and efficient query functions. Entity recognition refers to the process of identifying entities with specific meanings in the intent analysis results and classifying these entities. The entities here can be noun phrases, such as equipment names or workshop serial numbers. Relationship recognition can be the process of identifying relationships between entities. The relationships between entities describe the interactions and attributes between entities. Relationship recognition can help build connections between entities and form a complete knowledge graph structure.

[0084] S205b: query the pre-stored knowledge graph according to the entity link relationship to obtain graph analysis data, wherein the graph analysis data includes production anomaly prediction information, production anomaly causes and recommended solutions.

[0085] In this embodiment, entity linking relationship querying a pre-stored knowledge graph refers to the process of linking previously obtained entities and relationships to corresponding entities in the pre-stored knowledge graph.

[0086] S206: When it is detected that the intent analysis result contains business demand information, visualized production information is called out according to the intent analysis result.

[0087] In this embodiment, the business demand information may be information related to the business such as control, query, and scheduling proposed by the user. The visual production information may be a video screen or a production system operation page, production data in the production process, and the like.

[0088] Specifically, in an optional embodiment of the present application, calling out visualized production information according to the intention analysis result in step S206 includes:

[0089] S206a: Determine the device to be called according to the intention analysis result.

[0090] S206b: Retrieve visual production information from the equipment to be called.

[0091] In this embodiment, the device to be called can be a production device, a control device, an image acquisition device or a data acquisition device. Calling up visual production information refers to the process of downloading visual information from the device to be called and displaying it. For example: calling up a video screen or a production system operation page. This can achieve more accurate scheduling of various production tasks and real-time monitoring of the production site, thereby ensuring that every detail in the production process can be grasped by the user in a timely manner. The user only needs to use simple voice commands to automatically complete the subsequent steps, which not only greatly reduces the cumbersome operating steps in the traditional scheduling method, but also significantly improves the response speed and accuracy of scheduling decisions.

[0092] S207: Collect target production data from the visualized production information according to the intention analysis result, and perform recall processing from the historical database according to the target production data to obtain integrated data.

[0093] In this embodiment, the recall process can be a retrieval enhancement generation RAG technology, which can split the knowledge base used for retrieval into multiple small text blocks and finally perform context recall. Integrating data refers to the process of integrating and analyzing data after recalling data from the historical database according to the target production data.

[0094] Specifically, in an optional embodiment of the present application, step S207 includes:

[0095] S207a: According to the intention analysis result, data related to the business demand is collected from the visualized production information as target production data.

[0096] S207b: Select industry production knowledge and database meta-information from the historical database according to the target production data to determine as data to be integrated.

[0097] S207c: The data to be integrated is recalled and integrated according to the preset business logic to obtain integrated data.

[0098] In this embodiment, the preset business logic may be a calculation formula for relevant indicators of production data, and the historical database may include historical production data, historical summary data, and historical production status data. Industry production knowledge may be industry knowledge such as professional terms and operating procedures. Database meta-information may be an enumeration value. Integrated data may be data to be integrated into content that can be output for intuitive reference by users, for example: integrated data may be data such as tables, charts, texts, voices, or reports containing industry production knowledge and database meta-information.

[0099] S208: Clean and analyze the integrated data to generate retrieval analysis data.

[0100] Cleaning analysis refers to the process of deleting erroneous or incomplete data in the integrated data, identifying and selecting useful data. Retrieval analysis data is the raw data used to display visual information, such as arrays in chart data.

[0101] In an optional embodiment of the present application, step S208 includes:

[0102] S208a: Perform data cleaning processing based on the integrated data to obtain integrated data to be analyzed.

[0103] S208c: Perform analysis and extraction processing based on the integrated data to be analyzed, extract key indicators and production trend information in the integrated data to be analyzed, and generate retrieval analysis data.

[0104] In this embodiment, based on the input data and business needs, the pre-stored large model can automatically generate a report containing key information and analysis, that is, collect and integrate relevant data from the production system, and then perform data cleaning processing. After deleting a part of the data, the integrated data to be analyzed is obtained, and then key indicators and production trend information are extracted from the integrated data to be analyzed, and retrieval analysis data is obtained for users to query data and analysis results through natural language models.

[0105] S209: Intelligently summarize the search analysis data and / or the graph analysis data and output the information to be displayed.

[0106] In this embodiment, intelligent summarization refers to the process of integrating any one of the retrieval analysis data and the image analysis data or both of them into the same visualization screen for output. For example, production data such as production operation indicators of the corresponding equipment will be displayed in real time in the called monitoring screen.

[0107] Specifically, in an optional embodiment of the present application, step S209 includes:

[0108] S209a: Intelligently summarize and process the retrieved and analyzed data by means of format conversion, and output the information to be displayed in at least one format of text, voice, table, chart or analysis report document.

[0109] S209b: Or intelligently summarize and process the graph analysis data by format conversion, and output the information to be displayed in at least one format of text, voice, table, chart or document.

[0110] S209c: Alternatively, both the search analysis data and the graph analysis data may be intelligently summarized and processed by format conversion, and the information to be displayed may be output in at least one of the following formats: text, voice, table, chart or document.

[0111] In this embodiment, the information to be displayed may be text, table, voice, report, chart or report, document, or other information that is convenient for users to view.

[0112] S2010: Visualize the information to be displayed for viewing by the user.

[0113] In this embodiment, the visualization processing refers to the process of playing or displaying the information to be displayed on the display of the terminal, or the process of playing the information to be displayed in the form of sound by external speakers.

[0114] Based on the above embodiment, a large model-based human-machine collaboration method provided in an optional embodiment of the present application further includes:

[0115] Step A: Obtain historical data of the coal preparation industry and build a knowledge graph of the coal preparation industry based on the historical data of the coal preparation industry.

[0116] In this embodiment, the historical data of the coal preparation industry may include historical production summary data, historical operation logs, historical analysis reports, and production management knowledge and experience content provided by experienced users or coal preparation experts on site. In order to build a knowledge graph related to the coal preparation industry, complete the knowledge accumulation of the coal preparation industry, help subsequent operators solve practical problems in production, and improve work efficiency. It can also help new operators quickly understand and learn relevant production equipment and production processes and other knowledge. Reduce the time cost and material cost of personnel training. And based on the knowledge graph constructed in combination with the digital twin system, analyze the relationship between production operation data and upstream and downstream, so that users can promptly obtain analysis reports containing possible production abnormalities, causes of abnormalities, and recommended solutions.

[0117] Step B: Train the initial large model based on the historical data of the coal preparation industry to obtain an intermediate large model applied to the coal preparation industry.

[0118] Step C: Integrate the coal preparation industry knowledge graph into the intermediate large model to obtain the pre-stored large model.

[0119] In this embodiment, the initial large model is trained using historical data of the coal preparation industry, and then the knowledge graph of the coal preparation industry is integrated into the intermediate large model, which can greatly improve the practicality and accuracy of the pre-stored large model in the specific field of the coal preparation industry.

[0120] Figure 3 A schematic diagram of the human-computer interaction process of the large model-based human-computer collaboration method provided in an embodiment of the present application.

[0121] like Figure 3 As shown, the user can input the user interaction information into the human-computer interaction window of the pre-stored large model after startup through text, voice or file to enter the question and answer processing process. When there is only voice user interaction information, semantic understanding and intent determination are directly performed. When it is detected that the intent analysis result contains business demand information, recall processing can be performed. The recall uses business logic such as indicator calculation formulas, industry knowledge such as professional terms, and data meta information such as enumeration values. When it is detected that the intent analysis result contains knowledge graph elements, NL2CQL, entity recognition and relationship recognition can be performed, and then entity linking and graph query can be performed to obtain graph analysis data based on the pre-stored knowledge graph. Finally, the retrieval analysis data and graph analysis data are intelligently summarized, and the information to be displayed is output in the form of text, voice, tables, icons and reports for users to view.

[0122] In summary, the human-computer collaboration method based on a big model provided in the embodiment of the present application first wakes up the pre-stored big model through human-computer interaction, and then automatically realizes real-time collection of production operation data, calls up visual production information, generates retrieval analysis data and graph analysis data based on user interaction information and the predictive big model, and visualizes the final information to be displayed for user viewing, thereby optimizing the communication efficiency between the user and the terminal, and improving the user experience through a more intuitive interaction method. While improving the interaction efficiency, it also combines the knowledge graph and automatically executes the corresponding analysis or scheduling operations to improve the scheduling efficiency and data analysis accuracy during the production operation process.

[0123] At the same time, by utilizing historical production data containing rich coal preparation industry expertise and data to build knowledge graphs and train large models, the final pre-existing large model can provide accurate industry analysis, decision support and problem solving, thereby enhancing the practicality and accuracy of the model in specific fields.

[0124] At the same time, it also supports multiple input and output methods, including text, voice, statements, charts and reports, to meet the business needs and preferences of different users, and improve the flexibility and applicability of human-computer interaction.

[0125] Of course, the human-machine collaboration method based on a large model provided in an optional embodiment of the present application can also be applied to other specific industries of processing and production other than the coal preparation industry to improve production efficiency. The pre-stored large model can also be used to handle a variety of tasks and applications, not limited to a specific field. Such models are trained on large-scale multi-field data sets to learn a wide range of knowledge and skills. This embodiment is not limited to this.

[0126] Figure 4 The schematic diagram of the structure of the human-machine collaborative device based on the large model provided in this application is as follows: Figure 4 As shown, the human-computer collaboration device based on the large model provided in this embodiment includes: a human-computer interaction module 41, an intention analysis module 42, a knowledge graph analysis module 43, a scheduling module 44, a data processing module 45 and a summary output module 46.

[0127] The human-computer interaction module 41 is used to start the pre-stored large model in response to the user's wake-up operation.

[0128] The human-computer interaction module 41 is also used to obtain user interaction information and input the user interaction information into a pre-stored large model.

[0129] The intention analysis module 42 is used to perform intention analysis on user interaction information using a pre-stored large model to obtain an intention analysis result.

[0130] The human-computer interaction module 41 is also used to collect production operation data in real time.

[0131] The knowledge graph analysis module 43 is used to perform graph query processing based on the intention analysis results and the pre-stored knowledge graph to obtain graph analysis data when it is detected that the intention analysis results contain knowledge graph elements.

[0132] The scheduling module 44 is used to call out the visualized production information according to the intention analysis result when it is detected that the intention analysis result contains business demand information.

[0133] The data processing module 45 is also used to collect target production data from the visualized production information according to the intention analysis result, and to perform recall processing from the historical database according to the target production data to obtain integrated data.

[0134] The data processing module 45 is also used to clean and analyze the integrated data to generate retrieval analysis data.

[0135] The summary output module 46 is used to perform intelligent summary processing on the search analysis data and / or the graph analysis data and output the information to be displayed.

[0136] The summary output module 46 is also used to visualize the information to be displayed for users to view.

[0137] In an optional embodiment of the present application, the human-computer interaction module 41 is specifically used to: generate a wake-up instruction in response to a user's wake-up operation; and start the pre-stored large model according to the wake-up instruction.

[0138] In an optional embodiment of the present application, the intent analysis module 42 is specifically used to: use a pre-stored large model to perform semantic understanding processing on user interaction information to obtain a semantic analysis result; perform intent recognition processing based on the semantic analysis result to obtain an intent analysis result.

[0139] In an optional embodiment of the present application, the intent analysis module 42 is further specifically used to: utilize a pre-stored large model to perform format recognition processing on the user interaction information to detect the data format of the user interaction information; when it is detected that the user interaction information is in a text data format, then perform semantic understanding processing on the user interaction information to obtain a semantic analysis result; when it is detected that the user interaction information is in a non-text data format, then perform text conversion processing on the user interaction information to obtain text interaction information, and perform semantic understanding processing on the text interaction information to obtain a semantic analysis result.

[0140] In an optional embodiment of the present application, the knowledge graph analysis module 43 is specifically used to: analyze the upstream and downstream relationships of the data related to production operations in the intention analysis results in the pre-stored knowledge graph based on the digital twin system and the pre-stored knowledge graph according to the intention analysis results, and obtain the entity link relationship; query the pre-stored knowledge graph according to the entity link relationship to obtain graph analysis data, wherein the graph analysis data includes production anomaly prediction information, production anomaly causes and recommended solutions.

[0141] In an optional embodiment of the present application, the scheduling module 44 is specifically used to: determine the equipment to be called according to the intention analysis result; and call out the visualized production information from the equipment to be called.

[0142] In an optional embodiment of the present application, the data processing module 45 is specifically used to: collect data related to business needs from the visualized production information as target production data according to the intention analysis results; select industry production knowledge and database metadata from the historical database as data to be integrated according to the target production data; recall and integrate the data to be integrated according to preset business logic to obtain integrated data.

[0143] In an optional embodiment of the present application, the data processing module 45 is further specifically used to: perform data cleaning processing based on the integrated data to obtain the integrated data to be analyzed; perform analysis and extraction processing based on the integrated data to be analyzed, extract key indicators and production trend information in the integrated data to be analyzed to generate retrieval analysis data.

[0144] In an optional embodiment of the present application, the summary output module 46 is used to: perform intelligent summary processing on the retrieval analysis data by means of format conversion, and output the information to be displayed in at least one format of text, voice, table, chart or analysis report document; or perform intelligent summary processing on the graph analysis data by means of format conversion, and output the information to be displayed in at least one format of text, voice, table, chart or document; or perform intelligent summary processing on both the retrieval analysis data and the graph analysis data by means of format conversion, and output the information to be displayed in at least one format of text, voice, table, chart or document.

[0145] In an optional embodiment of the present application, the data processing module 45 is also used to: obtain historical data of the coal preparation industry, and construct a knowledge graph for the coal preparation industry based on the historical data of the coal preparation industry; perform model training on the initial large model based on the historical data of the coal preparation industry to obtain an intermediate large model applied to the coal preparation industry; integrate the knowledge graph of the coal preparation industry into the intermediate large model to obtain a pre-stored large model.

[0146] The large-model-based human-machine collaborative device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0147] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the electronic device further includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0148] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0149] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0150] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0151] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0152] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0153] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0154] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0155] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0156] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0157] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0158] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0159] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0162] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0163] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0164] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A human-machine collaboration method based on a large model, characterized in that: include: In response to a user's wake-up operation, starting the pre-stored large model; Acquiring user interaction information, and inputting the user interaction information into the pre-stored large model; Performing intent analysis on the user interaction information using the pre-stored large model to obtain an intent analysis result; Collect production operation data in real time; When it is detected that the intention analysis result contains a knowledge graph element, a graph query process is performed according to the intention analysis result and the pre-stored knowledge graph to obtain graph analysis data; When it is detected that the intention analysis result contains business demand information, visual production information is called out according to the intention analysis result; According to the intention analysis result, data related to the business demand is collected from the visualized production information as target production data; Selecting industry production knowledge and database meta-information from a historical database according to the target production data to determine as data to be integrated; Recalling and integrating the data to be integrated according to preset business logic to obtain integrated data; Performing cleaning and analysis processing on the integrated data to generate retrieval analysis data; Intelligently summarizing the search analysis data and / or the graph analysis data, and outputting information to be displayed; The information to be displayed is visualized for viewing by users.

2. The method according to claim 1, characterized in that The method of starting the pre-stored large model in response to the user's wake-up operation includes: In response to a user's wake-up operation, generating a wake-up instruction; Start the pre-stored large model according to the wake-up instruction.

3. The method according to claim 1, characterized in that The using the pre-stored large model to perform intent analysis on the user interaction information to obtain an intent analysis result includes: Using the pre-stored large model to perform semantic understanding processing on the user interaction information to obtain a semantic analysis result; Perform intent recognition processing based on the semantic analysis result to obtain an intent analysis result.

4. The method according to claim 3, characterized in that The using the pre-stored large model to perform semantic understanding processing on the user interaction information to obtain a semantic analysis result includes: Performing format recognition processing on the user interaction information using the pre-stored large model to detect the data format of the user interaction information; When it is detected that the user interaction information is in a text data format, semantic understanding processing is performed on the user interaction information to obtain a semantic analysis result; When it is detected that the user interaction information is in a non-text data format, the user interaction information is subjected to text conversion processing to obtain text interaction information, and semantic understanding processing is performed on the text interaction information to obtain a semantic analysis result.

5. The method according to claim 1, characterized in that The graph query processing is performed according to the intention analysis result and the pre-stored knowledge graph to obtain the graph analysis data, including: According to the intention analysis result, based on the digital twin system and the pre-stored knowledge graph, the upstream and downstream relationships of the data related to the production operation in the intention analysis result in the pre-stored knowledge graph are analyzed to obtain the entity link relationship; The pre-stored knowledge graph is queried according to the entity link relationship to obtain graph analysis data, wherein the graph analysis data includes production anomaly prediction information, production anomaly causes and recommended solutions.

6. The method according to claim 1, characterized in that The calling out of the visualized production information according to the intention analysis result includes: Determine the device to be called according to the intention analysis result; The visualized production information is called out from the equipment to be called.

7. The method according to claim 1, characterized in that The step of performing cleaning and analysis on the integrated data to generate retrieval analysis data includes: Performing data cleaning processing on the integrated data to obtain integrated data to be analyzed; An analysis and extraction process is performed on the integrated data to be analyzed, and key indicators and production trend information in the integrated data to be analyzed are extracted to generate retrieval analysis data.

8. The method according to claim 1, characterized in that The intelligent summarizing and processing of the search analysis data and / or the graph analysis data to output information to be displayed includes: Intelligently summarizing the search and analysis data by format conversion, and outputting the information to be displayed in at least one format of text, voice, table, chart or analysis report document; or Intelligently summarizing the graph analysis data by format conversion, and outputting the information to be displayed in at least one of the following formats: text, voice, table, chart or document; or The retrieval analysis data and the graph analysis data are both intelligently summarized and processed by means of format conversion, and the information to be displayed is output in at least one format of text, voice, table, chart or document.

9. The method according to any one of claims 1 to 8, characterized in that Also includes: Acquire historical data of the coal preparation industry, and construct a knowledge graph of the coal preparation industry based on the historical data of the coal preparation industry; Performing model training on the initial large model according to the historical data of the coal preparation industry to obtain an intermediate large model applied to the coal preparation industry; The coal preparation industry knowledge graph is integrated into the intermediate large model to obtain a pre-stored large model.

Images