A method for industrial data mining and interaction

Abstract

The present disclosure provides an industrial data mining and interaction method. It includes: constructing a knowledge base of the target industry based on the data of the target industry; pre-training a multimodal large model based on the data of the knowledge base; sending requirements to the multimodal large model through a human-computer interaction interface, and classifying the requirements when the multimodal large model understands the requirements correctly; obtaining corresponding data from a private database based on the classified requirements; and displaying the corresponding data on the human-computer interaction interface. By pre-training the data of the target industrial field based on the multimodal large model, and combining the human-computer interaction interface to understand the requirements and classify the data, extracting the corresponding data from the private database, and displaying it on the interactive interface, efficient mining and analysis of complex industrial data is achieved, and the accuracy of data processing and the intelligent level of interaction are improved.

Description

Technical Field

[0001] The present invention relates to the field of data mining research, and in particular to an industrial data mining and interaction method, electronic equipment and storage medium. Background Art

[0002] Industry is one of the important pillar industries in my country, and its intelligence level has a great impact on the development of my country's economy. With the development of technologies such as big data, artificial intelligence, and cloud computing, traditional industries have been promoted to transform into smart factories through intelligent transformation and digital transformation. However, in actual production, data mining is insufficient due to factors such as the different status of industrial equipment and the influence of process, and there is a dilemma that technical personnel are limited to business and business personnel are limited to technology.

[0003] Since the release of ChatGPT, artificial intelligence has entered the era of large models. Large models based on the Transformer technology base learn more complex features and patterns through hundreds of millions of parameters, thereby improving the ability to fit input data, improving the prediction accuracy of the model, and making the model more intelligent. However, large models are currently mainly used in general fields, and the application of large models in industry verticals is still under research and exploration.

[0004] In order to solve the above problems, an industrial data mining and interaction method, an electronic device and a storage medium are proposed here. Summary of the invention

[0005] The embodiments of the present disclosure aim to solve at least one of the technical problems existing in the prior art, and provide an industrial data mining and interaction method, an electronic device and a storage medium. By pre-training the data of the target industrial field based on a multimodal large model, and combining the human-computer interaction interface to understand the needs and classify the data, the corresponding data is extracted from the private database and displayed on the interactive interface, thereby realizing efficient mining and analysis of complex industrial data and improving the accuracy of data processing and the intelligence level of interaction.

[0006] A first aspect of the present disclosure provides an industrial data mining and interaction method, comprising:

[0007] Building a knowledge base of the target industry based on the data of the target industry;

[0008] Pre-training a multimodal large model based on the data of the knowledge base;

[0009] Sending requirements to the multimodal big model through a human-computer interaction interface, and classifying the requirements if the multimodal big model understands the requirements correctly;

[0010] According to the classified requirements, corresponding data is obtained from the private database;

[0011] The acquired corresponding data is displayed on the human-computer interaction interface.

[0012] In conjunction with the first aspect, constructing a knowledge base of the target industry based on the data of the target industry includes:

[0013] Obtain text, image, audio, and video data of the target industry, and name the data files according to the process, equipment type, and production index;

[0014] Importing the file names of the data files into the same text to extract word frequency features, and clustering the word frequency features through a clustering algorithm to form a similar file set;

[0015] According to the similar file set, the data files are classified and stored to form a knowledge base of the target industry.

[0016] In combination with the first aspect, pre-training the multimodal large model according to the data of the knowledge base includes:

[0017] Use the following formula to construct a multimodal large model training batch:

[0018] ,

[0019] in, is a collection of similar files; For the similar files in the collection kind, ; For the A collection of similar files; The percentage threshold of the selected files among similar files; No. A collection of similar files; No. A collection of similar files;

[0020] According to the multimodal large model training batches, all batches of training data are sequentially sent to the variant NExT-GPT multimodal large model for training to obtain an industrial multimodal large model.

[0021] In combination with the first aspect, the method further includes sending a requirement to the multimodal large model through the human-computer interaction interface, receiving a return understanding from the multimodal large model, and classifying the requirement when the user confirms that the requirement is understood correctly.

[0022] In combination with the first aspect, the method further includes prompting the user to supplement or explain the demand through the human-computer interaction interface when the user confirms that the demand is misunderstood, and allowing the multimodal large model to update the current round of dialogue state in combination with the previous round of dialogue state.

[0023] Combined with the first aspect, the classification of needs includes:

[0024] Establish matching verbs for the first type of requirements and the second type of requirements;

[0025] The similarity between the verb in the requirement and the matching verb is calculated according to the following formula:

[0026] ,

[0027] in, is the demand type, The first word vectors; is the first elements; is the word vector length; For the first type of matching verb set word vectors; For the second type of matching verb set word vectors;

[0028] The requirements are divided into first type requirements and second type requirements according to the similarity of the matching verbs.

[0029] In combination with the first aspect, the matching verbs for the first type of requirements include "analyze", "display", "calculate", and "statistics"; the matching verbs for the second type of requirements include "predict", "identify", "detect", "classify", and "cluster".

[0030] In combination with the first aspect, obtaining corresponding data from a private database according to the classified needs includes:

[0031] According to the verbs and nouns in the first type of demand, corresponding data of historical working conditions in the private database are retrieved to obtain the corresponding results of the first type of demand;

[0032] According to the verbs and nouns in the second type of demand, the corresponding data of the historical working conditions in the private database are retrieved and combined with the knowledge base of the target industry to obtain the corresponding results of the second type of demand.

[0033] According to a second aspect of the present disclosure, there is provided an electronic device, comprising:

[0034] one or more processors;

[0035] A storage unit is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any one of the industrial data mining and interaction methods.

[0036] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the industrial data mining and interaction methods can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flowchart of an industrial data mining and interaction method according to an embodiment of the present disclosure;

[0038] Figure 2 for Figure 1 Schematic diagram of the process of step S101;

[0039] Figure 3 for Figure 1 Schematic diagram of the process of step S102;

[0040] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the embodiments of the present disclosure.

[0042] The terms used in the disclosed embodiments are only for the purpose of describing specific embodiments and are not intended to limit the disclosed embodiments. The singular forms of "a", "said" and "the" used in the disclosed embodiments and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the disclosed embodiments, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the disclosed embodiments, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0044] like Figure 1 FIG. 1 is a flow chart of an industrial data mining and interaction method according to an embodiment of the present disclosure. The method comprises the following steps:

[0045] S101: constructing a knowledge base of the target industry according to the data of the target industry;

[0046] S102: Pre-training a multimodal large model according to the data of the knowledge base;

[0047] S103: Sending requirements to the multimodal big model through a human-computer interaction interface, and classifying the requirements if the multimodal big model understands the requirements correctly;

[0048] S104: Obtain corresponding data from a private database according to the classified requirements;

[0049] S105: Displaying the acquired corresponding data on the human-computer interaction interface.

[0050] Refer to the following Figure 2-Figure 4 The method is described in detail.

[0051] like Figure 2 FIG. 1 is a flow chart of step S101. The flow chart includes:

[0052] S201: Acquire text, image, audio, and video data of the target industry, and name the data files according to the process, equipment type, and production index;

[0053] S202: Import the file names of the data files into the same text to extract word frequency features, and cluster the word frequency features using a clustering algorithm to form a similar file set;

[0054] S203: According to the similar file set, the data files are classified and stored to form a knowledge base of the target industry.

[0055] Specifically, the data of the target industry include text, images, audio, and video data of the target industry, among which text data include electronic versions of professional books, papers, technical documents, website-related news reports, equipment introduction documents, production logs, work reports, etc.; image data include raw cotton images, yarn images, equipment key components operation images, etc.; audio data include meeting recording audio, professional report recording audio, business exchange audio, technical workers special recording audio, etc.; video data include equipment operation videos, digital twin videos, etc.

[0056] Furthermore, the collected data is classified according to the process, equipment type, and production index. For example, the data can be divided into equipment operation data, process parameter data, maintenance records, etc.

[0057] And name the classified data. For example, the naming rule can be "equipment type-process-timestamp-data type", such as:

[0058] "Pump1_Assembly_20240813_TemperatureData".

[0059] Furthermore, Python's os library is used to read all multimodal data file names and save them in the same text file. The TF-IDF algorithm is used to extract word frequency features from the multimodal file name text. The K-means clustering algorithm is used to cluster the word frequency features based on the extracted word frequency features to form a set of K similar files.

[0060] Finally, according to the similar file set, the data files are classified and stored to form a knowledge base of the target industry.

[0061] like Figure 3 FIG. 1 is a flow chart of step S102. The flow chart includes:

[0062] S301: Build a multimodal large model training batch;

[0063] S302: According to the multimodal large model training batches, all batches of training data are sequentially sent to the variant NExT-GPT multimodal large model for training to obtain an industrial multimodal large model.

[0064] Among them, the following formula is used to construct a multimodal large model training batch:

[0065] ,

[0066] in, is a collection of similar files; For the similar files in the collection kind, ; For the A collection of similar files; The percentage threshold of the selected files among similar files; No. A collection of similar files; No. A collection of similar files.

[0067] Exemplarily, when K is 4 and 30, the multimodal data is divided into 4 batches for large model training. The first round is fed with the first type of multimodal data and 30% of the second type of multimodal data, the second round is fed with 30% of the first type of multimodal data, the second type of multimodal data and 30% of the third type of multimodal data, the third round is fed with 30% of the second type of multimodal data, the third type of multimodal data and 30% of the fourth type of multimodal data, and the fourth round is fed with 30% of the third type of multimodal data and the fourth type of multimodal data.

[0068] The variant NExT-GPT multimodal large model framework includes a multimodal data input layer, a multimodal encoding layer, an input linear layer, a general large model reasoning layer, and a data output layer. The LanguageBind framework is used to encode image data, audio data, and video data, and the three modalities are aligned with the text data in a shared embedding space. The aligned data is input into the general open source large model Falcon through the linear layer for reasoning, and the text information is output after the linear layer. The linear layer parameters are updated during the training process. At this time, the pre-trained large model is an understanding model of the user input requirements. The output of the multimodal large model is not the output that is finally presented in the human-computer interaction interface, but an intermediate model.

[0069] The multimodal alignment process is as follows: Let the text modal data be , other modal data are , for data pairs , given a text and another modality data , after an optional deep neural network , encoded into a normalized embedding , , and use the loss function ,

[0070] Perform iterative optimization to align .in, is the temperature coefficient. Optional multi-head attention network, The value can be 0.5.

[0071] The requirements are sent to the multimodal big model through a human-computer interaction interface, and the requirements are classified if the multimodal big model understands the requirements correctly.

[0072] Specifically, the user sends the task requirements in the form of text or voice in the human-computer interaction interface, where the task requirements at least clearly specify the object to be analyzed and the analysis period. The trained big model understands the user's natural language and repeats the understood requirements. If the user confirms that the requirements are understood correctly, the next process is entered; if the requirements are understood incorrectly, the user is asked to supplement or explain the requirements. The big model updates the current round of dialogue status through dialogue state tracking combined with the previous round of dialogue status until the user confirms that the requirements are understood correctly.

[0073] For example, the user sends task requirements such as "analyze the production of all varieties in a workshop last week" and "detect abnormal equipment based on the data of the last month" in the human-computer interaction interface. After understanding the requirements, the big model returns "whether to analyze abnormal equipment through the production data of the equipment in the last month". If the model understands it incorrectly, it will add "after modeling based on historical data, perform real-time anomaly detection" until the model understands it correctly.

[0074] Further, the requirements can be classified into:

[0075] Establish the first type of demand and the second type of demand matching verbs;

[0076] The similarity between the verb in the requirement and the matching verb is calculated according to the following formula:

[0077] ,

[0078] in, is the demand type, The first word vectors; is the first elements; is the word vector length; For the first type of matching verb set word vectors; For the second type of matching verb set word vectors;

[0079] The requirements are divided into first type requirements and second type requirements according to the similarity of the matching verbs.

[0080] Exemplarily, the matching verbs for the first type of requirements are: {"analyze", "display", "calculate", "statistic"}; the matching verbs for the second type of requirements are: {"predict", "identify", "detect", "classify", "cluster"}.

[0081] The demand verb set obtained by part-of-speech tagging for “analyzing the production of all varieties in a workshop last week” is {“analysis”}, and the demand noun set is {“variety”, “production”}; the second type of demand “analyzing the production of all varieties in a workshop last week” is {“detection”}, and the demand noun set is {“abnormality”, “equipment”}, among which the concept of the demand noun set is the key non-verb.

[0082] The similarity between the required verb set and the verbs in the two verb matching sets is calculated as follows:

[0083] ,

[0084] For example, the verb similarity between the demand verb set of "analyzing the production of all varieties in a workshop last week" and the two verb matching sets is:

[0085] ,

[0086] The similarity between the demand verb set of "detect abnormal equipment based on the data of the last month" and the verbs in the two verb matching sets is:

[0087] ,

[0088] If it is a first type of demand, searching the corresponding data of the historical working conditions in the private database according to the verbs and nouns in the demand to obtain the corresponding results of the first type of demand;

[0089] For example, when “analyzing the production of all varieties in a workshop last week”, the corresponding data of historical working conditions in the private database is retrieved according to {“varieties”, “production”}.

[0090] If it is a second type of demand, then searching the corresponding data of historical working conditions in the private database according to the verbs and nouns in the second type of demand and combining with the knowledge base of the target industry to obtain the corresponding results of the second type of demand;

[0091] If the second type of demand is "detect abnormal equipment based on data from the last month", the historical working conditions in the private database are retrieved based on {"broken ends per thousand spindle hours", "average spindle speed", "temperature", "humidity", "energy consumption", "output", "front roller speed", "spindle speed"} and data is searched in the spinning field knowledge base.

[0092] Finally, the corresponding data is displayed on the human-computer interaction interface.

[0093] Table display: Displaying data in a table can show detailed items and related indicators. For example, display the yield, date and other related information of each variety.

[0094] Graphical display: Use charts (such as bar charts, line charts, pie charts, etc.) to intuitively display data trends and distribution. For example, display the device operation data of the past month and use charts to show the frequency of abnormal devices.

[0095] Data summary: Provides a brief summary of the data and analysis results. For example, for detected abnormal devices, it can provide a detailed description of the abnormality, the time of occurrence, and the impact.

[0096] The embodiments of the present disclosure provide an industrial data mining and interaction method, electronic device, and storage medium. By pre-training the data of the target industrial field based on a multimodal large model, and combining the human-computer interaction interface to understand the needs and classify the data, the corresponding data is extracted from the private database and displayed on the interactive interface, efficient mining and analysis of complex industrial data is achieved, and the accuracy of data processing and the level of intelligent interaction are improved.

[0097] The electronic device 400 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 400 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 400 and does not constitute a limitation of the electronic device 400. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0098] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0099] The memory 402 may be an internal storage unit of the electronic device 400, for example, a hard disk or memory of the electronic device 400. The memory 402 may also be an external storage device of the electronic device 400, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 400. Further, the memory 402 may also include both an internal storage unit of the electronic device 400 and an external storage device. The memory 402 is used to store the computer program 403 and other programs and data required by the electronic device. The memory 402 may also be used to temporarily store data that has been output or is to be output.

[0100] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present disclosure. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0101] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0103] In the embodiments provided in the present disclosure, it should be understood that the disclosed / electronic devices and methods can be implemented in other ways. For example, the / electronic device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, multiple units or components may be combined or may be integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection of units, which may be electrical, mechanical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or recording medium that can carry computer program code, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0107] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. An industrial data mining and interaction method, characterized in that: include: Building a knowledge base of the target industry based on the data of the target industry; Pre-training the multimodal large model based on the data of the knowledge base includes: Use the following formula to construct a multimodal large model training batch: ， in, is a collection of similar files; For the similar files in the collection kind, ; For the A collection of similar files; The percentage threshold of the selected files among similar files; No. A collection of similar files; No. A collection of similar files, According to the multimodal large model training batch, all batches of training data are sequentially sent to the variant NExT-GPT multimodal large model for training to obtain an industrial multimodal large model; Sending requirements to the multimodal big model through a human-computer interaction interface, and classifying the requirements when the multimodal big model understands the requirements correctly, specifically including: Establish matching verbs for the first type of requirements and the second type of requirements, The similarity between the verb in the requirement and the matching verb is calculated according to the following formula: ， in, is the demand type, The first word vectors; is the first elements; is the word vector length; For the first type of matching verb set word vectors; For the second type of matching verb set word vectors, According to the similarity of the matching verbs, the requirements are divided into first type requirements and second type requirements; According to the classified requirements, corresponding data is obtained from the private database; The acquired corresponding data is displayed on the human-computer interaction interface.

2. The industrial data mining and interaction method according to claim 1, characterized in that: The constructing of the target industry knowledge base according to the target industry data includes: Obtain text, image, audio, and video data of the target industry, and name the data files according to the process, equipment type, and production index; Importing the file names of the data files into the same text to extract word frequency features, and clustering the word frequency features through a clustering algorithm to form a similar file set; According to the similar file set, the data files are classified and stored to form a knowledge base of the target industry.

3. The industrial data mining and interaction method according to claim 1, characterized in that: The method also includes sending the requirements to the multimodal large model through the human-computer interaction interface, receiving the multimodal large model's returned understanding, and classifying the requirements when the user confirms that the requirements are understood correctly.

4. The industrial data mining and interaction method according to claim 3, characterized in that: The method also includes prompting the user to supplement or explain the demand through the human-computer interaction interface when the user confirms that the demand is misunderstood, and allowing the multimodal large model to update the current round of dialogue state in combination with the previous round of dialogue state.

5. The industrial data mining and interaction method according to claim 1, characterized in that: The matching verbs for the first type of requirements include "analyze", "display", "calculate", and "statistics"; the matching verbs for the second type of requirements include "predict", "identify", "detect", "classify", and "cluster".

6. The industrial data mining and interaction method according to claim 1, characterized in that: The obtaining of corresponding data from the private database according to the classified requirements includes: According to the verbs and nouns in the first type of demand, corresponding data of historical working conditions in the private database are retrieved to obtain the corresponding results of the first type of demand; According to the verbs and nouns in the second type of demand, the corresponding data of the historical working conditions in the private database are retrieved and combined with the knowledge base of the target industry to obtain the corresponding results of the second type of demand.

7. An electronic device, characterized in that: include: one or more processors; A storage unit, used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the industrial data mining and interaction method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the industrial data mining and interaction method according to any one of claims 1 to 6 can be implemented.

Images