Mine AI visual training system design and implementation method based on large language model

Abstract

The invention provides a mine AI visual training system design and implementation method based on a large language model. The method comprises the steps that user demand information is acquired; processing the user demand information based on a large language model in a mine AI visual training system to obtain a system control instruction; a mine AI visual training system constructs a target model based on a system control instruction, and the mine AI visual training system comprises a data acquisition center, an AI data management center, an AI development environment, an AI model training center, an AI model deployment center, an AI algorithm management center, an AI configuration management center and an AI question and answer center. By implementing the method disclosed by the invention, the large language model can be introduced to realize the function intelligence of the system, simplify the operation process and ensure that multiple functions can be automatically and intelligently generated, and the large language model can be continuously improved through training, so that the functions of the system gradually tend to an ideal system in the heart of a user.

Description

Technical Field

[0001] The present disclosure relates to the field of smart mine technology, and in particular to a design and implementation method of a mine AI vision training system based on a large language model. Background Art

[0002] The existing coal mine video training system is generally implemented using traditional software development methods, with coders coding each functional module one by one. If the system is a CS architecture, C++ is used to develop backend services, and QT is used to develop visual interactive interfaces. If the system is a BS architecture, a single technology stack solution or a separated front-end and back-end technology stack solution is generally used, depending on the development time of the system, and the coding language is generally java, vue, html, etc.

[0003] As mentioned above, although the implementation schemes and development languages ​​used are different, all functions need to be customized and developed, and coders need to implement them line by line. When there are changes in user requirements, functional modifications, etc., coders will need to code, internally verify, iterate development, unit test, integration test, external release, on-site deployment, and customer functional verification. The entire process is implemented, which has a long cycle and extremely low customer participation, resulting in the system functions and customer expectations often not matching by 100%, and customers are dissatisfied with the functions and return for modification. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the purpose of the present invention is to propose a design and implementation method of a mining AI vision training system based on a large language model, which can introduce a large language model to realize the intelligentization of system functions, simplify the operation process, and ensure that many functions can be automatically and intelligently generated. The large language model can be continuously improved through training, so that the functions of the system gradually tend to the ideal system in the minds of users.

[0006] To achieve the above objectives, the first aspect of the present disclosure proposes a design and implementation method of a mine AI visual training system based on a large language model, including:

[0007] Obtain user demand information;

[0008] Process the user demand information based on the large language model in the mine AI vision training system to obtain system control instructions;

[0009] The mine AI vision training system constructs a target model based on the system control instructions, wherein the mine AI vision training system includes: a data acquisition center, an AI data management center, an AI development environment, an AI model training center, an AI model deployment center, an AI algorithm management center, an AI configuration management center, and an AI question and answer center.

[0010] The design and implementation method of the mine AI vision training system based on the large language model provided by the present disclosure obtains user demand information; processes user demand information based on the large language model in the mine AI vision training system to obtain system control instructions; the mine AI vision training system builds a target model based on the system control instructions, wherein the mine AI vision training system includes: data collection center, AI data management center, AI development environment, AI model training center, AI model deployment center, AI algorithm management center, AI configuration management center, AI question and answer center. Thus, the large language model can be introduced to realize the intelligentization of system functions, simplify the operation process, ensure that many functions can be automatically and intelligently generated, and the large language model can be continuously improved through training, so that the functions of the system gradually tend to the ideal system in the minds of users.

[0011] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1 It is a flowchart of a design and implementation method of a mine AI visual training system based on a large language model proposed in an embodiment of the present disclosure;

[0014] Figure 2 It is a schematic diagram of the framework of the mine AI vision training system proposed in the present disclosure;

[0015] Figure 3 It is a schematic diagram of the process of creating a helmet wearing detection model proposed in the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and are not to be construed as limitations of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications, and equivalents that fall within the spirit and connotation of the appended claims.

[0017] Figure 1 It is a flowchart of the design and implementation method of a mine AI visual training system based on a large language model proposed in an embodiment of the present disclosure.

[0018] Among them, it should be noted that the executor of the method for designing and implementing a mining AI visual training system based on a large language model in this embodiment is a device for designing and implementing a mining AI visual training system based on a large language model. The device can be implemented by software and / or hardware. The device can be configured in a computer device. The computer device may include but is not limited to a terminal, a server, etc. For example, the terminal may be a mobile phone, a handheld computer, etc.

[0019] like Figure 1 As shown, the design and implementation method of the mine AI visual training system based on the large language model includes:

[0020] S101: Obtain user demand information.

[0021] S102: Process the user demand information based on the large language model in the mine AI vision training system to obtain system control instructions.

[0022] S103: The mine AI vision training system builds a target model based on the system control instructions, wherein the mine AI vision training system includes: a data acquisition center, an AI data management center, an AI development environment, an AI model training center, an AI model deployment center, an AI algorithm management center, an AI configuration management center, and an AI question and answer center.

[0023] In this embodiment, by obtaining user demand information; processing user demand information based on the large language model in the mine AI vision training system to obtain system control instructions; the mine AI vision training system builds a target model based on the system control instructions, wherein the mine AI vision training system includes: data acquisition center, AI data management center, AI development environment, AI model training center, AI model deployment center, AI algorithm management center, AI configuration management center, AI question and answer center. Thus, the large language model can be introduced to realize the intelligentization of system functions, simplify the operation process, ensure that many functions can be automatically and intelligently generated, and the large language model can be continuously improved through training, so that the functions of the system gradually tend to the ideal system in the minds of users.

[0024] Optionally, in some embodiments, the functions of the data collection center include at least:

[0025] Support access and management services for multimodal data;

[0026] Support general management operations of data sets;

[0027] Support local video and audio import;

[0028] The introduction of a large language model operation portal supports the system to automatically execute relevant data management commands through input instructions or voice, thereby realizing multimodal data collection and management.

[0029] Optionally, in some embodiments, the functions of the AI ​​data management center include at least:

[0030] Supports annotation task management, data management, multi-person annotation, third-party annotation, and intelligent annotation;

[0031] Support multiple annotation types when annotating;

[0032] Support enhanced processing of data;

[0033] The large language model operation entry is introduced to support the system to automatically execute relevant labeling instructions through input commands or voice, thereby realizing batch labeling of N data.

[0034] Optionally, in some embodiments, the functions of the AI ​​development environment center include at least:

[0035] Provides online development, remote development plug-ins, remote development SDK, large language model management, and automatic debugging functions;

[0036] The large language model operation entry is introduced to support the system to automatically execute relevant instructions through input commands or voice.

[0037] Optionally, in some embodiments, the functions of the AI ​​model training center include at least:

[0038] Provide scene construction and model training related functions;

[0039] It provides one-stop intelligent model training function, based on the large language model operation portal, realizes the full process training of the model according to user commands, and provides intelligent analysis reports of multiple model training results.

[0040] Optionally, in some embodiments, the functions of the AI ​​model deployment center include at least:

[0041] Provides an image center, supports image management functions and custom images;

[0042] Provide deployment configuration and support cloud, edge, and end deployment;

[0043] Provide deployment simulation and support pre-deployment operations;

[0044] Based on the large language model operation portal, the deployment plan is automatically generated according to user commands and pre-deployment operations are performed.

[0045] Optionally, in some embodiments, the functions of the AI ​​algorithm management center include at least:

[0046] Provide algorithm management;

[0047] Supports algorithm classification and service binding functions;

[0048] It supports operating the portal through a large language model and automating the entire algorithm management process based on user commands.

[0049] Optionally, in some embodiments, the functions of the AI ​​question answering center include at least:

[0050] Support 24-hour expert online Q&A;

[0051] Supports network connection to obtain the latest cutting-edge technologies and problem-solving methods through the network.

[0052] Optionally, in some embodiments, the implementation architecture adopted by the mine AI vision training system based on some functions of the large language model is the Transformr model, which consists of an encoder Encoder and a decoder Decoder, and processes sequence data through a self-attention mechanism.

[0053] Optionally, in some embodiments, the large language model supports a third-party platform using model training results or system-related algorithms through an API interface.

[0054] Based on the above embodiments, the mining AI vision training system proposed in the present disclosure integrates "data + algorithm + computing power + platform", adopts a unified "cloud-edge-end" three-level architecture system, and builds an innovative industrial Internet platform.

[0055] The mining AI visual training system and method based on the large language model is implemented by relying on cloud-edge collaboration, microservices, and separation of front-end and back-end technology stack solutions. The system implementation architecture is as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of the framework of the mine AI visual training system proposed in this disclosure. It mainly includes: data collection, AI data management center, AI development environment, AI model training center, AI model deployment center, AI algorithm management center, AI configuration management center, AI problem center, etc.

[0056] The main technical route of the system: the front-end uses VUE+HTML+CSS+TypeScript to implement visual interface development; the back-end uses Srping Cloud Alibaba+Java to implement back-end interface development; the database is implemented using Mysql, etc.

[0057] The functions implemented include:

[0058] Data collection: Supports access and management services for multimodal data such as video, audio, and point cloud, and supports routine management operations such as adding, filtering, modifying, and deleting data sets. It also supports local video and audio import. The large language model operation portal is introduced to support the system to automatically execute relevant data management commands through input instructions or voice, realizing multimodal data collection and management. Engineers only need to make minor adjustments, corrections, and personalized modifications.

[0059] AI Data Management Center: The visualization system supports annotation task management, data management, multi-person annotation, third-party annotation, and intelligent annotation. When annotating, it supports multiple annotation types such as target detection. At the same time, according to the business scenario needs of the coal mine site, it supports enhanced processing of video and other data, such as defogging during coal mining. In intelligent annotation, a large language model operation portal is introduced to support the system to automatically execute related annotation instructions through input commands or voice, and realize batch annotation of N data.

[0060] AI Development Environment Center: Provides online development, remote development plug-ins, remote development SDK, large language model management, automatic debugging and other functions. Introduces a large language model operation portal, supports the system to automatically execute related instructions through input commands or voice, such as automatically developing an SDK with a certain function based on input and output, or automatically developing related configuration pages based on existing components according to descriptions, etc., to achieve unmanned and intelligent process operations.

[0061] AI Model Training Center: Provides scene construction and model training related functions, such as model management, model fine-tuning, model distributed training, model performance evaluation, model optimization, model quality inspection, etc. It also provides a one-stop intelligent model training function, based on the large language model operation portal, according to user commands, to achieve full-process model training, and provide intelligent analysis reports of multiple model training results, etc., shortening the AI ​​model training and online cycle, reducing the technical threshold requirements for model training operators, and achieving the goal of customers being able to get started.

[0062] AI Model Deployment Center: Provides an image center that supports image management and custom images; provides deployment configuration that supports cloud, edge, and end deployment; provides deployment simulation that supports pre-deployment operations, etc. Through the large language model operation portal, according to user commands, it can realize automatic generation of deployment plans and pre-deployment operations.

[0063] AI Algorithm Management Center: Provides algorithm management, such as adding, deleting, modifying, and checking operations; and supports functions such as algorithm classification and binding with services. It supports the automation of the entire algorithm management process through the large language model operation portal according to user commands.

[0064] AI Q&A Center: Supports 24-hour expert online questions, solves problems related to coal mine model training, model deployment, algorithms, system configuration, etc., can solve any problems in coal mine visual training, and can quickly find answers and methods to solve problems. The Q&A Center can also connect to the Internet to obtain the latest cutting-edge technologies and solutions to problems through the Internet to meet the business needs of users' continuous recommendations.

[0065] The system introduces a large language model to assist in realizing the above-mentioned related functions. The implementation architecture adopted for some functions based on the large language model is the Transformr model, which is mainly composed of an encoder and a decoder. It processes sequence data through a self-attention mechanism, which can effectively capture long-distance dependencies within the sequence, and is used to implement natural language processing (NLP) tasks and speech recognition. In order to better serve customers, multimodal processing functions are implemented, which can simultaneously process different types of data such as text and voice, perform related operations, generate related records, etc. as needed. It makes it more flexible and efficient when performing complex tasks, reduces the basic requirements of professional industrial software for users, and lowers the threshold for users. The encoder is used to understand and extract the semantic information of the input text, while the decoder is responsible for generating responses or translation outputs. This structure allows the model to learn end-to-end from input to output, thereby better capturing contextual relationships. In addition, it also includes multi-head self-attention, where each head can independently focus on different aspects of information, thereby enhancing the model's understanding and generalization capabilities. Position encoding, because the Transformer does not have built-in position perception capabilities, position encoding needs to be added to provide the position information of words in the sequence. This allows the model to remain sensitive to order even in parallel computing. A fully connected feedforward network, located after the attention module of each layer, is used to further process features and increase the expressiveness of the model. Large-scale pre-training, using a large amount of unlabeled data sets for pre-training to learn common language representations. Then fine-tuning for specific tasks can significantly improve the performance of the model on downstream tasks. Distributed training, in order to speed up the training process, is usually executed in parallel on multiple GPUs or TPUs, using data parallel or model parallel strategies to shorten training time. Continuous updates and improvements, the model is regularly trained on new data, as well as advances in algorithms and technologies to continuously improve accuracy and response quality.

[0066] The introduced large language model can be used by third-party platforms through API interfaces to use model training results or related algorithms of the system. It can also be trained on the model training platform, that is, the corresponding entrance can be found in the training of the AI ​​development environment center. The method generally adopts supervised learning and reinforcement learning. Supervised learning can effectively use a large amount of labeled data to quickly improve model performance, while reinforcement learning can optimize the decision-making process through continuous trial and error in complex environments. According to the system performance requirements, we use a combination of the two methods to achieve the best learning effect.

[0067] In addition to the large language model, the mine AI visual training system based on the large language model is also designed to be embedded in other systems and platforms, making it highly flexible and portable, so as to adapt to the needs of different coal mine sites, different underground and above-ground application scenarios. Integration with other software or services is achieved through the API interface. If the API interface is insufficient, the API generation function of the large language model (the AI ​​development environment center provides relevant operation entrances) can be used to inform the input and output content, and automatically generate API interfaces, plug-ins, etc.

[0068] If customers think that the big language model that comes with the mining AI visual training system based on the big language model does not meet their needs, they can switch to the popular big language model on the market through the big language model switching function (the relevant visual operation entrance is provided in the AI ​​development environment center). This function supports expansion, and customers can introduce and test it according to their needs.

[0069] Data transmission relationship:

[0070] Coal mine on-site terminal equipment such as visible light cameras, lidars, and microphones provide data sources or locally import relevant videos, etc.; then access through the edge layer and transmit to the collection service on the Paas platform through 5G, which collects and stores relevant data, and then the data layer performs relevant cache and processing operations on the data. The large language model, through the large language model server proxy layer, implements operations such as data stream push, request and response processing, and provides services to the upper platform layer. After the data and large models are ready, the platform layer can implement operations such as model management, model training, model quality inspection, and model deployment. Users click to view relevant model training results or view related videos through a third-party platform or management and control platform.

[0071] When the user enters relevant instructions at the platform layer, the relevant operations are sent from top to bottom, from the platform layer to the corresponding large language model service layer, platform service layer, basic layer, etc., and finally reach the device end.

[0072] User operation process:

[0073] (1) Users can manually click on each step to achieve: data collection / local upload -> data management -> data labeling -> data processing -> model training -> model quality inspection and evaluation -> model upload -> application scenario application evaluation -> pre-simulation deployment -> formal model deployment.

[0074] (2) The user enters the command through the large language model operation portal, and intelligently implements the above-mentioned related operation steps. Figure 3 As shown, Figure 3 This is a schematic diagram of the process of creating a helmet wearing detection model proposed in the present disclosure, and other functions are similar.

[0075] Based on the above embodiments, the technical points of the technical solution proposed in this disclosure mainly include:

[0076] 1. The existing coal mine vision training system requires professional developers to develop and algorithm engineers to complete model training, and the training process is cumbersome. The introduction of a large language model to realize the mine AI vision training system solves the problem of complex operation process and low system flexibility of the coal mine AI vision training system.

[0077] 2. The existing coal mine vision training system adopts traditional development methods and development languages, which require manual coding one by one. The mine AI vision training system and method based on the large language model realizes the integration of large models and traditional systems, and can realize automatic coding, automatic SDK development, and automatic training, which improves the level of coal mine intelligence and promotes the intelligent development of coal mines.

[0078] 3. The mine AI visual training system and method based on the large language model solves the problems of unreasonable software architecture design, lack of intelligence, unusable and difficult to use, and low customer participation in coal mines. It achieves the purpose of customer customization, self-training and self-deployment. It liberates developers, engineers and operation and maintenance personnel, saves coal mine labor costs, and shortens the development and deployment cycle and model training cycle.

[0079] The technical effects of the technical solution proposed in this disclosure mainly include:

[0080] (1) The present invention realizes the mine AI vision training system and method by adopting a large language model, thereby realizing the intelligence of the system. At the same time, compared with existing products that adopt third-party large models, the present invention also realizes the autonomous control of the large language model technology. The large language model can be trained and adjusted independently according to the on-site business scenarios and business needs of the coal mine, without being restricted by the technology of a third party. Moreover, there will be no related cost issues between self-developed technology and self-developed products, thus saving costs.

[0081] (2) The large language model implementation architecture of the present invention is implemented through the Transformr model. Compared with other architectures, it has significant advantages in parallel computing capability, scalability, expression capability, and generalization capability. Therefore, the system of the present invention will have significant advantages in functions such as intelligent question answering and intelligent training.

[0082] (3) The present invention solves the problems of the existing coal mine visual training system, such as difficulty in getting started, high technical threshold, the need for professional personnel to complete the training, and low customer participation, thereby realizing an intelligent and easy-to-use system.

[0083] (4) The present invention enables non-algorithm personnel to realize functions such as automatic model training, automatic deployment, and automatic development interface through text and language description. The relevant functional requirements can be completed without coding by coders. The development and deployment cycle is greatly shortened, and the manpower and material costs are saved.

[0084] (5) The present invention realizes the AI ​​question-and-answer function, namely the 24-hour online function of "experts", which provides guarantee and strong technical support for the 24-hour operation of coal mines.

[0085] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0086] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0087] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0088] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0089] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0090] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0091] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0092] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0093] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0094] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A design and implementation method of a mine AI visual training system based on a large language model, characterized in that: include: Obtain information about user needs; Process the user demand information based on the large language model in the mine AI vision training system to obtain system control instructions; The mine AI vision training system constructs a target model based on the system control instructions, wherein the mine AI vision training system includes: a data acquisition center, an AI data management center, an AI development environment, an AI model training center, an AI model deployment center, an AI algorithm management center, an AI configuration management center, and an AI question and answer center.

2. The method according to claim 1, characterized in that The functions of the data collection center include at least: Support access and management services for multimodal data; Support general management operations of data sets; Support local video and audio import; The introduction of a large language model operation portal supports the system to automatically execute relevant data management commands through input instructions or voice, thereby realizing multimodal data collection and management.

3. The method according to claim 1, characterized in that The functions of the AI ​​data management center include at least: Supports annotation task management, data management, multi-person annotation, third-party annotation, and intelligent annotation; Support multiple annotation types when annotating; Support enhanced processing of data; The large language model operation entry is introduced to support the system to automatically execute relevant labeling instructions through input commands or voice, thereby realizing batch labeling of N data.

4. The method according to claim 1, characterized in that The functions of the AI ​​development environment center include at least: Provides online development, remote development plug-ins, remote development SDK, large language model management, and automatic debugging functions; The large language model operation entry is introduced to support the system to automatically execute relevant instructions through input commands or voice.

5. The method according to claim 1, characterized in that The functions of the AI ​​model training center include at least: Provide scene construction and model training related functions; It provides one-stop intelligent model training function, based on the large language model operation portal, realizes the full process training of the model according to user commands, and provides intelligent analysis reports of multiple model training results.

6. The method according to claim 1, characterized in that The functions of the AI ​​model deployment center include at least: Provides an image center, supports image management functions and custom images; Provide deployment configuration and support cloud, edge, and end deployment; Provide deployment simulation and support pre-deployment operations; Based on the large language model operation portal, the deployment plan is automatically generated according to user commands and pre-deployment operations are performed.

7. The method according to claim 1, characterized in that The functions of the AI ​​algorithm management center include at least: Provide algorithm management; Supports algorithm classification and service binding functions; It supports operating the portal through a large language model and automating the entire algorithm management process based on user commands.

8. The method according to claim 1, characterized in that The functions of the AI ​​Q&A center include at least: Support 24-hour expert online Q&A; Supports network connection to obtain the latest cutting-edge technologies and problem-solving methods through the network.

9. The method according to claim 1, characterized in that The implementation architecture of some functions based on the large language model in the mine AI vision training system is the Transformr model, which consists of an encoder and a decoder, and processes sequence data through a self-attention mechanism.

10. The method according to claim 1, characterized in that The large language model supports the use of model training results or system-related algorithms by a third-party platform through an API interface.

Images