Intelligent assistant system based on port major large model

Abstract

The invention relates to the technical field of computer software systems, and particularly discloses an intelligent assistant system based on a large port professional model, which comprises an online data acquisition module, an offline database, a data processing module, a data code, a large port model, a model fine tuning pipeline, an intelligent assistant service and an intelligent assistant user side. The beneficial effects are that the intelligent assistant service is constructed based on the port large model, and the intelligent assistant service can optimize and finely adjust the model according to the data collected by the online data collection module, assists a user to carry out safety training and information retrieval services, facilitates the customization of training contents according to the actual condition of the port, and improves the training efficiency. And the model is continuously optimized by continuously increasing the stored data, so that port safety training and retrieval information are more comprehensive and diversified, and meanwhile, the accuracy of a prediction result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer software systems, and in particular to an intelligent assistant system based on a large port professional model. Background Art

[0003] In order to improve the operational efficiency and safety of ports, many ports have begun to introduce advanced technologies, such as the Internet of Things (IoT), big data analysis, and artificial intelligence (AI). The application of these technologies has enabled ports to achieve great progress in data collection, processing, and decision-making. For example, IoT-based sensor networks can monitor port equipment and environmental conditions in real time, and big data analysis technology can deeply mine and analyze massive data to optimize port operations.

[0004] In terms of artificial intelligence technology, especially the development of deep learning and natural language processing technology, it is possible to build intelligent assistants. The current port management system has the following defects:

[0005] 1. Most of the port safety training systems are computer-based and provide training through videos, graphics and text. These systems usually lack interactivity and personalization, and the training effect is limited. Traditional safety training systems mainly rely on preset training content and cannot be adjusted according to the actual situation of trainees. The interactivity is poor; and the training content is single and limited by the update of the training database, trainees cannot receive targeted training according to their own shortcomings.

[0006] 2. Although some ports have deployed equipment management systems to monitor equipment status in real time through IoT technology, these systems usually operate independently and lack integration with other port management systems. They have a slow response speed when handling complex tasks, especially when responding to emergencies, and are unable to provide effective solutions in a timely manner. Summary of the invention

[0007] The present invention proposes an intelligent assistant system based on a large port professional model to solve the above problems, so as to improve the efficiency of port management, enhance the effect of safety training, optimize the information retrieval function, and thus realize the intelligent and efficient port management.

[0008] The technical solution of the present invention is achieved in this way:

[0009] The intelligent assistant system based on the port professional large model includes online data collection module, offline database, data processing module, data encoding, port large model, model fine-tuning pipeline, intelligent assistant service, and intelligent assistant user end.

[0010] The online data collection module is used to collect data from various online sources in real time, including obtaining autonomous driving vehicles and port machinery dispatching control instructions from the cluster dispatching system, business data, port machinery status, and autonomous driving vehicle status from the real-time business database, and obtaining port intersection traffic status, safety event records, and port road condition fusion target perception data from the intelligent roadside fusion perception system to ensure the real-time and accuracy of the data;

[0011] The offline database is used to store port big data, including port equipment operation manuals, job training question banks, event safety warning videos in the safety training database, and ships, cargo, facilities and equipment as well as economic and management records in the data retrieval database. It also stores safety event records and corresponding video clips detected by the online data collection module.

[0012] The data processing module is used to clean, convert and integrate the collected data, process missing data, duplicate data, abnormal data, and standardize the data format to ensure data quality and consistency, and provide a reliable data foundation for subsequent analysis and modeling;

[0013] Data encoding is used to convert processed data into a format that the model can understand and use, including feature extraction, data annotation and encoding, to ensure that data can be efficiently input into the model for training and reasoning; the port large model is a virtual port training model with language understanding and generation capabilities. Through training and fine-tuning, the model can handle specific port tasks, such as ship scheduling, cargo management, equipment maintenance prediction, information retrieval, safety training, etc.;

[0014] The model fine-tuning pipeline is used to fine-tune the port model on specific task data to improve the performance of the model in specific application scenarios, including model training, verification, evaluation and optimization processes to ensure that the model has high accuracy and high performance;

[0015] The intelligent assistant service communicates with the security training system, data retrieval system, and equipment management system, and can call the system's interface to obtain results. The intelligent assistant service is used to deploy the fine-tuned model as an online service, process users' real-time requests, and provide intelligent question-and-answer, information retrieval, and security training functions;

[0016] The smart assistant user end is used for users to interact with the smart assistant, display analysis results and decision-making suggestions, and provide corresponding teaching and accident warning video links for safety training.

[0017] Furthermore, the port model stores an environmental scenario library that simulates different port scenarios, a business scenario library that simulates different business processes, and an event-driven engine that simulates different safety accident processes.

[0018] Furthermore, the services selected by users through the intelligent assistant user end include intelligent question and answer, information retrieval, and security training.

[0019] Furthermore, the online data collection module obtains data including accurate time identification from the cluster scheduling system, and the scheduled data is simulated on the virtual business simulator to obtain simulation results of the corresponding data.

[0020] Furthermore, the data collected by the online data collection module are continuously stored in the offline database after being processed by the data processing module, and the database is automatically updated to facilitate continuous fine-tuning and optimization of the port model, making port safety training and retrieval information more comprehensive and diverse.

[0021] Furthermore, the model fine-tuning pipeline compares the obtained data simulation results with the obtained real results to continuously optimize the model capabilities.

[0022] Furthermore, the content input by the intelligent assistant user terminal includes one or more of text, voice, and pictures, and the content output by the intelligent assistant user terminal includes one or more of text, voice, pictures, and videos.

[0023] Furthermore, when a security incident occurs when simulating business data through a virtual business simulator, the intelligent assistant service can automatically push the event results simulated by the virtual business simulator to the intelligent assistant user terminal.

[0024] By adopting the above technical scheme, the beneficial effects of the present invention are as follows: an intelligent assistant service is constructed based on a large port model. The intelligent assistant service can optimize and fine-tune the model according to the data collected by the online data collection module, assist users in safety training and information retrieval services, and facilitate the customization of training content according to the actual situation of the port. By continuously increasing the amount of stored data and continuously optimizing the model, port safety training and retrieval information are made more comprehensive and diverse, while improving the accuracy of prediction results; by continuously learning and continuously improving the model, the results can be automatically predicted through simulation after obtaining business data and control instructions, and timely prompts can be given when safety risks are found in the simulation, so as to facilitate timely changes in scheduling strategies and improve the safety of port transportation; multiple systems can share data to achieve information sharing and interoperability. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 is a system block diagram of the present invention;

[0027] Figure 2 It is a collection flow chart of the online data collection module of the present invention;

[0028] Figure 3 It is the offline database associated data content graph of the present invention;

[0029] Figure 4 It is a flow chart of data processing by the data processing module of the present invention;

[0030] Figure 5 is a diagram of the system deployment mode of the present invention;

[0031] Figure 6 It is a flow chart of constructing the intelligent assistant service of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] like Figure 1 As shown in the figure, the intelligent assistant system based on the port professional large model includes an online data acquisition module, an offline database, a data processing module, data encoding, a port large model, a model fine-tuning pipeline, an intelligent assistant service, and an intelligent assistant user terminal.

[0034] The online data collection process is as follows Figure 2 ,The online data collection module is used to collect data in real time from various ,online sources, including obtaining autonomous vehicles and port machinery ,dispatching control instructions from the cluster dispatching system, business data, port machinery status, and autonomous ,vehicle status from the real-time business library, and obtaining the traffic status of the ,port intersection section, safety event records and target perception data ,fused with port road conditions from the intelligent road-side fusion perception system, to ensure the ,realtime and accuracy of the data, and the dispatched data is simulated on the ,virtual business simulator to obtain the simulation results of the ,corresponding data;

[0035] The cluster scheduling system provides a cluster scheduling algorithm for the virtual business simulator. The virtual business simulator builds a virtual port corresponding to the actual port, and provides the cluster scheduling system with vehicle transportation routes, transportation speed control indicators, and port machinery operation control indicators. By loading different types of real-time businesses into the virtual business simulator, the output results of cluster scheduling control under different businesses are obtained, thereby improving the speed of data set acquisition. The data should be accompanied by an accurate time mark of the occurrence moment to ensure data time alignment.

[0036] like Figure 3 As shown, the offline database is used to store port big data, including the port equipment operation manual, job training question bank, event safety warning video in the safety training database, and the ships, cargoes, facilities and equipment as well as economic and management records in the data retrieval database, and also stores the safety event records and corresponding video clips detected by the online data acquisition module;

[0037] like Figure 4 As shown, the data processing module is used to clean, transform and integrate the collected data, process missing data, duplicate data, abnormal data, and standardize the data format to ensure data quality and consistency, and provide a reliable data foundation for subsequent analysis and modeling;

[0038] Data encoding is used to convert processed data into a format that the model can understand and use, including feature extraction, data labeling and encoding, to ensure that data can be efficiently input into the model for training and reasoning;

[0039] The port big model is a virtual training model for the port, which has the ability to understand and generate language. Through training and fine-tuning, the model can handle specific port tasks, such as ship scheduling, cargo management, equipment maintenance prediction, information retrieval, safety training, etc.

[0040] The model fine-tuning pipeline is used to fine-tune the port model on specific task data to improve the performance of the model in specific application scenarios, including model training, verification, evaluation and optimization processes to ensure that the model has high accuracy and high performance;

[0041] The intelligent assistant service communicates with the security training system, data retrieval system, and equipment management system, and can call the system's interface to obtain results. The intelligent assistant service is used to deploy the fine-tuned model as an online service, process users' real-time requests, and provide intelligent question-and-answer, information retrieval, and security training functions;

[0042] The smart assistant user terminal is used for users to interact with the smart assistant, display analysis results and decision-making recommendations, and provide corresponding teaching and accident warning video links for safety training. The content input by the smart assistant user terminal includes one or more of text, voice, and pictures, and the content output by the smart assistant user terminal includes one or more of text, voice, pictures, and videos.

[0043] In this embodiment, the port large model stores an environmental scenario library that simulates different port scenarios, a business scenario library that simulates different business processes, and an event-driven engine that simulates different safety accident processes.

[0044] In this embodiment, the services selected by the user through the intelligent assistant user terminal include intelligent question and answer, information retrieval, and safety training.

[0045] In this embodiment, the model fine-tuning pipeline compares the obtained data simulation results with the obtained real results to continuously optimize the model capabilities.

[0046] In this embodiment, the data collected by the online data collection module is continuously stored in the offline database after being processed by the data processing module, and the database is automatically updated to facilitate continuous fine-tuning and optimization of the port model, making port safety training and retrieval information more comprehensive and diverse.

[0047] In this embodiment, when a security incident occurs when simulating business data through a virtual business simulator, the intelligent assistant service can automatically push the event results simulated by the virtual business simulator to the intelligent assistant user terminal.

[0048] like Figure 5 As shown, the intelligent assistant service of the present invention is divided into three sections for deployment. The data section (data processing module, offline database, online data acquisition module) is deployed in the same network as the cluster dispatching system and the intelligent roadside fusion perception system, so that users can obtain real-time business and perception data. The model fine-tuning pipeline can be deployed separately in an independent training server. The intelligent assistant service is deployed in the same network as the port safety training system, data retrieval system, and equipment management system. They can communicate with each other and can directly call the system interface to obtain results.

[0049] like Figure 6 As shown, the workflow of the smart assistant service is as follows:

[0050] (1) Build the database required for intelligent assistant training in the offline database

[0051] 1.1 Safety training database, including port equipment operation manuals, job training question banks, safety incident warning videos, and all content materials of port training;

[0052] 1.2 Data retrieval database, which collects information that port personnel need to query on a daily basis, such as ships, cargo, facilities and equipment, and economic management information;

[0053] (2) Real-time data collection and processing

[0054] 2.1 The online data acquisition module receives the perception data of the intelligent roadside fusion perception system;

[0055] 2.2 The online data collection module receives the business data and scheduling instructions of the cluster scheduling system;

[0056] Among them, the perception data includes safety event records, traffic status of intersections and sections, and bird's-eye view of the port. If the safety event information is not empty, the corresponding accident video clip is captured; business data and dispatch instructions include autonomous driving and port machinery dispatch instructions, port machinery status, autonomous driving status, and business demand data;

[0057] 2.3 The data processing module aligns the data of the two systems;

[0058] 2.3.1 Processing of missing values, duplicate values, and outliers;

[0059] 2.3.2 If the security event information is not empty (a security incident is detected), obtain the corresponding security event video, the event description of the perception detection and the port bird's-eye view, the dispatch instruction issued by the cluster dispatch control system when the event occurs, and the business data description, form the security event background and event analysis description, match it with the event video clip and store it in the security training database;

[0060] 2.3.3 The data processing module classifies and divides the data based on business needs, control instructions, and traffic status, and then stores it in the offline database to achieve the purpose of data expansion, realize multi-angle description of data, and generalize the model learning ability;

[0061] (3) Data encoding

[0062] Convert the safety training database, data retrieval database and real-time collected port data into a format that the model can understand and use;

[0063] (4) Model fine-tuning and evaluation

[0064] Obtain model parameters from the port model, freeze the backbone network, and only train the output head corresponding to the downstream task; establish queries with three-party systems (safety training system, data retrieval system, and equipment management system) for model evaluation, compare the model output results with the results returned by calling the corresponding system query, and evaluate the reliability of the model output results. When the accuracy rate is ≥ the specified value, the model fine-tuning is completed;

[0065] (5) Smart Assistant Service Release Application

[0066] Deploy the model fine-tuned in step (4), provide a calling interface, return the model results, and input the desired content (select security training content, ask for processing opinions, information retrieval query) through the intelligent assistant user end to obtain the results (security training links, processing opinions and suggestions, retrieval results).

[0067] The intelligent assistant system based on the port professional large model of the present invention forms an intelligent database by continuously collecting data on port safety training and information retrieval, and continuously fine-tuning and learning. Users only need to simply describe their needs through the intelligent assistant user end to understand their needs and promote results, thus solving the problem of messy original information and difficult and time-consuming queries. The system focuses on professional fields such as port operations, logistics, cargo management, and equipment maintenance, and can provide in-depth industry knowledge and solutions. When dealing with specific port issues, it is more targeted and accurate than general large models. It can customize safety training content and provide intelligent retrieval services according to the specific needs of the port, for example, providing detailed guidelines for port safety operations, simulated training environments, risk assessment tools, etc.; through the online data acquisition module, it can obtain port business data and scheduling data in real time, and use the data of the port industry for real-time monitoring and prediction, helping managers to promptly discover potential problems and make adjustments, thereby improving the efficiency and safety of port operations; it automatically updates the database and continuously fine-tunes and optimizes the large port model, making port safety training and retrieval information more comprehensive and diverse; the port safety events obtained through the intelligent roadside fusion perception system continuously expand the content of the safety accident database, so as to achieve timely discovery and learning of problems, avoid the same mistakes from repeating in port transportation, and give decision-making suggestions and predictions in a timely manner through virtual simulation results, so as to facilitate the adjustment of scheduling strategies, reduce the occurrence of safety accidents, and realize intelligent management of port management.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. Intelligent assistant system based on port professional large model, characterized by: It includes online data collection module, offline database, data processing module, data encoding, port model, model fine-tuning pipeline, intelligent assistant service, and intelligent assistant user end. The online data collection module is used to collect data from various online sources in real time, including obtaining autonomous driving vehicles and port machinery dispatching control instructions from the cluster dispatching system, business data, port machinery status, and autonomous driving vehicle status from the real-time business database, and obtaining port intersection traffic status, safety event records, and port road condition fusion target perception data from the intelligent roadside fusion perception system to ensure the real-time and accuracy of the data; The offline database is used to store port big data, including port equipment operation manuals, job training question banks, event safety warning videos in the safety training database, and ships, cargo, facilities and equipment as well as economic and management records in the data retrieval database. It also stores safety event records and corresponding video clips detected by the online data collection module. The data processing module is used to clean, convert and integrate the collected data, process missing data, duplicate data, abnormal data, and standardize the data format to ensure data quality and consistency, and provide a reliable data foundation for subsequent analysis and modeling; Data encoding is used to convert processed data into a format that the model can understand and use, including feature extraction, data labeling and encoding, to ensure that data can be efficiently input into the model for training and reasoning; The port big model is a virtual training model for the port, which has the ability to understand and generate language. Through training and fine-tuning, the model can handle specific port tasks, such as ship scheduling, cargo management, equipment maintenance prediction, information retrieval, safety training, etc. The model fine-tuning pipeline is used to fine-tune the port model on specific task data to improve the performance of the model in specific application scenarios, including model training, verification, evaluation and optimization processes to ensure that the model has high accuracy and high performance; The intelligent assistant service communicates with the security training system, data retrieval system, and equipment management system, and can call the system's interface to obtain results. The intelligent assistant service is used to deploy the fine-tuned model as an online service, process users' real-time requests, and provide intelligent question-and-answer, information retrieval, and security training functions; The smart assistant user end is used for users to interact with the smart assistant, display analysis results and decision-making recommendations, and provide corresponding teaching and accident warning video links for safety training.

2. The intelligent assistant system based on the port professional large model according to claim 1 is characterized in that: The port big model stores an environmental scenario library simulating different port scenarios, a business scenario library simulating different business processes, and an event-driven engine simulating different safety accident processes.

3. The intelligent assistant system based on the port professional large model according to claim 1 is characterized in that: The services that users can select through the smart assistant client include intelligent question and answer, information retrieval, and security training.

4. The intelligent assistant system based on the port professional large model according to claim 1 is characterized in that: The online data acquisition module obtains data including accurate time identification from the cluster scheduling system, and the scheduled data is simulated on the virtual business simulator to obtain simulation results of the corresponding data.

5. The intelligent assistant system based on the port professional large model according to claim 1 is characterized in that: The data collected by the online data collection module are continuously stored in the offline database after being processed by the data processing module, and the database is automatically updated to facilitate continuous fine-tuning and optimization of the port model, making port safety training and retrieval information more comprehensive and diverse.

6. The intelligent assistant system based on the port professional large model according to claim 4 is characterized in that: The model fine-tuning pipeline compares the simulated results of the acquired data with the real results acquired to continuously optimize the model capabilities.

7. The intelligent assistant system based on the port professional large model according to claim 1 is characterized in that: The content input by the intelligent assistant user terminal includes one or more of text, voice, and pictures, and the content output by the intelligent assistant user terminal includes one or more of text, voice, pictures, and videos.

8. The intelligent assistant system based on the port professional large model according to claim 4 is characterized in that: When a security incident occurs when simulating business data through a virtual business simulator, the intelligent assistant service can automatically push the event results simulated by the virtual business simulator to the intelligent assistant user end.

Images