Navigation analysis model development and application system based on visual data

Abstract

The invention discloses a navigation analysis model development and application system based on visual data. The navigation analysis model development and application system comprises a model construction module, a model training module, a model evaluation module, a model achievement publishing module, a model deployment application module and a model monitoring module. The model construction module is used for constructing a navigation analysis model online in a visual form, and storing a model result in a preset model library for version management; the model training module is used for calling computing resources for online model training, recording each training, comparing the effect of each training and screening an ideal navigation analysis model; the model evaluation module is used for evaluating the model effect of the navigation analysis model and optimizing and upgrading the model according to the evaluation result; the model achievement publishing module is used for publishing the navigation analysis model to an external portal for sharing and opening; and the model deployment application module is used for deploying the navigation analysis model to an application service.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a navigation analysis model development and application system based on visual data. Background Art

[0002] With the booming development of the global shipping industry, navigation analysis plays a crucial role in ship operation, route planning, maritime safety, etc. However, most traditional navigation analysis methods rely on manual experience and simple data statistics, and it is difficult to effectively cope with the complex and changeable navigation environment and massive data. This not only limits the accuracy and efficiency of navigation analysis, but also increases the risks and costs of maritime transportation.

[0003] In recent years, the rapid development of technologies such as big data and artificial intelligence has brought new opportunities to navigation analysis. By introducing advanced data processing and analysis technologies, we can more accurately grasp the internal laws and characteristics of navigation data and provide more scientific decision-making support for ship operation. At the same time, the rise of visualization technology also enables complex data and information to be presented intuitively, greatly improving the readability and usability of navigation analysis. Currently, the data analysis methods and tools in the field of navigation analysis often seem inadequate when dealing with complex and multi-dimensional navigation data, not only with low analysis efficiency, but also difficult to discover the deep-seated laws and patterns hidden behind the data. Summary of the Invention

[0004] The purpose of the present invention is to provide a navigation analysis model development and application system based on visual data to solve the above problems.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A navigation analysis model development and application system based on visual data, including a model construction module, a model training module, a model evaluation module, a model result publishing module, a model deployment and application module, and a model monitoring module; the model construction module is used to construct a navigation analysis model online in a visual form and save the model results to a preset model library for version management; the model training module is used to call computing resources to perform model training online, record each training, compare the effects of each training, and select an ideal navigation analysis model; the model evaluation module is used to evaluate the model effect of the navigation analysis model and optimize and upgrade the model according to the evaluation results; the model result publishing module is used to publish the navigation analysis model to an external portal for sharing and opening; the model deployment and application module is used to deploy the navigation analysis model to an application service; the model monitoring module is used to monitor the entire process of the model development and application process.

[0007] Preferably, the online construction process of the navigation analysis model in the model construction module is specifically as follows:

[0008] A1. Obtain the navigation ship data resource set, select the required data set and preview it;

[0009] A2. Create a navigation analysis model for shipping-related analysis projects based on the selected data set;

[0010] A3. Select the required functional components for model construction from the pre-set data model toolbox and navigation operator model set according to the research process, drag them to the visualization canvas for use, build the model workflow of the navigation analysis model, and add text for description;

[0011] A4. Package the built model workflow, save it as a reusable process, or share it with others for reuse.

[0012] Preferably, the model construction module is built with a model library, which is used to save and record the status and situation of each update during the construction and training of the navigation analysis model; among them, in the new training of the navigation analysis model, the required algorithms can be directly mounted and selected. The system will download the corresponding algorithm files and their dependencies to the storage space in the same environment as the training data to improve the loading performance during actual training and use them in the form of pre-trained models.

[0013] Preferably, the training process of the navigation analysis model in the model training module is specifically as follows:

[0014] B1. Invoke computing resources to divide the training set and test set in a visual form;

[0015] B2. Select the model training method;

[0016] B3. Adjust and transform the main parameters and features of the algorithm and evaluation visualization components, thereby set the model iteration, and record the performance and training results of each model iteration training process;

[0017] B4. Compare and analyze the effects of each training, and screen the optimal training effect of the model.

[0018] Preferably, the evaluation method of the navigation analysis model in the model evaluation module is specifically as follows: select an evaluation method that matches the navigation analysis model, drag the evaluation method to the visualization canvas, set the evaluation index, evaluate the model effect, and based on the evaluation result, edit a suitable optimization algorithm through code to solve the optimal parameters of the model, and continuously optimize and upgrade the navigation analysis model.

[0019] Preferably, the model result publishing module includes a model saving unit, a model result sharing unit, and a model version management unit; the model saving unit is used to publish and save the internal model file of the navigation analysis model trained by the model training module to the model library, and supports the direct uploading of external model files to the model library; the model success sharing unit is used to publish the models in the model library to an external portal for public sharing, and supports the sharing and co-management of the models by different parties; the model version management unit records the iteration situation through a management algorithm model preset in the model library, realizes the sorting, sharing, and reuse of model versions, and supports the modification, adjustment, deletion, and taking off the shelf of models.

[0020] Preferably, the model deployment module supports one-stop convenient deployment, including direct publishing and deployment in a project, and also including deployment from the model library.

[0021] Preferably, the application service methods of the model deployment module include API services and web application services. The API services are used for developers inside and outside the platform to develop or call models, and the inference results can be obtained through instant calls. The web application services are used for external users to obtain model services.

[0022] Preferably, the model monitoring module includes an operation monitoring unit, a call monitoring unit, and a log recording unit. The operation monitoring unit is used to view the usage of hardware resources of each replica of the current service. The call monitoring unit is used to query the statistics and records of REST service calls. The log recording unit is used to record the service logs of the REST service.

[0023] After adopting the above technical solutions, compared with the background technology, the present invention has the following advantages:

[0024] The present invention provides a navigation analysis model development and application system based on visual data, which realizes the full-chain automatic processing from data input to result output. Users can easily build and customize navigation analysis models through a visual interface, and use the computing resources of the system for efficient model training and evaluation, providing strong support for the intelligent and efficient development of the shipping industry. By optimizing route planning, improving ship operation efficiency, reducing maritime transportation risks, etc.; the system also supports publishing model results to an external portal for sharing and opening, providing strong support for the popularization and application of navigation analysis technology, and bringing significant economic and social benefits to shipping enterprises. At the same time, the scalability and customizability of the system also enable it to adapt to the diverse needs of different users, providing a broad space for the sustainable development of the shipping industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a system structure diagram of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] Embodiment

[0028] Please refer to Figure 1 As shown, the present invention discloses a development and application system for a navigation analysis model based on visual data, including a model construction module, a model training module, a model evaluation module, a model result publishing module, a model deployment and application module, and a model monitoring module; the model construction module is used to construct a navigation analysis model online in a visual form and save the model results to a pre-set model library for version management; the model training module is used to retrieve computing resources to perform model training online, record each training, compare the effects of each training, and screen for an ideal navigation analysis model; the model evaluation module is used to evaluate the model effect of the navigation analysis model and optimize and upgrade the model according to the evaluation results; the model result publishing module is used to publish the navigation analysis model to an external portal for sharing and opening; the model deployment and application module is used to deploy the navigation analysis model to an application service; the model monitoring module is used to monitor the entire process of the development and application of the model.

[0029] The specific process of online construction of the navigation analysis model in the model construction module is as follows:

[0030] A1. Obtain a navigation ship data resource set, select the required data set and preview it;

[0031] The user can obtain the navigation ship data resource set presented according to the asset catalog after applying for approval through the platform. The structured data supports the browsing and application of the navigation ship data resource set, including but not limited to global ship AIS data, global container tracking data, sea freight bulk cargo tracking data, sea freight oil and gas energy tracking data, etc. For structured data, content preview can be performed online, and at the same time, visual statistics can be automatically generated, including but not limited to average, mode, median, etc.; at the same time, a distribution frequency histogram of single-field data can be provided to facilitate the user to quickly understand the data situation.

[0032] A2. Create a navigation analysis model for a shipping-related analysis project based on the selected data set;

[0033] A3. Select the required functional components for model construction from a pre-set data model toolbox and a navigation operator model set according to the research process, drag them to the visual canvas for use, build the model workflow of the navigation analysis model, and add text for explanation;

[0034] A4. Package the built model workflow, save it as a reusable process, or share it with others for reuse.

[0035] The model building module has a built-in model library, which is used to save and record the status and situation of each update during the construction and training of the navigation analysis model; among them, in the new training of the navigation analysis model, the required algorithms can be directly mounted and selected. The system will download the corresponding algorithm files and their dependencies to the storage space in the same environment as the training data to improve the loading performance during actual training and use it in the form of a pre-trained model.

[0036] For each update of the algorithm model, the platform supports users to generate versions and record the current model situation. Users can define the name, description, build time, release time, and creator of the model. Users can view past versions in the version list, select the corresponding version to create a model service, create an analysis project, and invite collaborators.

[0037] The training process of the navigation analysis model in the model training module is specifically as follows:

[0038] B1. Invoke computing resources to divide the training set and test set in a visual form. Taking the dataset in csv format as an example, pull the data reading component on the canvas, and the reading ratio can be set as the training set or test set. Input the original dataset to complete the division and loading of the dataset. Similarly, datasets in other formats can be read.

[0039] B2. Select the model training method. The platform supports users to perform online training. In addition, for long-term training or large-scale analysis tasks, it supports users to offline host the Notebook development file and visual development file on the platform for offline training, and can return the training results and the resource usage during the training process in real time, and supports synchronizing and saving the training results within the system.

[0040] B3. Adjust and transform the main parameters and features of the algorithm and evaluation visualization components to set the model iteration, and record the performance and training results of each model iteration training process;

[0041] B4. Compare and analyze the effects of each training and screen the optimal training effect of the model.

[0042] The system supports the model training record function of mainstream frameworks such as TensorFlow, Pytorch, and Keras, provides the record monitoring of the performance and training results of each training process. The performance includes but is not limited to GPU utilization, CPU utilization, storage occupancy, etc. It supports users to set the model Metrics information of the training results by themselves. The Metrics information includes but is not limited to the records of best epoch, train acc, train loss, test acc, test loss (val acc, val loss). All the recorded training information is displayed through appropriate visualization methods, supports the visualization comparison analysis of different training records of the same model, and can intuitively view the optimal training effect of the model.

[0043] The specific evaluation method of the navigation analysis model in the model evaluation module is as follows: select an evaluation method that matches the navigation analysis model, drag the evaluation method to the visualization canvas, set the evaluation indicators, evaluate the model effect, and based on the evaluation results, edit a suitable optimization algorithm through code to solve the optimal parameters of the model, and continuously optimize and upgrade the navigation analysis model. Optimization algorithms such as gradient descent, least squares method, and Bayesian can be used to optimize and solve the parameters.

[0044] The evaluation methods include regression model evaluation, clustering model evaluation, and confusion matrix. The evaluation indicators include accuracy, precision, and recall. The platform supports users to set the model Metrics information of the training results by themselves. The Metrics information includes but is not limited to the records of best epoch, train acc, train loss, test acc, test loss (val acc, val loss). All the recorded training information is displayed through appropriate visualization methods.

[0045] The model achievement release module includes a model saving unit, a model achievement sharing unit, and a model version management unit; the model saving unit is used to publish and save the internal model file of the navigation analysis model trained by the model training module to the model library, and supports the direct upload of external model files to the model library; the model success sharing unit is used to publish the model in the model library to the external portal for external public sharing, and supports the use and joint management of the model; the model version management unit records the iteration situation through the management algorithm model preset in the model library, realizes the sorting, sharing, and reuse of the model version, and supports the modification, adjustment, deletion, and removal of the model.

[0046] The model files obtained through training support users to directly publish and save them in the model library. They can be saved in the PMML (Predictive Model Markup Language) format and allow for export, sharing, and loading. For external model files, users are supported to directly upload them within the model library. It supports visual parsing of model files output by mainstream machine learning frameworks, enabling users to intuitively obtain the internal structure of the model, view the model composition, model structure, as well as the input, output, and corresponding parameter descriptions of each layer of network nodes:

[0047] (1) For machine learning models trained and stored using the following training frameworks, it supports parsing and visualizing the structure and parameters: ONNX (.onnx,.pb,.pbtxt), Keras (.h5,.keras), Core ML (.mlmodel), Caffe (.caffemodel,.prototxt), Caffe2 (predict_net.pb, predict_net.pbtxt), MXNet (.model, -symbol.json), TorchScript (.pt,.pth), NCNN (.param), TensorFlow Lite (.tflite);

[0048] (2) For machine learning models trained and stored using the following training frameworks, it supports experimental parsing and visualizing the structure and parameters: PyTorch (.pt,.pth), Torch (.t7), CNTK (.model,.cntk), Deeplearning4j (.zip), PaddlePaddle (.zip, __model__), Darknet (.cfg), scikit - learn (.pkl), TensorFlow.js (model.json,.pb), and TensorFlow (.pb,.meta,.pbtxt).

[0049] Model files can be loaded into the project for use in the form of data sources. To assist users in managing the entire life cycle of the model, model services, training records, source code (and usage data) support associated traceability. The system can automatically associate its source code, usage data, experiments, and file details.

[0050] The model deployment module supports one - stop convenient deployment, including directly publishing and deploying in the project, as well as deploying from the model library.

[0051] The application service methods of the model deployment module include API services and web application services. API services are used by developers inside and outside the platform for model development or invocation, and inference results can be obtained through immediate invocation. Web application services are used for external users to obtain model services. Batch deployment services are supported.

[0052] The model files generated during model research and development support building RESTful API services through one-click deployment. It supports direct publishing and deployment in a project, and also supports deployment from the model repository. The platform automatically constructs REST services with the model files output by the project as the carrier, and supports creators to flexibly configure the images and computing resources required for inference services. The number of CPU cores and the memory size used by the service can be configured. Batch deployment and multi-concurrent request invocation are supported, and the maximum and minimum number of replicas for model invocation can be set. The computing power used by the service will be flexibly scaled and expanded as needed.

[0053] It supports one-click publishing of REST model services as web applications. Users can configure the inputs and outputs of the inference services to build externally accessible web applications. The inputs support simultaneous input in formats such as files and text (floating-point numbers, integers, boolean values, etc.), and the outputs support simultaneous output in formats such as text, structured data, and pictures. Obtaining real-time inference results on the web side is supported, controlling and managing the access rights to the web page is supported, and setting data privacy protocols for web applications is supported.

[0054] The model monitoring module supports monitoring the entire process of model construction, training, evaluation, publishing, deployment, and invocation, records the operation logs of the entire process, monitors and analyzes the entire process of model development and application on the entire scientific research platform, and can analyze situations such as popular models, abnormal invocations, and time-consuming models.

[0055] The model monitoring module includes an operation monitoring unit, an invocation monitoring unit, and a log recording unit. The operation monitoring unit is used to view the usage of hardware resources of each replica of the current service when the service type is a REST service, supports viewing the video memory and GPU usage of the GPU model service to track and understand the actual operation of the service, and supports timely recycling of model service instances that are no longer in use (such as some old instances after configuration modification) to avoid waste caused by resource occupation. The invocation monitoring unit is used to query REST service invocation statistics and records, view monitoring logs, export model application data in actual scenarios as a dataset for secondary training of the model and optimizing the model. The log recording unit is used to record the service logs of REST services.

[0056] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A navigation analysis model development and application system based on visual data, characterized in that: It includes a model construction module, a model training module, a model evaluation module, a model result publishing module, a model deployment and application module, and a model monitoring module; the model construction module is used to online construct a navigation analysis model in a visual form and save the model results to a preset model library for version management; the model training module is used to retrieve computing resources to online train the model, record each training, compare the effects of each training, and select an ideal navigation analysis model; the model evaluation module is used to evaluate the model effect of the navigation analysis model and optimize and upgrade the model according to the evaluation results; the model result publishing module is used to publish the navigation analysis model to the external portal for sharing and opening; the model deployment and application module is used to deploy the navigation analysis model to the application service; the model monitoring module is used to monitor the entire process of the model development and application process.

2. The development and application system of a navigation analysis model based on visual data according to claim 1, characterized in that: The specific process of online construction of the navigation analysis model in the model construction module is as follows: A1. Obtain the navigation ship data resource set, select the required data set and preview it; A2. Create a navigation analysis model for the shipping-related analysis project based on the selected data set; A3. Select the functional components required for model construction from the preset data model toolbox and navigation operator model set according to the research process, drag them to the visual canvas for use, build the model workflow of the navigation analysis model, and add text for explanation; A4. Package the built model workflow, save it as a reusable process, or share it with others for reuse.

3. The nautical analysis model development and application system based on visual data according to claim 2, wherein: The model construction module is built-in with a model library, which is used to save and record the status and situation of each update during the construction and training of the navigation analysis model; among them, in the new training of the navigation analysis model, the required algorithm can be directly mounted and selected, and the system will download the corresponding algorithm file and its dependencies to the storage space in the same environment as the training data to improve the loading performance during actual training and use it in the form of a pre-trained model.

4. The nautical analysis model development and application system based on visual data according to claim 3, characterized in that: The specific process of training the navigation analysis model in the model training module is as follows: B1. Retrieve computing resources to divide the training set and the test set in a visual form; B2. Select the model training method; B3. Set the model iteration by adjusting and transforming the main parameters and features of the algorithm and the evaluation visualization components, and record the performance and training results of each model iteration training process; B4. Compare and analyze the effects of each training and select the optimal training effect of the model.

5. The nautical analysis model development and application system based on visual data according to claim 3, characterized in that: The specific evaluation method of the navigation analysis model in the model evaluation module is as follows: select an evaluation method that matches the navigation analysis model, drag the evaluation method to the visual canvas, set the evaluation index, evaluate the model effect, and based on the evaluation results, edit a suitable optimization algorithm through code to solve the optimal parameters of the model and continuously optimize and upgrade the navigation analysis model through iteration.

6. The nautical analysis model development and application system based on visual data according to claim 3, wherein: The model result publishing module includes a model saving unit, a model result sharing unit, and a model version management unit; the model saving unit is used to publish and save the internal model file of the navigation analysis model trained by the model training module to the model library, and supports the direct upload of external model files to the model library; The model success sharing unit is used to publish the models in the model library to the external portal for external public sharing, and supports the use and co-management of the models in pairs; the model version management unit records the iteration situation through the management algorithm model preset in the model library, realizes the sorting, sharing and reuse of model versions, and supports the modification, adjustment, deletion and removal of models.

7. The development and application system of a navigation analysis model based on visual data according to claim 3, wherein: The model deployment module supports one-stop convenient deployment, including direct publishing and deployment in the project, and also including deployment from the model library.

8. The nautical analysis model development and application system based on visual data according to claim 7, characterized in that: The application service methods of the model deployment module include API services and web application services. The API service is used for developers inside and outside the platform to develop or call models, and obtains inference results through instant calls. The web application service is used for external users to obtain model services.

9. The nautical analysis model development and application system based on visual data according to claim 3, characterized in that: The model monitoring module includes an operation monitoring unit, a call monitoring unit, and a log recording unit. The operation monitoring unit is used to view the usage of hardware resources of each replica of the current service. The call monitoring unit is used to query the REST service call statistics and records. The log recording unit is used to record the service logs of the REST service.

Images