Visual identification-based ship equipment standardized operation process prompting system and method

Abstract

The invention discloses a ship equipment standardized operation process prompting system based on visual identification, and the system comprises an equipment regulation mapping knowledge base which is used for storing the mapping relation between the visual key features of ship equipment and the standardized operation process; the intelligent agent server comprises a visual identification model and retrieval logic and is used for processing the image uploaded by the user terminal and returning corresponding regulation information; an application program is installed on the user mobile terminal, and the application program comprises a camera shooting control module, an image uploading module and an information display module; the ship equipment standardized operation process prompting system and method based on visual identification have instantaneity and convenience, standardization and safety, efficient knowledge inheritance, digitization and structuralization of knowledge and paper regulations, and capability of active pushing, so that new employees can quickly and accurately learn and master complex equipment operation, and the operation efficiency of the ship equipment is improved. And the training cost and period are greatly reduced.

Description

Technical Field

[0001] This invention relates to the fields of industrial safety and artificial intelligence application technology, and in particular to a system and method for prompting standardized operating procedures for ship equipment based on visual recognition. Background Technology

[0002] In the shipbuilding industry, there are numerous types of equipment with complex structures, and their operation and maintenance have strict standardized procedures. Currently, employees, especially new employees or those unfamiliar with working in cabins, mainly rely on the following methods to learn about equipment operating procedures: consulting paper manuals, which is inefficient, extremely inconvenient in emergencies or on-site work, and the manuals are not updated in a timely manner; memory and experience, which rely on personal memory and instruction from mentors, and are prone to forgetting or improper operation of equipment that is not frequently operated; centralized training, which is costly, time-consuming, cannot cover all the details of all equipment, and is difficult to conduct anytime and anywhere. The above methods have obvious problems of "information silos" and "knowledge lag". When facing unfamiliar equipment, employees are unable to quickly and accurately obtain standard operating procedures (SOPs) and safety precautions, which can easily lead to equipment damage, production stoppages, or even personal safety accidents due to improper operation. Summary of the Invention

[0003] The purpose of this invention is to provide a visual recognition-based system and method for prompting standardized operating procedures for ship equipment, thereby reducing safety risks at the source and achieving standardized operation.

[0004] The technical solution adopted by the present invention to achieve the above objectives is: a visual recognition-based standardized operation procedure prompting system for ship equipment, comprising: An equipment procedure mapping knowledge base is used to store the mapping relationship between the visual key features of ship equipment and standardized operating procedures; An intelligent agent server, containing a visual recognition model and retrieval logic, is used to process images uploaded by user terminals and return corresponding procedural information; A user mobile terminal having an application installed thereon, the application including a camera control module, an image uploading module, and an information display module.

[0005] Another technical solution adopted by the present invention to achieve the above objectives is: a method for prompting standardized operating procedures for ship equipment based on visual recognition, comprising the following steps: a. Construct an equipment procedure mapping knowledge base, collect image data of various ship equipment, construct a training dataset, use a large visual model to train and extract features from the image data, learn and store the key visual features of each type of equipment, and associate and map the key visual features of each type of equipment with its corresponding structured standardized operating procedures and safety precautions. b. Build and deploy an intelligent agent server. Deploy an intelligent agent server that can receive image input and output text information, and integrate the device procedure mapping knowledge base built in step a into the intelligent agent server. c. Equipment identification and procedure push: Real-time images of target ship equipment are acquired through the camera module of the mobile terminal and uploaded to the intelligent agent server. The uploaded images are analyzed in real time through the visual big data model in the intelligent agent server to extract their key visual features. The extracted key visual features are then quickly matched and retrieved with records in the equipment procedure mapping knowledge base.

[0006] In step a, the standardized operating procedures and safety precautions are displayed on the mobile terminal in the form of structured text, images, or lists.

[0007] In step c, the mobile terminal is a smartphone or tablet computer running an application that enables image capture, uploading, and information display.

[0008] In step c, if a match is successful, the corresponding standardized operating procedures and safety precautions are retrieved from the equipment procedure mapping knowledge base, and the results are returned to the mobile terminal application. The standardized operating procedures and safety precautions are clearly and intuitively displayed on the mobile terminal's display interface.

[0009] This invention discloses a visual recognition-based standardized operating procedure prompting system and method for ship equipment. It offers immediacy and convenience, enabling "instant learning and instant application," allowing personnel to obtain precise operational guidance without leaving the work site, significantly improving work efficiency and emergency response speed. Standardization and safety are also enhanced by pushing unified and standardized operating procedures, effectively eliminating operational irregularities caused by differences in individual understanding and experience, thus preventing safety hazards at the source and significantly reducing the accident rate. Furthermore, the system facilitates efficient knowledge transfer by digitizing and structuring expert knowledge and paper-based procedures, and giving them the ability to proactively push information. This allows new employees to quickly and accurately learn and master the operation of complex equipment, greatly reducing training costs and time. Detailed Implementation

[0010] A visual recognition-based system for prompting standardized operating procedures for ship equipment includes: an equipment procedure mapping knowledge base for storing the mapping relationship between key visual features of ship equipment and standardized operating procedures; an intelligent agent server containing a visual recognition model and retrieval logic for processing images uploaded by user terminals and returning corresponding procedure information; and a user mobile terminal with an application installed, the application including a camera control module, an image uploading module, and an information display module.

[0011] A visual recognition-based method for prompting standardized operating procedures for ship equipment includes the following steps: a) Constructing an equipment procedure mapping knowledge base: collecting image data of various ship equipment, building a training dataset, using a large visual model to train and extract features from the image data, learning and storing the key visual features of each type of equipment, and associating the key visual features of each type of equipment with its corresponding structured standardized operating procedures and safety precautions. The standardized operating procedures and safety precautions are displayed on a mobile terminal in the form of structured text, images, or lists; b) Constructing and deploying an intelligent agent server: deploying an intelligent agent server capable of receiving image input and outputting text information, and integrating the equipment procedure mapping knowledge base constructed in step a into the intelligent agent server; c) Equipment recognition and procedure... The push notification process is executed through an application on the user's mobile terminal, which is a smartphone or tablet running an application that enables image capture, uploading, and information display. The mobile terminal's camera module captures real-time images of the target ship's equipment, which are then uploaded to an intelligent agent server. The server uses a large visual model to analyze the uploaded images in real time, extracting key visual features. These features are then quickly matched and retrieved against records in the equipment procedure mapping knowledge base. If a match is found, the corresponding standardized operating procedures and safety precautions are retrieved from the knowledge base and returned to the mobile terminal application. The standardized operating procedures and safety precautions are then clearly and intuitively displayed on the mobile terminal's interface.

[0012] In an example, taking the identification of an "emergency shut-off valve" in a ship's engine room as an example, firstly, an equipment procedure mapping knowledge base is constructed. Hundreds of images of ship emergency shut-off valves from different angles and of different models are collected and trained using a large visual model. The model learns the unique visual features of this type of valve, such as the handwheel shape, valve body structure, and color. Subsequently, these features are associated with the "Emergency Shut-off Valve Operation Procedure" document, which is structured as: "Step 1, confirm the upstream and downstream pressure...; Step 2, rotate the handwheel clockwise until closed...; Safety Precautions: Wear protective gloves before operation and ensure the ground is dry..." This mapping relationship is stored in a database. Then, an intelligent agent server is deployed, and the knowledge base and trained visual model are deployed to a cloud server or edge computing node, providing API interfaces. Finally, in field application, an employee... When operating an unfamiliar valve, the user opens a dedicated app on their mobile phone, points the camera at the valve, and the app automatically captures a clear image and uploads it to the intelligent agent server. The visual model on the intelligent agent server quickly identifies that the image features match the "emergency shut-off valve" in the knowledge base with a matching degree exceeding 99%. It then retrieves the corresponding operating procedures and safety precautions and returns the information to the employee's mobile app interface within 1-2 seconds. The employee does not need to consult any paper documents; they can clearly see each step of the operation and the necessary safety warnings on their mobile screen, enabling them to complete the operation safely and in accordance with regulations. Through the above embodiments, this invention demonstrates a visual recognition-based standardized operating procedure prompting system and method for ship equipment, effectively addressing the pain point of on-site personnel "not knowing how to operate," and achieving intelligent and precise knowledge acquisition.

Claims

1. A standardized operating procedure prompting system for ship equipment based on visual recognition, characterized in that, include: An equipment procedure mapping knowledge base is used to store the mapping relationship between the visual key features of ship equipment and standardized operating procedures; An intelligent agent server, containing a visual recognition model and retrieval logic, is used to process images uploaded by user terminals and return corresponding procedural information; A user mobile terminal having an application installed thereon, the application including a camera control module, an image uploading module, and an information display module.

2. A prompting method for a visual recognition-based standardized operating procedure prompting system for ship equipment according to claim 1, characterized in that, Includes the following steps: a. Construct an equipment procedure mapping knowledge base, collect image data of various ship equipment, construct a training dataset, use a large visual model to train and extract features from the image data, learn and store the key visual features of each type of equipment, and associate and map the key visual features of each type of equipment with its corresponding structured standardized operating procedures and safety precautions. b. Build and deploy an intelligent agent server. Deploy an intelligent agent server that can receive image input and output text information, and integrate the device procedure mapping knowledge base built in step a into the intelligent agent server. c. Equipment identification and procedure push: Real-time images of target ship equipment are acquired through the camera module of the mobile terminal and uploaded to the intelligent agent server. The uploaded images are analyzed in real time through the visual big data model in the intelligent agent server to extract their key visual features. The extracted key visual features are then quickly matched and retrieved with records in the equipment procedure mapping knowledge base.

3. The method for providing standardized operating procedure prompts for ship equipment based on visual recognition according to claim 2, characterized in that: In step a, the standardized operating procedures and safety precautions are displayed on the mobile terminal in the form of structured text, images, or lists.

4. The method for providing standardized operating procedure prompts for ship equipment based on visual recognition according to claim 2, characterized in that: In step c, the mobile terminal is a smartphone or tablet computer running an application that enables image capture, uploading, and information display.

5. The method for providing standardized operating procedure prompts for ship equipment based on visual recognition according to claim 2, characterized in that: In step c, if a match is successful, the corresponding standardized operating procedures and safety precautions are retrieved from the equipment procedure mapping knowledge base, and the results are returned to the mobile terminal application. The standardized operating procedures and safety precautions are clearly and intuitively displayed on the mobile terminal's display interface.