Auxiliary maintenance system for ship equipment faults

Abstract

The invention provides an auxiliary maintenance system for ship equipment faults, and relates to the technical field of ship equipment maintenance. The ship equipment fault auxiliary maintenance system comprises a server and terminal equipment, a construction module in the server is used for constructing three-dimensional models of all ship equipment, and a collection module is used for collecting maintenance scene information of subjects used by all the ship equipment. A shooting device in the terminal equipment is used for shooting a real maintenance scene picture and determining to-be-maintained ship equipment; the virtual scene generation module is used for searching the three-dimensional model and the maintenance scene information in the server and generating virtual auxiliary maintenance information of the ship equipment to be maintained; the display device is used for projecting virtual auxiliary maintenance information, and a real maintenance scene can be observed at the same time. According to the method and the device, the virtual auxiliary maintenance information is implanted into the terminal equipment, so that the maintenance personnel can simultaneously see the real scene information and the virtual auxiliary maintenance information, the complex maintenance information becomes easier to understand, and the maintenance efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship equipment maintenance, and more particularly, to an auxiliary maintenance system for ship equipment failures. Background Art

[0002] During the life cycle of a ship, the maintenance cost in the later stage accounts for more than half. For ship products, it is common for emergencies to occur during long voyages when equipment fails and needs to be repaired urgently. Currently, for electromechanical products such as ship equipment with complex structures and intensive technologies, during the process of fault repair, technicians usually have to rely on consulting technical documents to obtain auxiliary maintenance information. This traditional method of obtaining auxiliary maintenance support has problems such as poor information comprehensibility, heavy personnel operation burden, low maintenance efficiency, and will also distract the attention of maintenance personnel to a certain extent, easily leading to operation errors.

[0003] Therefore, in order to reduce the difficulty of maintenance operations and improve the maintenance efficiency of ship equipment, how to propose a maintenance system that can provide good maintainability assistance functions to maintenance personnel has become an urgent technical problem to be solved. Summary of the Invention

[0004] In view of this, the present invention provides a maintenance system that can provide good auxiliary maintenance services for ship crew members in the event of ship equipment failures.

[0005] Specifically, the present invention is implemented through the following technical solutions:

[0006] According to a first aspect of the present invention, there is provided an auxiliary maintenance system for ship equipment failures, including a server, including a construction module and a collection module. The construction module is used to construct three-dimensional models of all ship equipment in the ship, and the collection module is used to collect maintenance scenario information for all subjects used by the ship equipment; a terminal device capable of establishing communication with the server, the terminal device including a photographing device, a virtual scene generation module, and a display device; the photographing device is used to photograph real maintenance scene pictures and can determine the ship equipment to be repaired according to the photographed real maintenance scene pictures; the virtual scene generation module is used to find the corresponding three-dimensional model and maintenance scenario information of the ship equipment to be repaired in the server, and generate virtual auxiliary maintenance information for the ship equipment to be repaired according to the corresponding three-dimensional model and maintenance scenario information; the display device is used to project the virtual auxiliary maintenance information and can observe the real maintenance scene at the same time.

[0007] In some embodiments, the construction module is further used to assign a unique identification code to the three-dimensional models of all ship equipment to form a ship model division table.

[0008] In some embodiments, the identification code is in the form of a digital code, including a compartment number, a deck level number, a system number, and a device number.

[0009] In some embodiments, the server further includes: an interaction module, configured to obtain, identify, and analyze user input information, and determine three-dimensional models of multiple marine equipment to be repaired and multiple repair subjects according to the input information; a processing module, configured to download a combination of the three-dimensional models of the multiple marine equipment to be repaired and the multiple repair subjects to form virtual assisted repair information for combined download; the virtual scene generation module is further configured to generate the virtual assisted repair information for combined download; and the display device is further configured to project the virtual assisted repair information for combined download.

[0010] In some embodiments, the virtual assisted repair information includes at least one of repair operation procedures, failure mechanisms, operation instructions, and safety precautions.

[0011] In some embodiments, the presentation form of the virtual assisted repair information includes at least one of graphics, text, symbols, markers, animations, and 3D models.

[0012] In some embodiments, the display device is an optical see-through helmet, which includes a semi-transmissive and semi-reflective lens. The lens is used to project the virtual assisted repair information and can also directly observe the real environment through the lens.

[0013] In some embodiments, the terminal device further includes: a position adjustment module, connected to the display device, configured to determine the position of the lens relative to the real environment, and adjust the position of the projected virtual assisted repair information according to the position of the lens relative to the real environment.

[0014] In some embodiments, the terminal device further includes: a gesture reference module, which stores key features of multiple different gestures and sets corresponding instructions for each key feature of the gesture; a gesture recognition module, configured to capture the gesture features of the maintenance personnel and compare them with the key features, and determine the instructions corresponding to the gesture features of the maintenance personnel according to the comparison result; and a control module, which controls the assisted repair system according to the instructions corresponding to the gesture features of the maintenance personnel.

[0015] In some embodiments, the gesture reference module includes: a gesture image acquisition unit, configured to acquire multiple gesture images; an image segmentation unit, configured to segment the images of the hands in the multiple gesture images; and a storage unit, configured to extract and store the key features in each image of the hand, form a feature vector set using all the key features, and set corresponding instructions for each key feature.

[0016] In some embodiments, the instructions include confirm, cancel, browse, forward, backward, zoom in, and zoom out.

[0017] The technical solution provided by the present invention has at least the following beneficial effects: After the maintenance personnel enter the ship equipment maintenance scenario, by implanting virtual auxiliary maintenance information into the terminal device, it is no longer necessary to blindly observe the fault information and consult various maintenance manuals and maintenance information, making complex maintenance information easier to understand, strengthening the maintenance guarantee ability of the maintenance personnel for ship equipment, improving the operation efficiency of the maintenance work, and thus reducing the difficulty of maintenance operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is one of the block diagrams of the auxiliary maintenance system for ship equipment faults provided by the embodiments of the present invention.

[0021] Figure 2 It is the second block diagram of the auxiliary maintenance system for ship equipment faults provided by the embodiments of the present invention.

[0022] Figure 3 It is the third block diagram of the auxiliary maintenance system for ship equipment faults provided by the embodiments of the present invention.

[0023] Figure 4 It is the schematic diagram of the principle of the optical see-through helmet provided by the embodiments of the present invention.

[0024] Figure 5 It is the schematic diagram of the auxiliary maintenance process based on augmented reality provided by the embodiments of the present invention.

[0025] Figure 6 It is the schematic diagram of the gesture recognition process provided by the embodiments of the present invention.

[0026] Among them, Figure 1 and Figure 2 The corresponding relationship between the reference numerals and the component names in is as follows:

[0027] 10 Auxiliary maintenance system for ship equipment failures, 1 server, 12 construction module, 14 collection module, 16 interaction module, 18 processing module, 2 terminal device, 21 photographing device, 22 virtual scene generation module, 23 display device, 24 position adjustment module, 25 gesture reference module, 252 gesture image acquisition unit, 254 image segmentation unit, 256 storage unit, 26 gesture recognition module, 27 control module. Detailed implementation manners

[0028] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0029] Refer to Figure 1 , the embodiments of the present invention provide an auxiliary maintenance system 10 for ship equipment failures, including: a server 1, including a construction module 12 and a collection module 14, where the construction module 12 is used to construct three-dimensional models of all ship equipment in the ship, and the collection module 14 is used to collect maintenance scenario information of all subjects used by the ship equipment; a terminal device 2, capable of establishing communication with the server 1, and the terminal device 2 includes a photographing device 21, a virtual scene generation module 22 and a display device 23; the photographing device 21 is used to photograph real maintenance scenario pictures and can determine the ship equipment to be repaired according to the photographed real maintenance scenario pictures; the virtual scene generation module 22 is used to search in the server 1 for the three-dimensional model and maintenance scenario information corresponding to the ship equipment to be repaired, and generate virtual auxiliary maintenance information of the ship equipment to be repaired according to the corresponding three-dimensional model and maintenance scenario information; the display device 23 is used to project the virtual auxiliary maintenance information and can observe the real maintenance scenario at the same time.

[0030] According to the auxiliary maintenance system 10 for ship equipment faults provided by the present invention, ship equipment refers to various devices and facilities in the whole ship that are used for functions such as the normal operation of the ship, safety guarantee, personnel life, and cargo unloading, that is, the whole ship is composed of multiple ship equipment. The server 1 stores three-dimensional models of all ship equipment in the ship and preserves the complete structural information of the ship. At the same time, it also preserves the maintenance scenario information of various equipment-related subjects on the ship (which may include knowledge fields involved in equipment such as machinery, electronics, and hydraulics). These maintenance scenario information are preset. For example, when a certain fault occurs in the equipment, detailed information such as corresponding maintenance steps, required tools, and personnel operation positions. The photographing device 21 in the terminal device 2 is used to photograph the real maintenance scenario picture, determine the ship equipment to be repaired, and then the virtual scenario generation module 22 finds the corresponding equipment model and maintenance scenario information in the server 1, and thereby generates virtual auxiliary maintenance information in a targeted manner. Finally, through the virtual-real fusion processing technology, these virtual auxiliary maintenance information are superimposed on the real maintenance scenario (that is, the real scenario of the ship equipment actually seen by the maintenance personnel). For example, when a maintenance personnel uses the terminal device 2 to view an engine on the ship, the terminal device 2 will superimpose auxiliary maintenance information such as the virtual internal structure and maintenance tips of the engine on the picture of the real engine, so that the maintenance personnel can see both the real engine and the virtual auxiliary maintenance information at the same time, thereby establishing a hybrid scenario with both real ship equipment and virtual maintenance guidance, which is convenient for the maintenance personnel to carry out maintenance work. In this application, by implanting the virtual auxiliary maintenance information into the terminal device 2, the complex maintenance information becomes easier to understand, the maintenance guarantee ability of the ship equipment by the maintenance personnel is strengthened, the operation efficiency of the maintenance work is improved, and thus the difficulty of the maintenance work is reduced.

[0031] In some embodiments, the construction module 12 is further configured to assign a unique identification code to the three-dimensional models of all ship equipment to form a ship model division table.

[0032] In this embodiment, by processing the three-dimensional models of all ship equipment in the whole ship and using the coding method to maintain and form a ship model division table for the whole ship. This division table enables the terminal device 2 to download the three-dimensional model scenario in segments from the server 1 instead of downloading the entire huge three-dimensional model of the whole ship at one time, which can improve the download efficiency of the ship model and is also convenient for flexibly obtaining partial ship equipment models according to requirements.

[0033] In some embodiments, the manifestation form of the identification code is a digital code, including a compartment number, a deck layer number, a system number, and an equipment number.

[0034] In this embodiment, considering the characteristics of the ship model itself, digital coding division is carried out according to the hierarchy of "section - deck layer - system - equipment". In this way, a unique digital coding identifier is used to clarify the 3D model of each section. The digital coding is represented by a 64-bit number. In these 64 bits, 8 bits are reserved at the beginning as the header reserved field (which may be used for future expansion or other special purposes). For the section number, deck layer number, and system number, each is assigned 8 bits of numbering, and the numbering starts from 1. The equipment number is assigned the most bits, with 32 bits, and it also starts numbering from 1. For a simple section model (that is, a section model that does not involve specific deck layers, systems, and equipment), since there is no information about the subsequent levels, except for the 8 bits of the section number numbered according to the actual situation, the fields of the deck layer number, system number, and equipment number are all default set to 0. In this way, through this set of identification codes, the 3D model of the entire ship is clearly and accurately managed in the server 1, and it is convenient for the terminal device 2 to obtain the corresponding section model.

[0035] In some embodiments, the server 1 further includes: an interaction module 16, configured to obtain, identify, and analyze the input information of the user, and determine the 3D models of multiple ship equipment to be repaired and multiple repair subjects according to the input information; a processing module 18, configured to download the 3D models of multiple ship equipment to be repaired and multiple repair subjects in combination to form virtual auxiliary repair information for combined download; the virtual scene generation module 22 is further configured to generate the virtual auxiliary repair information for combined download; the display device 23 is further configured to project the virtual auxiliary repair information for combined download.

[0036] In this embodiment, the server 1 is like a resource library, which stores a lot of content related to the repair subjects of ship equipment. According to the input information of the user, the server 1 can manage and combine the repair knowledge parts (repair subjects, including mechanical repair, electrical repair, software debugging, etc.) of various equipment (such as engines, communication equipment, navigation equipment, etc. on the ship). Maintenance personnel can select multiple repair subjects (such as the mechanical failure repair subject of the engine and the signal debugging repair subject of the communication equipment) of different equipment (such as selecting from the engine and communication equipment) in the interface or operation options provided by the server 1. Then, the server 1 packs only the repair scenarios (including repair steps, display of required tools, precautions, etc.) corresponding to the specific repair subjects of these specific equipment according to the selection of the maintenance personnel, so that users can download these combined contents together. Moreover, this kind of download is on demand. Users do not need to download all the repair scenarios of all equipment, but only obtain the specific part required for the current repair task, thereby improving the download efficiency and saving resources.

[0037] In some embodiments, the virtual assisted maintenance information includes at least one of maintenance operation procedures, failure mechanisms, operation instructions, and safety precautions.

[0038] In some embodiments, the presentation form of the virtual assisted maintenance information includes at least one of graphics, text, symbols, markings, animations, and 3D models.

[0039] In some embodiments, as Figure 4 shown, the display device 23 is an optical see-through helmet, and the optical see-through helmet includes a semi-transmissive and semi-reflective lens, and the lens is used to project virtual assisted maintenance information and can also directly observe the real environment through the lens.

[0040] In this embodiment, the optical see-through helmet includes a semi-transmissive and semi-reflective lens. Part of the lens can project virtual assisted maintenance information, and the other part of the lens can also directly observe the real environment through the lens. In this way, during the process of repairing ship equipment, the maintenance personnel can not only see the real scene of the ship equipment, but also see the virtual assisted maintenance information. That is to say, the present application adopts a virtual-reality fusion technology to establish a hybrid scene in which the virtual and the real coexist in harmony, enabling the operator to simultaneously observe the virtual auxiliary guiding information and the real on-site environment, making complex maintenance information easier to understand, getting rid of the maintenance personnel's dependence on the manual, and improving the operation efficiency.

[0041] In some embodiments, the terminal device 2 further includes: a position adjustment module 24, connected to the display device 23, for determining the position of the lens relative to the real environment and adjusting the position of the projected virtual assisted maintenance information according to the position of the lens relative to the real environment.

[0042] In this embodiment, the position adjustment module 24 is used to determine the position of the lens relative to the real environment, and thereby adjust the position of the projected virtual assisted maintenance information. In this way, the virtual assisted maintenance information can be well presented in front of the user at a suitable position and perspective effect, that is, the fusion effect of the virtual assisted maintenance information and the real environment can be ensured, and the maintenance work can be better assisted.

[0043] In some embodiments, the terminal device 2 further includes: a gesture reference module 25, which stores the key features of various different gestures and sets corresponding instructions for the key features of each gesture; a gesture recognition module 26, for capturing the gesture features of the maintenance personnel and comparing them with the key features, and determining the instructions corresponding to the gesture features of the maintenance personnel according to the comparison result; a control module 27, which controls the assisted maintenance system according to the instructions corresponding to the gesture features of the maintenance personnel.

[0044] In this embodiment, maintenance personnel can control the auxiliary maintenance system through gestures. After the input gesture is recognized as a specific instruction gesture, the corresponding interaction operation is executed, thereby realizing the human-computer interaction function of the auxiliary maintenance system. In this way, there is no need to operate the physical buttons or touch screen on the device by hand, thus improving the convenience of operation for maintenance personnel.

[0045] In some embodiments, as Figure 2 shown, the gesture reference module 25 includes: a gesture image acquisition unit 252 for acquiring multiple gesture images; an image segmentation unit 254 for segmenting the images of the hands in the multiple gesture images; and a storage unit 256 for extracting and storing the key features in each image of the hand, forming a feature vector set using all the key features, and setting corresponding instructions for each key feature.

[0046] In this embodiment, as Figure 6 shown, the gesture reference module 25 distinguishes between the online stage and the offline stage. In the offline stage, first, multiple gesture images are acquired, and the images of the hands are segmented. Then, by extracting the key features in each image of the hand and combining them, a feature vector set is obtained. At the same time, corresponding instructions are assigned to each gesture. In the online stage, after the gesture recognition module 26 recognizes the gesture features of the maintenance personnel, the feature vector set constructed in the offline stage is used for instruction gesture recognition, thereby realizing the interactive control of the auxiliary maintenance system.

[0047] In some embodiments, the instructions include confirmation, cancellation, browsing, forward, backward, zoom in, and zoom out.

[0048] The following further introduces the auxiliary maintenance system 10 for ship equipment failures in this application in combination with a specific embodiment.

[0049] For the current maintenance of complex ship equipment, it generally involves the maintenance and use of a large number of instruments and equipment, making the maintenance and repair steps numerous and cumbersome. In addition, there are problems such as difficulty in consulting manuals and human errors, which require those with long-term training and rich experience to be able to perform the work. At the same time, the maintenance and repair of complex equipment often need to be borne by the equipment unit during the berthing at the dock, and it is difficult to handle emergencies during navigation. Therefore, it is necessary to further provide good maintenance assistance for the ship's crew, thereby reducing the difficulty of maintenance operations and improving the maintenance efficiency.

[0050] Based on the characteristics of a huge amount of ship models and complex equipment maintenance subjects, this application proposes an application mode of server + terminal. Users can configure the auxiliary maintenance scenario according to the requirements of the auxiliary maintenance scenario and download the configured scenario to the mobile terminal to realize the auxiliary maintenance of ship products through the terminal device 2.

[0051] As Figure 3As shown in the figure, the overall system architecture is an application mode of server + terminal. The full-ship 3D model and the auxiliary maintenance scenarios of equipment are produced in the background; at the same time, the 3D model division table of the full ship and the equipment maintenance subject scenario table are maintained in the server.

[0052] For specific auxiliary maintenance requirements, maintenance personnel can configure the 3D model area to be repaired and the maintenance subjects of the maintenance equipment on the server, and realize the combination of specific maintenance scenarios through user customization, and can download the combined scenarios to the augmented reality auxiliary device terminal.

[0053] After the maintenance 3D model and the maintenance scenario are downloaded to the terminal, the terminal device 2 can realize the fusion display of the auxiliary information and the real scene to assist the maintenance personnel in carrying out maintenance support operations; at the same time, the maintenance operator can interact with the system through gestures to realize the interactive operation with the auxiliary maintenance system.

[0054] As Figure 5 shown, the main technical points of this application include:

[0055] a. The overall architecture is an application mode of server + terminal; the server maintains the 3D model of the full ship and the maintenance scenarios of the subjects used by the equipment; the terminal adopts a wearable augmented reality device, and the augmented reality device can generate targeted virtual maintenance support information with rich details, and through the virtual-real fusion processing, the system superimposes the above-mentioned maintenance support information onto the maintenance scenario to establish a hybrid scenario in which the virtual and the real coexist in harmony.

[0056] b. In the server, the full-ship model is encoded to maintain and form a 3D model division table of the full ship, and this division table can realize the segmented download of the 3D model scenario in the server by the terminal.

[0057] Based on the characteristics of the ship model, it can be encoded and divided according to hold section - deck layer - system - equipment; according to the above division standard, a unique identifier is used to define each segmented model, and the definition is as follows:

[0058] (hold section number i, deck layer number i, system function ID, equipment ID)

[0059] It is expressed by a 64-bit number, in which 8 bits are reserved at the head, 8 bits for the hold section number, 8 bits for the deck number and the system ID respectively, and 32 bits are set for the equipment ID, all starting from 1. For a pure hold section model, other fields are not required and are default set to 0.

[0060]

[0061] c. In the server, implement the combined configuration function for maintenance subjects of multiple devices. Users can select multiple maintenance subjects of different devices to achieve the combined download of maintenance scenarios for specific devices and specific subjects, and download specific maintenance scenarios as needed.

[0062] d. Through wearable terminal devices, achieve the visual presentation of auxiliary maintenance information, including maintenance operation procedures, fault mechanisms, operation instructions, safety precautions, etc., and the forms include graphics, text, symbols, marks, animations, 3D models, etc.

[0063] In this application, an optical see-through helmet is mainly used to achieve virtual-real fusion display. The display module is composed of semi-transmissive and semi-reflective optical devices. The display projects the virtual information generated by the virtual scene generator onto the lens. While seeing the virtual information, the user can also directly observe the real environment through the lens. During this process, the visual scene coordinates and perspective transformation relationship of the virtual information on the display plane formed by the lens are obtained after system tracking data and related calibration operations.

[0064] As Figure 6 shown, the embodiment of the present invention provides a gesture recognition method, which may include the following steps:

[0065] S602. Reference image.

[0066] S604. Extract image features.

[0067] S606. Establish a description of the key frames of the scene and provide a reference feature template.

[0068] S608. Video stream.

[0069] S610. Extract the features of the on-site pictures.

[0070] S612. Compare with the feature library to check if they are consistent. If so, execute S614; otherwise, return to S608.

[0071] S614. Feature point matching.

[0072] S616. Feature point screening.

[0073] S618. Compare the number of feature points with the threshold.

[0074] S620. Render the model after successful registration.

[0075] e. The wearable device can recognize the gestures of maintenance personnel through the camera. The camera captures the gesture pictures, performs image segmentation processing on the obtained input gesture image information, and performs instruction gesture recognition after obtaining the feature vector set of the input gesture, so as to achieve system interaction control.

[0076] 3. Beneficial effects

[0077] As Figure 5 shown, in this application, the ship equipment control computer based on augmented reality generates targeted and detailed virtual maintenance support information; at the same time, through the virtual-real fusion process, the system superimposes the above-mentioned maintenance support information onto the maintenance scene, thereby establishing a hybrid scene in which virtual and real coexist and cooperate.

[0078] Compared with the traditional maintenance support mode, augmented reality technology can provide guiding and assisting prompt information for operators in real time at the site through the display device 23. The virtual-real fusion-assisted maintenance information enables operators to observe both the virtual assisting guiding information and the real environment on-site, getting rid of the dependence on manuals, and at the same time, the open field of vision ensures the safety of operations.

[0079] Through the implantation of augmented reality technology, complex maintenance information becomes easier to understand, the operation efficiency of maintenance work is improved, and the maintenance support ability of maintenance personnel for ship equipment is strengthened.

[0080] In the embodiments according to the present invention, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance; the term "plurality" means two or more, unless otherwise clearly defined. Terms such as "installed", "connected", "connected to", "fixed" and other terms should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments according to the present invention can be understood according to specific circumstances.

[0081] In addition, although the operations are depicted in a specific order, it should be understood that such operations are required to be performed in the specific order shown or in a sequential order, or that all the illustrated operations should be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present invention. Certain features described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, the various features described in the context of a single implementation can also be implemented separately or in any suitable sub-combination in multiple implementations.

[0082] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

[0083] The above are only preferred embodiments according to the embodiments of the present invention, and are not used to limit the embodiments according to the present invention. For those skilled in the art, various changes and modifications can be made according to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments according to the present invention shall be included within the protection scope of the embodiments according to the present invention.

Claims

1. An auxiliary maintenance system (10) for ship equipment failure, characterized in that: include: The server (1) comprises a construction module (12) and a collection module (14), wherein the construction module (12) is used to construct a three-dimensional model of all ship equipment in the ship, and the collection module (14) is used to collect maintenance scenario information of subjects used by all the ship equipment; A terminal device (2) capable of establishing communication with the server (1), the terminal device (2) comprising a shooting device (21), a virtual scene generation module (22) and a display device (23); The photographing device (21) is used to photograph a real maintenance scene image, and can determine the ship equipment to be repaired based on the photographed real maintenance scene image; The virtual scene generation module (22) is used to search the server (1) for a three-dimensional model and maintenance scene information corresponding to the ship equipment to be repaired, and to generate virtual auxiliary maintenance information of the ship equipment to be repaired based on the corresponding three-dimensional model and the maintenance scene information; The display device (23) is used to project the virtual auxiliary maintenance information and simultaneously observe the real maintenance scene.

2. The auxiliary maintenance system (10) for ship equipment failure according to claim 1, characterized in that: The construction module (12) is also used to assign unique identification codes to the three-dimensional models of all the ship equipment to form a ship model division table.

3. The auxiliary maintenance system (10) for ship equipment failure according to claim 2, characterized in that: The identification code is in the form of a digital code, including a compartment number, a deck number, a system number and an equipment number.

4. The auxiliary maintenance system (10) for ship equipment failure according to claim 1, characterized in that: The server (1) further comprises: An interactive module (16) is used to obtain, identify and analyze user input information, and determine a plurality of three-dimensional models of ship equipment to be repaired and a plurality of repair subjects according to the input information; A processing module (18) is used to download a plurality of three-dimensional models of the ship equipment to be repaired and a plurality of the repair subjects in combination to form virtual auxiliary repair information downloaded in combination; The virtual scene generation module (22) is also used to generate virtual auxiliary maintenance information for the combined download; The display device (23) is also used to project the virtual auxiliary maintenance information downloaded by the combination.

5. The auxiliary maintenance system (10) for ship equipment failure according to claim 1, characterized in that: The virtual auxiliary maintenance information includes at least one of maintenance operation procedures, failure mechanisms, operation instructions and safety precautions; and / or The presentation form of the virtual auxiliary maintenance information includes at least one of graphics, text, symbols, marks, animations and 3D models.

6. The auxiliary repair system (10) for ship equipment failure according to any one of claims 1 to 5, characterized in that: The display device (23) is an optical perspective helmet, which includes a semi-transparent and semi-reflective lens, which is used to project the virtual auxiliary maintenance information, and at the same time, the real environment can be directly observed through the lens.

7. The auxiliary repair system (10) for ship equipment failure according to claim 6, characterized in that: The terminal device (2) further includes: A position adjustment module (24) is connected to the display device (23) and is used to determine the position of the lens relative to the real environment and adjust the position of projecting the virtual auxiliary maintenance information according to the position of the lens relative to the real environment.

8. The auxiliary repair system (10) for ship equipment failure according to any one of claims 1 to 5, characterized in that: The terminal device (2) further includes: A gesture reference module (25), wherein the gesture reference module (25) stores key features of a plurality of different gestures and sets corresponding instructions for the key features of each gesture; A gesture recognition module (26) is used to capture the gesture features of the maintenance personnel and compare them with the key features, and determine the instructions corresponding to the gesture features of the maintenance personnel according to the comparison results; A control module (27) controls the auxiliary maintenance system according to instructions corresponding to the gesture characteristics of the maintenance personnel.

9. The auxiliary repair system (10) for ship equipment failure according to claim 8, characterized in that: The gesture reference module (25) comprises: A gesture image acquisition unit (252), used for acquiring a plurality of gesture images; An image segmentation unit (254), used for segmenting the hand images from the plurality of gesture images; The storage unit (256) is used to extract and store key features in each of the hand images, use all of the key features to form a feature vector set, and set a corresponding instruction for each of the key features.

10. The auxiliary repair system (10) for ship equipment failure according to claim 8, characterized in that: The instructions include confirm, cancel, browse, forward, backward, zoom in, and zoom out.

Images