AR-based ship mooring test method
Automatically record data through image recognition and speech recognition technology in ship mooring tests through AR glasses and remote guidance, solving the errors and confidence differences caused by traditional manual recording, and achieving fast and accurate data recording and testing processes .
Patent Information
- Application Number
- CN202411063261.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-12
AI Technical Summary
In traditional ship mooring tests, manual data recording is prone to errors, resulting in waste of resources in repeated tests and poor test confidence.
Using AR-based ship mooring test method, data recording is performed through AR glasses, image recognition and voice recognition technology are used to automatically record equipment data, and remote expert guidance is provided when problems occur, and electronic test data tables with hyperlinks are generated for production. Upload.
Improve the accuracy and test confidence of data recording, reduce the number of repeated tests, and the generated electronic documents can be used throughout the life cycle, enabling fast, accurate recording and remote guidance of data.
Smart Images

Figure CN118939123A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of ship engineering technology, and in particular to an AR-based ship mooring test method. Background Art
[0002] The ship mooring test is to check whether the various systems and equipment of the ship are operating normally during the ship construction process. Through a series of tests, it can be ensured that it can operate safely and reliably before use. The mooring test is designed to cover many aspects of the ship's structure, machinery, equipment and boilers. Due to the large number of mooring test items and the need for repeated tests, a large amount of data is recorded during the test process. In the traditional mooring test process, the test personnel manually record the mooring test outline like a book, and the recorded data is entered into an electronic document through a computer for backup, which often causes data recording errors and other phenomena; often when the quality inspector or the ship owner conducts an inspection, the mooring test personnel need to restart the equipment for the quality inspector or the ship owner to check the ship's operating status, which leads to repeated tests and waste of resources.
[0003] In summary, it is necessary to provide an improved technical solution to address the above-mentioned deficiencies in the prior art. Summary of the invention
[0004] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a ship mooring test method based on AR, which can effectively and quickly record the content of the ship mooring test. Based on the existing paper version of the electronic test outline of the field test, an electronic test outline document is quickly generated, and this program is published in the AR glasses. The on-site mooring test personnel wear AR glasses to conduct mooring tests, record data through voice, image recognition, etc., and calculate the corresponding data through operational relationships, and analyze the data. Expert remote guidance is provided through AR glasses for places where problems occur. The whole process is recorded through text, voice, pictures, videos, etc. Solve problems such as recorded data errors and poor test confidence.
[0005] To achieve the above-mentioned object and other related objects, the present invention provides a ship mooring test method based on AR, comprising the following steps:
[0006] S1. Preparation before mooring test: input the mooring test outline form through automatic recognition to form an electronic test outline;
[0007] S2. Start the mooring test: record the current test date and test personnel and enter the inspection area, then divide all the test equipment and the mooring test area into test sections. As the mooring test personnel wear AR glasses and walk to different locations, they can inspect the equipment contained in different test sections in front of them;
[0008] S3. Record the test data of the corresponding equipment through image recognition and voice recognition. For regular instruments and meters on the equipment, the AR device can automatically recognize and record data; for specific instruments and meters on the equipment, the data can be recorded by voice;
[0009] S4. During the test, the equipment test data is judged to determine whether the test data is within the normal data range;
[0010] S5. If the test data is within the normal data range, use AR glasses to save photos and videos of the data recording process, or further use AR glasses to explain the situation by voice; after completion, return to step S2 to record the mooring test data for the next test part;
[0011] S6. When the test data does not conform to the normal data range, the abnormal situation is explained by voice through the AR glasses, or further remote connection is made to apply for remote guidance to solve the problem; during the data recording process, AR glasses are used to record photos and videos, and after completion, return to step S2 to record the mooring test data for the next test section.
[0012] S7. After all experimental data are completed or part of the data is recorded, an electronic mooring test data table with a hyperlink is generated. The electronic mooring test data table includes recorded data, pictures, and videos. The electronic mooring test data table is automatically uploaded to the cloud when there is a network.
[0013] Optionally, some mooring test outline forms contain calculation relationships between different data and range requirements for filling in data, and the calculation relationships and range requirements are synchronously built into the electronic test outline form; when the test personnel fill in the corresponding test data, the data to be calculated is directly calculated and filled into the corresponding form through the calculation relationship; the test data has corresponding color prompts in the electronic form, reflecting whether the test data is within the range requirements.
[0014] Optionally, in step S2, when the test personnel moves to switch the test equipment, they input the equipment name in the AR glasses by voice, and the AR glasses automatically plan the shortest test route.
[0015] Optionally, the voice method includes the test personnel directly speaking the corresponding equipment keyword or the test personnel calling up the test outline for a full review and then speaking which step of the test to be performed.
[0016] Optionally, automatically planning the shortest test route includes: obtaining the spatial position data of each device of the moored test ship in advance, publishing the spatial position data and the model framework in the AR glasses, and planning the shortest test route for the test personnel to move on the ship by superimposing the virtual ship position data and the real ship position data during the test and knowing the current position of the test personnel.
[0017] Optionally, in step S2, when conducting experiments in a certain area, the AR glasses mark the spatial data and test items through SLAM spatial grid technology. When the experimenter conducts the same project test under different energy consumption, when entering the area again, he can quickly switch to the test content of the current area.
[0018] Optionally, in step S3, the AR device realizes automatic recognition of regular instruments and meters by model training, and the regular instruments and meters include rectangles, circles, squares and lines; and the image recognition model is trained by taking photos of each regular instrument and meter at different angles.
[0019] Optionally, in step S6, remote guidance is provided through AR glasses and remote experts. The remote experts use PAD, computer or VR glasses to complete problem guidance. On-site test personnel use AR glasses and spatial annotation technology to mark the problem on the object, and the remote experts mark the corresponding solutions on the tablet. During the communication process, information is transmitted through text, pictures, files and voice.
[0020] Optionally, an electronic mooring test data sheet with a hyperlink and a model package of the mooring test generated by the test data can be directly viewed in the system. The model package of the mooring test can display the progress of the mooring test. The model package can be instantly viewed in the background after the test personnel upload the data. The model package can display the mooring test outline logic and hierarchical tree. On the hierarchical tree, the test personnel can view the equipment test data recorded during the test, including equipment test data including data, pictures and videos.
[0021] Optionally, when the AR glasses are tested without a network, the data, pictures and videos during the test can be cached in the AR glasses, and the data can be uploaded when there is a network.
[0022] As described above, the present invention provides a ship mooring test method based on AR, which has the following beneficial effects:
[0023] 1. Paper documents can be quickly converted into electronic documents through image recognition and AI analysis;
[0024] 2. This method can free your hands. You don’t need to hold paper documents to record data. You can quickly record data through voice and image recognition technology to prevent human errors caused by multiple records.
[0025] 3. Improve the confidence of mooring test data and reduce frequent tests. Under normal circumstances, testers only need to conduct the required number of tests and upload the test photos and videos to the cloud for quality assurance personnel and ship owners to check at any time. The generated electronic document report can be hyperlinked to the text, voice, pictures and videos recorded in the mooring test to generate an electronic ship mooring outline report for use throughout the ship's life cycle.
[0026] 4. If you encounter any difficult problems during the mooring test, you can connect with remote experts at any time through AR glasses for remote guidance and quick problem solving. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 Shown is a flow chart of the ship mooring test method based on AR in Embodiment 1 of the present invention. DETAILED DESCRIPTION
[0028] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.
[0029] For example, when describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional view showing the device structure will not be partially enlarged according to the general scale, and the schematic view is only an example, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional space dimensions of length, width and depth should be included.
[0030] For ease of description, spatial relational terms such as "under", "below", "below", "below", "above", "on", etc. may be used herein to describe the relationship of one element or feature shown in the drawings to other elements or features. It will be understood that these spatial relational terms are intended to include other directions of the device in use or operation in addition to the directions depicted in the drawings. In addition, when a layer is referred to as being "between" two layers, it can be the only layer between the two layers, or there can be one or more intervening layers. As used herein, "between..." means including the end point values.
[0031] In the context of the present application, a structure in which a first feature is described as being "above" a second feature may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features are formed between the first and second features, such that the first and second features may not be in direct contact.
[0032] It should be noted that the illustrations provided in this embodiment are only used to illustrate the basic concept of the present invention in a schematic manner, and therefore the illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0033] Embodiment 1
[0034] like Figure 1 As shown, this embodiment provides a ship mooring test method based on AR, comprising the following steps:
[0035] S1. AR glasses application preparation. Before the start of the mooring test, it is necessary to determine the type of ship for the mooring test. By using image recognition and AI technology to automatically identify and input the mooring test outline form, an electronic test outline for the mooring test is formed. Since the paper version of the mooring test outline has more than 500 pages, the mooring content can be trained through image recognition and AI analysis, and the paper document can be quickly scanned into an electronic document.
[0036] Secondly, establish the operation relationship between different interrelated data; then, add normal data range judgment to all data, and give a prompt when the test data exceeds this range. The same type of ship uses the same set of mooring test outlines; for ships of the same type but different sizes, the mooring test outline can be edited and modified accordingly. If there is an AR test outline for this type of ship, the mooring test can be carried out directly.
[0037] S2. When starting the test, record the current test date and test personnel and enter the inspection area. Then divide all the test equipment and mooring test areas into test sections. As the mooring test personnel wear AR glasses and walk to different locations, they can inspect the different equipment in front of them.
[0038] When switching the moving position of the test equipment, the device name can be quickly entered into the AR glasses by voice, and the AR glasses can quickly plan the shortest test route.
[0039] The voice method can search for inspection items through fuzzy logic methods. For example, you can quickly retrieve and locate the test items by saying the corresponding equipment keywords. You can also check which step of the mooring test the test area belongs to by calling up the test outline content, and directly say the inspection step to switch to the test location.
[0040] Planning the shortest test route requires obtaining the spatial position data of each device on the moored test ship in advance, publishing these data and the model framework in the AR glasses, and superimposing the virtual ship position data and the real ship position data during the test. By knowing the current position of the test personnel, the shortest test path for movement on the ship can be planned for the test personnel.
[0041] In addition, when conducting experiments in a certain area, the AR glasses mark the spatial data and test items through SLAM spatial grid technology. When the experimenters conduct experiments on the same project under different energy consumption, they can enter the area again and quickly switch the test content.
[0042] S3. After entering the corresponding inspection area, the corresponding equipment test data is recorded through image recognition and voice recognition. There are the following classifications for different recording methods. There are many equipment data that need to be recorded on board, including instruments and meters of pressure, flow, temperature, etc., and these instruments and meters are divided into various shapes such as round and long strips. This AR device performs a large amount of image acquisition and model training for common round instrument meters, long strip instrument meters, and instrument meters that display text and numbers, so as to meet the AR device's rapid recognition of the data of the inspected object. For a small number of equipment and specific equipment, image recognition is not performed, and the equipment data on the equipment can be recorded by voice.
[0043] Preferably, in an implementable manner, photographs of pressure, flow, temperature and other types of instruments and meters at different angles are first collected, and the instruments are divided into rectangular, circular, square and linear types; image recognition models are trained on photographs of various instruments at different angles, and the trained model can collect data while the mooring test personnel are carrying AR glasses.
[0044] S4. During the test, the equipment test data is judged to determine whether the test data is within the normal data range.
[0045] S5. If the test data is within the normal data range, use AR glasses to save photos and videos of the data recording process, and use AR glasses to provide voice explanations when necessary; after completion, return to step S2 to record the mooring test data for the next test section;
[0046] S6. When the test data does not conform to the normal data range, the abnormal situation is explained by voice through the AR glasses. If necessary, a remote connection is made to apply for remote guidance to solve the problem. During the data recording process, AR glasses are used to record photos and videos. After completion, return to step S2 to record the mooring test data for the next test section.
[0047] In one feasible method, when remote guidance is provided to remote experts at any time through AR glasses, the remote experts can use PAD, computer or VR glasses to complete the problem guidance. On-site test personnel can use AR glasses and spatial annotation technology to mark the problem on the object, and the expert can also solve the problem by marking the corresponding problem on the tablet. During the communication process, information such as text, pictures, and files can also be transmitted.
[0048] S7. After all test data are recorded or part of the data are recorded, an electronic mooring test data sheet with a hyperlink is generated. The electronic mooring test data sheet includes recorded data, pictures, and videos. The electronic mooring test data sheet is automatically uploaded to the cloud when there is a network. Mooring tests based on AR glasses are not affected by or restricted by the network. The recorded data can be stored locally and uploaded when there is a network or in a local area network. Remote AR guidance by experts is not possible without a network. After all tests are completed, electronic documents, test descriptions, voice, images, and videos will be uploaded to the cloud for other test personnel, shipowners, and quality inspectors to view.
[0049] Preferably, an electronic mooring test data sheet with a hyperlink and a model package of the mooring test generated from the test data can be directly viewed in the system. The model package of the mooring test can display the progress of the mooring test. The model package allows test personnel to upload data and immediately view it in the background model package. The model package can display the logic of the mooring test outline and the hierarchical tree. On the hierarchical tree, test personnel can view the data, pictures, videos and other contents recorded during the test.
[0050] Preferably, the electronic mooring test data sheet has a hyperlink, and the electronic data sheet format is the same as the paper version of the mooring test outline. By clicking on the data in the electronic report, pictures and videos of the test can be quickly viewed.
[0051] Preferably, some ship interior environments are without network conditions. When the AR glasses are tested without network conditions, the data, pictures and videos of the test process will be cached in the AR glasses, and the data will be uploaded when there is network. The electronic data sheet uploaded to the cloud is a complete and easy-to-access file generated by the AR mooring test program (the document recording some data will also be uploaded and will be replaced in the cloud when it is updated and more complete); the ship hull is a steel structure with strong signal shielding, and the internal space of the ship is mostly without network conditions, so it must be automatically uploaded when there is network.
[0052] In summary, the present invention provides a ship mooring test method based on AR, which can effectively and quickly record the contents of the ship mooring test with the help of AR equipment. The test method is based on the existing paper version of the electronic test outline on site, quickly generates an electronic version of the test outline document, and publishes this program in AR glasses. On-site mooring test personnel wear AR glasses to perform mooring tests, record data through voice, image recognition, etc., calculate corresponding data through operation relationships, and analyze data. Expert remote guidance is provided through AR glasses for places where problems occur. The entire process is recorded through text, voice, pictures, videos, etc. It solves problems such as recorded data errors and poor test confidence.
[0053] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.
Claims
1. A ship mooring test method based on AR, characterized in that: The steps include: S1. Preparation before mooring test: input the mooring test outline form through automatic recognition to form an electronic test outline; S2. Start the mooring test: record the current test date and test personnel and enter the inspection area, then divide all the test equipment and the mooring test area into test sections. As the mooring test personnel wear AR glasses and walk to different locations, they can inspect the equipment contained in different test sections in front of them; S3. Record the test data of the corresponding equipment through image recognition and voice recognition. For regular instruments and meters on the equipment, the AR device can automatically recognize and record data; for specific instruments and meters on the equipment, the data can be recorded by voice; S4. During the test, the equipment test data is judged to determine whether the test data is within the normal data range; S5. If the test data is within the normal data range, use AR glasses to save photos and videos of the data recording process, or further use AR glasses to provide voice explanations; After completion, return to step S2 to record the mooring test data for the next test section; S6. When the test data does not conform to the normal data range, the abnormal situation is explained by voice through the AR glasses, or further remote connection is made to apply for remote guidance to solve the problem; during the data recording process, photos and videos are recorded with AR glasses, and after completion, the process returns to step S2 to record the mooring test data for the next test section; S7. After all experimental data are completed or part of the data is recorded, an electronic mooring test data table with a hyperlink is generated. The electronic mooring test data table includes recorded data, pictures, and videos. The electronic mooring test data table is automatically uploaded to the cloud when there is a network.
2. According to the AR-based ship mooring test method of claim 1, some mooring test outline forms contain operation relationships between different data and range requirements for filling in data, and the operation relationships and range requirements are synchronously built into the electronic test outline form; when the test personnel fill in the corresponding test data, the data to be calculated is directly calculated and filled into the corresponding form through the operation relationship; the test data has corresponding color prompts in the electronic form, reflecting whether the test data is within the range requirements.
3. The ship mooring test method based on AR according to claim 1, characterized in that: In step S2, when the test personnel moves to switch the test equipment, they input the equipment name in the AR glasses by voice, and the AR glasses automatically plan the shortest test route.
4. The ship mooring test method based on AR according to claim 3, characterized in that: The voice method includes the test personnel directly saying the corresponding equipment keyword or the test personnel calling up the test outline and then saying which step of the test to be performed.
5. The ship mooring test method based on AR according to claim 3, characterized in that: Automatic planning of the shortest test route includes: obtaining the spatial position data of each device of the moored test ship in advance, publishing the spatial position data and the model framework in the AR glasses, and planning the shortest test route for the test personnel on the ship by superimposing the virtual ship position data and the real ship position data and knowing the current position of the test personnel during the test.
6. The ship mooring test method based on AR according to claim 3, characterized in that: In step S2, when conducting an experiment in a certain area, the AR glasses mark the spatial data and test items through the SLAM spatial grid technology. When the experimenter conducts the same project experiment under different energy consumption, when entering the area again, he can quickly switch to the test content of the current area.
7. The ship mooring test method based on AR according to claim 1, characterized in that: In step S3, the AR device realizes automatic recognition of regular instruments and meters through model training, and the regular instruments and meters include rectangles, circles, squares and lines; the image recognition model is trained by taking photos of each regular instrument and meter at different angles.
8. The ship mooring test method based on AR according to claim 1, characterized in that: In step S6, remote guidance is provided through AR glasses and remote experts. The remote experts use PAD, computer or VR glasses to complete problem guidance. The on-site test personnel use AR glasses and spatial annotation technology to mark the problem on the object, and the remote experts mark the corresponding solutions on the tablet. During the communication process, information is transmitted through text, pictures, files and voice.
9. The ship mooring test method based on AR according to claim 1, characterized in that: In the system, one can directly view the electronic mooring test data sheet with hyperlinks and the model package of the mooring test generated by the test data. The model package of the mooring test can display the progress of the mooring test. The model package can be viewed immediately in the background after the test personnel upload the data. The model package can display the logic and hierarchical tree of the mooring test outline. On the hierarchical tree, the test personnel can view the equipment test data recorded during the test, including equipment test data including data, pictures and videos.
10. The ship mooring test method based on AR according to claim 1, characterized in that: When AR glasses are tested without a network, the data, pictures and videos of the test process can be cached in the AR glasses, and the data can be uploaded when there is a network.
