A launch pad intelligent visualization inspection method and system based on digital twin

Abstract

The present invention discloses a method and system for intelligent visual inspection of a transmitter station based on digital twins. A three-dimensional station inspection scene model is constructed based on a three-dimensional model of elements. A trigger area is created for the equipment model and equipment attributes are added. An overlapping event is added according to the created trigger area. When the inspection personnel model enters the area, the equipment operation information is automatically obtained and an information page is generated. An array of inspection navigation points is obtained, and inspection route navigation points are automatically generated to complete the automatic inspection of the station. The three-dimensional model provides overlapping responses and newly added attributes, and has good application scalability; the navigation point array is automatically generated, and the navigation point position change can be realized following the change of the equipment position, and it has high flexibility; an inspection process for orderly and automated acquisition of equipment information is created, which can provide information acquisition of a group of related equipment at a time, thereby improving inspection efficiency and information relevance.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection technology, and in particular to a method and system for intelligent visual inspection of a launch station based on digital twins. Background Art

[0002] In the existing operation and maintenance system of transmitter stations, the inspection method of equipment is mainly manual inspection. In the station operation and maintenance software system, equipment information is usually acquired at a fixed time, stored in the database, and displayed on the web page. The existing flat operation and maintenance system cannot quickly and accurately locate the position of the equipment. When a problem occurs in the equipment, it is impossible to carry out subsequent treatment in time; and the existing three-dimensional operation and maintenance system requires the mouse to click on the model to obtain data. The occlusion of the three-dimensional model will make it difficult to click the mouse, and the data acquisition efficiency is low. In addition, the cruise point needs to be manually configured, the navigation point generation efficiency is low, and the operation and use are inconvenient. Summary of the invention

[0003] To this end, the present invention provides a launch station intelligent visualization inspection method and system based on digital twins to solve the problems of low data acquisition efficiency and poor inspection efficiency in the existing launch station operation and maintenance system.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] According to a first aspect of an embodiment of the present invention, a method for intelligent visual inspection of a launch station based on digital twins is proposed, the method comprising:

[0006] Conduct three-dimensional modeling of the components of the station inspection scene, and construct a three-dimensional station inspection scene model based on the three-dimensional model of the components, wherein the three-dimensional model of the components includes a building model, an equipment model, and an inspection personnel model;

[0007] Create a trigger area for the device model and add device attributes, add overlapping events according to the created trigger area, and when the inspection personnel model enters the area, automatically obtain the device operation information and generate an information page;

[0008] According to the device attributes and the location coordinates of the device trigger area, the inspection navigation point coordinates are obtained and the inspection navigation point array is obtained, and the inspection route navigation points are automatically generated;

[0009] Based on the inspection route navigation points, the inspection personnel model is controlled to move to each navigation point one by one, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station.

[0010] Furthermore, a three-dimensional station inspection scene model is constructed based on the element three-dimensional model, specifically including:

[0011] Import the model files of each element into the UE4 engine to build a three-dimensional station inspection scene model based on each element.

[0012] Furthermore, a trigger area is created for the device model and device attributes are added, specifically including:

[0013] Create a box trigger area, the length, width and height of the area are respectively equal to the length, width and height of the inspection personnel model;

[0014] The device attributes include device ID and whether to perform inspection.

[0015] Furthermore, the device operation information is automatically obtained and an information page is generated, including:

[0016] When the inspection personnel model enters the trigger area, it sends a device operation information request containing the device ID to the Web server interface. After receiving the request, the Web server accesses the real remote device API to obtain the device operation information. After receiving the device return information, in the UE, it uses the UE's built-in browser to render the information page in the client software.

[0017] Furthermore, the method further comprises:

[0018] Add an exit overlap event based on the created trigger area, and close the information page when the inspector model leaves the area.

[0019] Furthermore, the method further comprises:

[0020] The inspection equipment model is placed at the corresponding position in the three-dimensional station inspection scene model with reference to the actual position and the equipment properties are set.

[0021] Furthermore, according to the device attributes and the location coordinates of the device triggering area, the inspection navigation point coordinates are obtained and the inspection navigation point array is obtained, which specifically includes:

[0022] Traverse all inspection devices. If the inspection is true in the device's properties, obtain the location range of the device's trigger area, obtain the coordinates of the middle position of the area as the inspection navigation point of the device, and add it to the inspection navigation point array.

[0023] Furthermore, the method further includes: creating an inspection AI based on the inspection personnel model, specifically:

[0024] Create a behavior tree based on the navigation component. The behavior tree controls the inspection model to move to navigation points one by one.

[0025] Furthermore, the method further comprises:

[0026] Use the navigation component to create a navigation area, implement an automatic path-finding system, and place the created inspection personnel model into the navigation area to complete automatic inspection.

[0027] According to a first aspect of an embodiment of the present invention, a launch station intelligent visualization inspection system based on digital twin is proposed, and the system includes:

[0028] A three-dimensional station inspection scene model construction module is used to perform three-dimensional modeling on the components of the station inspection scene, and to construct a three-dimensional station inspection scene model based on the three-dimensional model of the components, wherein the three-dimensional model of the components includes a building model, an equipment model and an inspection personnel model;

[0029] A navigable inspection equipment establishment module is used to create a trigger area for the equipment model and add equipment attributes. An overlapping event is added according to the created trigger area. When the inspection personnel model enters the area, the equipment operation information is automatically obtained and an information page is generated.

[0030] Inspection route navigation point generation module, used to obtain inspection route navigation point coordinates and obtain inspection route navigation point array according to device attributes and location coordinates of device trigger area, and automatically generate inspection route navigation points;

[0031] The automatic inspection module is used to control the inspection personnel model to move to each navigation point one by one based on the inspection route navigation points, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station.

[0032] The present invention has the following advantages:

[0033] The present invention proposes a method and system for intelligent visualization inspection of a transmitter based on digital twins. The method constructs a three-dimensional station inspection scene model based on a three-dimensional model of elements, creates a trigger area for the equipment model and adds equipment attributes, adds overlapping events according to the created trigger area, and automatically obtains equipment operation information and generates an information page when the inspection personnel model enters the area. The inspection navigation point coordinates are obtained according to the equipment attributes and the position coordinates of the equipment trigger area, and the inspection navigation point array is obtained, and the inspection route navigation points are automatically generated to complete the automatic inspection of the station. Compared with the ordinary three-dimensional modeling model, it provides overlapping responses and newly added attributes, and has application scalability that the original three-dimensional model does not have; it can automatically generate a navigation point array based on the equipment model, which can improve the efficiency of navigation point generation compared to manually configuring the cruise point, and can follow the change of the equipment position to realize the change of the navigation point position, which has high flexibility; based on the navigation point and the navigable model, an inspection process for orderly and automated acquisition of equipment information is created, in which the equipment status information can be obtained in real time and orderly, so that information acquisition of a group of related equipment can be provided at a time, improving the inspection efficiency and information relevance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0035] Figure 1 A schematic diagram of a flow chart of a method for intelligent visual inspection of a launch station based on digital twins provided in Example 1 of the present invention;

[0036] Figure 2 A schematic diagram of the process of obtaining a navigation point array in a method for intelligent visual inspection of a launch station based on digital twins provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0037] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Example 1

[0039] like Figure 1 As shown, this embodiment proposes a launch station intelligent visualization inspection method based on digital twins, which includes:

[0040] S100, three-dimensional modeling is performed on the constituent elements of the station inspection scene, and a three-dimensional station inspection scene model is constructed based on the element three-dimensional model. The element three-dimensional model includes a building model, an equipment model, and an inspection personnel model.

[0041] S200, create a trigger area for the device model and add device attributes, add overlapping events according to the created trigger area, and when the inspection personnel model enters the area, automatically obtain the device operation information and generate an information page.

[0042] S300, according to the device attributes and the location coordinates of the device triggering area, the coordinates of the inspection navigation point are obtained and the inspection navigation point array is obtained, and the inspection route navigation point is automatically generated.

[0043] S400, based on the inspection route navigation points, control the inspection personnel model to move to each navigation point one by one, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station.

[0044] The specific implementation process is as follows:

[0045] (1) Use 3D modeling software to build the 3D building model, equipment model, and inspection personnel model of the transmitter station. Output the model as an Fbx format model file.

[0046] (2) Import the Fbx format model file into the UE4 engine to form a three-dimensional station monitoring model based on buildings, equipment, and personnel.

[0047] (3) Establish a navigable inspection equipment Actor based on the equipment model.

[0048] A. Create a box trigger area in the positive direction of the device panel of the model. The length, width and height of the area are equal to the length, width and height of the inspection personnel model respectively;

[0049] B. Add the properties [Device ID] and [Whether to inspect] to the blueprint;

[0050] C. Add an overlap event to the box trigger area created by A. When an object enters the area, the device ID is sent to the Web server interface to obtain the device information. After receiving the request, the Web server accesses the real remote device API to obtain the device operation information. After the Web server receives the information returned by the device, it uses the built-in browser of the UE to render the information page in the client software.

[0051] D. Add an exit overlap event to the box trigger area created in A. When the object leaves the area, close the browser page.

[0052] (4) Place the inspection equipment at the corresponding position in the three-dimensional scene of the station established in [2] with reference to the actual position.

[0053] After placing the equipment, you need to set the relevant properties of the navigable inspection equipment Actor.

[0054] A. [Device ID] is a required option and its value must be set correctly. It can be any unique string. It is recommended to set it according to the existing system. If there is no existing system, the uniqueness of the ID must be guaranteed.

[0055] B. The setting of [Whether to inspect] attribute is related to the establishment of navigation point:

[0056] Can be directly set to True or False through manual configuration;

[0057] Save the device ID and the inspection attribute in a csv configuration file. Create an import blueprint and read the configuration file in the level blueprint. Then traverse all the navigable device Actors in the level. If the Actor's device ID exists in the configuration file, read the attribute value in the configuration file and set it in the [Inspection] attribute of the navigable device Actor in the level.

[0058] (5) Create an array of inspection navigation points, automatically generate inspection route navigation points and add them to the navigation point array.

[0059] Traverse all the inspection devices of the type created in step 3. If the device's attribute [Is it inspected] is true, obtain the location range of the device trigger area, obtain the coordinates of the middle position of the cube as the inspection navigation point of the device, and add it to the inspection navigation point array. The process of obtaining the navigation point coordinates is as follows: Figure 2 As shown, first obtain the coordinates of the axis point of the object trigger area (x1, y1, z1), then obtain the trigger area range size (w, l, h), and get the navigation point coordinates (x1+w / 2, y1+l / 2, z1+h / 2), and add the coordinates to the navigation point array.

[0060] (6) Create inspection AI based on the inspection personnel model.

[0061] A behavior tree is created based on the built-in navigation component of UE. The behavior tree controls the inspection model to move to navigation points one by one.

[0062] (7) Use the built-in navigation component of the UE to create a navigation area to realize the automatic path-finding system. Place the patrol personnel model created in step (6) into the navigation area. During the movement of the patrol personnel model, the relevant data is read and displayed in real time through step (4), thereby realizing automatic inspection of the station.

[0063] This embodiment proposes a launch station intelligent visualization inspection method based on digital twins, which has the following advantages:

[0064] Compared with the common 3D model, the model created by this method provides overlapping response and two new attributes, and has application scalability that the original 3D model does not have. The subsequent method of establishing navigation points utilizes the attribute components added by this type of model.

[0065] This method can automatically generate a navigation point array based on the navigation device model. Compared with manually configuring cruise points, it can improve the efficiency of navigation point generation and can change the navigation point position according to the device position change, which has high flexibility.

[0066] Based on navigation points and navigable models, UE4's AI is used to create an orderly and automated inspection process for obtaining device information. In this process, device status information can be obtained in real time and in an orderly manner. This can provide information acquisition for a group of related devices at a time, improve inspection efficiency and information relevance, and use UE4's 3D scene to improve the sense of presence.

[0067] Example 2

[0068] Corresponding to the above-mentioned embodiment 1, this embodiment proposes a launch station intelligent visualization inspection system based on digital twins, and the system includes:

[0069] The 3D station inspection scene model building module is used to perform 3D modeling on the elements of the station inspection scene, and build a 3D station inspection scene model based on the element 3D model. The element 3D model includes a building model, an equipment model, and an inspection personnel model.

[0070] A navigable inspection equipment establishment module is available to create a trigger area for the equipment model and add equipment attributes. Overlapping events are added based on the created trigger area. When the inspection personnel model enters the area, the equipment operation information is automatically obtained and an information page is generated.

[0071] Inspection route navigation point generation module, used to obtain inspection route navigation point coordinates and obtain inspection route navigation point array according to device attributes and location coordinates of device trigger area, and automatically generate inspection route navigation points;

[0072] The automatic inspection module is used to control the inspection personnel model to move to each navigation point one by one based on the inspection route navigation points, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station.

[0073] The functions performed by each component in the digital twin-based launch station intelligent visualization inspection system provided by an embodiment of the present invention have been introduced in detail in the above-mentioned embodiment 1, so they will not be elaborated here.

[0074] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for intelligent visual inspection of a launch station based on digital twins, characterized in that: The method comprises: Conduct three-dimensional modeling of the components of the station inspection scene, and construct a three-dimensional station inspection scene model based on the three-dimensional model of the components, wherein the three-dimensional model of the components includes a building model, an equipment model, and an inspection personnel model; Create a trigger area for the device model and add device attributes, add overlapping events according to the created trigger area, and when the inspection personnel model enters the area, automatically obtain the device operation information and generate an information page; According to the device attributes and the location coordinates of the device trigger area, the inspection navigation point coordinates are obtained and the inspection navigation point array is obtained, and the inspection route navigation points are automatically generated; Based on the inspection route navigation points, control the inspection personnel model to move to each navigation point one by one, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station; Constructing a three-dimensional station inspection scene model based on the three-dimensional model of the elements, specifically including: importing the model files of each element into the UE4 engine, and constructing a three-dimensional station inspection scene model based on each element; Creating a trigger area for the device model and adding device attributes, specifically including: creating a box trigger area, the length, width and height of the area are respectively equal to the length, width and height of the inspection personnel model; the device attributes include device ID and whether to inspect; Automatically obtain equipment operation information and generate information pages, specifically including: when the inspection personnel model enters the trigger area, send a device operation information request containing the device ID to the Web server interface. After receiving the request, the Web server accesses the real remote device API to obtain the equipment operation information, and after receiving the device return information, in the UE, use the UE's built-in browser to render the information page in the client software.

2. According to claim 1, a method for intelligent visual inspection of a launch station based on digital twins is characterized in that: The method further comprises: Add an exit overlap event based on the created trigger area, and close the information page when the inspector model leaves the area.

3. According to the method of intelligent visual inspection of a launch station based on digital twins according to claim 1, it is characterized in that: The method further comprises: The inspection equipment model is placed at the corresponding position in the three-dimensional station inspection scene model with reference to the actual position and the equipment properties are set.

4. According to claim 1, a method for intelligent visual inspection of a launch station based on digital twins is characterized in that: According to the device attributes and the location coordinates of the device trigger area, the inspection navigation point coordinates are obtained and the inspection navigation point array is obtained, which includes: Traverse all inspection devices. If the inspection is true in the device's properties, obtain the location range of the device's trigger area, obtain the coordinates of the middle position of the area as the inspection navigation point of the device, and add it to the inspection navigation point array.

5. According to claim 1, a method for intelligent visual inspection of a launch station based on digital twins is characterized in that: The method further includes: creating an inspection AI based on the inspection personnel model, specifically: Create a behavior tree based on the navigation component. The behavior tree controls the inspection model to move to navigation points one by one.

6. According to claim 4, a method for intelligent visual inspection of a launch station based on digital twins is characterized in that: The method further comprises: Use the navigation component to create a navigation area, implement an automatic path-finding system, and place the created inspection personnel model into the navigation area to complete automatic inspection.

7. A launch station intelligent visualization inspection system based on digital twins, characterized in that: The system comprises: A three-dimensional station inspection scene model construction module is used to perform three-dimensional modeling on the components of the station inspection scene, and to construct a three-dimensional station inspection scene model based on the three-dimensional model of the components, wherein the three-dimensional model of the components includes a building model, an equipment model and an inspection personnel model; A navigable inspection equipment establishment module is used to create a trigger area for the equipment model and add equipment attributes. An overlapping event is added according to the created trigger area. When the inspection personnel model enters the area, the equipment operation information is automatically obtained and an information page is generated. Inspection route navigation point generation module, used to obtain inspection route navigation point coordinates and obtain inspection route navigation point array according to device attributes and location coordinates of device trigger area, and automatically generate inspection route navigation points; The automatic inspection module is used to control the inspection personnel model to move to each navigation point one by one based on the inspection route navigation points, read the real-time displayed equipment operation information data, and complete the automatic inspection of the station; Constructing a three-dimensional station inspection scene model based on the three-dimensional model of the elements, specifically including: importing the model files of each element into the UE4 engine, and constructing a three-dimensional station inspection scene model based on each element; Creating a trigger area for the device model and adding device attributes, specifically including: creating a box trigger area, the length, width and height of the area are respectively equal to the length, width and height of the inspection personnel model; the device attributes include device ID and whether to inspect; Automatically obtain equipment operation information and generate information pages, specifically including: when the inspection personnel model enters the trigger area, send a device operation information request containing the device ID to the Web server interface. After receiving the request, the Web server accesses the real remote device API to obtain the equipment operation information, and after receiving the device return information, in the UE, use the UE's built-in browser to render the information page in the client software.

Images