A three-dimensional visualization system and method for equipment failure

Abstract

The present invention discloses a three-dimensional visualization system and method for equipment failure, and relates to the field of visualization technology. The system utilizes radio frequency to collect the internal vibration, structure and associated spatial information of the equipment, the internal vibration, structure and associated spatial information of the equipment, and a prefabricated internal structure model diagram of a healthy equipment to establish a three-dimensional model of the equipment, the actual position information of the internal structure and parts of the equipment, establish a real-time mapping relationship between the internal vibration, structure and associated spatial information of the equipment and the internal three-dimensional model of the equipment, and the real-time mapping relationship displays the three-dimensional model of the equipment.

Description

Technical Field

[0001] The invention discloses a visualization system and method, and relates to the field of visualization technology, in particular to a three-dimensional visualization system and method for equipment failure based on radio frequency signals and augmented reality. Background Art

[0002] In addition to the external shape and functional layout of the equipment, its internal structure often realizes its specific functions. Especially for large and complex equipment, the internal structure is its real essence.

[0003] Many devices, such as large industrial equipment, have complex internal structures, and it is difficult to accurately locate the specific location where the risk of failure exists. This requires a lot of manpower and material resources to verify, and consumes a lot of operation and maintenance costs. Summary of the invention

[0004] In view of the problems of the prior art, the present invention provides a three-dimensional visualization system and method for equipment failure, which is easy to deploy and has high data collection accuracy. A three-dimensional visualization model can be constructed for the specific internal structure of the equipment, and the time and location of equipment failure can be accurately predicted, thereby minimizing maintenance costs and increasing equipment life.

[0005] The specific scheme proposed by the present invention is:

[0006] A 3D visualization method for equipment failures uses radio frequency to collect the internal vibration, structure and related spatial information of the equipment.

[0007] The three-dimensional model of the interior of the device is established by using augmented reality technology through the internal vibration, structure and related spatial information of the device and the prefabricated internal structure model of the health device.

[0008] Identify the actual location information of the internal structure and parts of the equipment, and establish a real-time mapping relationship between the internal vibration, structure and associated spatial information of the equipment and the internal 3D model of the equipment.

[0009] Use real-time mapping to display the 3D model inside the device.

[0010] A three-dimensional visualization system for equipment failures includes a radio frequency capture module, a three-dimensional modeling module, a mapping module, and a display module.

[0011] The RF capture module uses RF to collect the internal vibration, structure and related spatial information of the device.

[0012] The 3D modeling module uses augmented reality technology to build a 3D model of the inside of the device through the internal vibration, structure and associated spatial information of the device and the prefabricated internal structure model diagram of the health device.

[0013] The mapping module identifies the actual location information of the internal structure and parts of the equipment, and establishes a real-time mapping relationship between the actual location information of the internal structure and parts of the equipment and the internal three-dimensional model of the equipment.

[0014] The display module uses real-time mapping to display the three-dimensional model inside the device.

[0015] The radio frequency capture module in the three-dimensional visualization system for equipment failure includes a multi-channel antenna system, which makes the reflection signal of the static object zero and tracks the moving object by using the inverse SAR technology.

[0016] The radio frequency capture module in the three-dimensional visualization system for equipment failure also includes a retrieval module for retrieving the three-dimensional model, obtaining the area in the three-dimensional model mapped to the internal structure of the equipment and the actual position information of the parts, and obtaining the internal vibration, structure and associated spatial information of the equipment corresponding to the part.

[0017] The three-dimensional modeling module in the three-dimensional visualization system for equipment failure includes an image acquisition module for receiving the internal vibration, structure and associated spatial information of the equipment collected by the radio frequency capture module, as well as the internal structure model diagram of the healthy equipment.

[0018] The three-dimensional modeling module in the three-dimensional visualization system for equipment failure includes a modeling module for generating a corresponding three-dimensional model of the inside of the equipment using a relative coordinate system and a plane according to information received by the image acquisition module.

[0019] A three-dimensional visualization device for equipment failure, comprising: at least one memory and at least one processor;

[0020] The at least one memory is used to store a machine-readable program;

[0021] The at least one processor is used to call the machine-readable program to execute the three-dimensional visualization method for equipment failure.

[0022] A computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the processor is caused to execute the three-dimensional visualization method for equipment failure.

[0023] The benefits of the present invention are:

[0024] The present invention provides a three-dimensional visualization system and method for equipment failure, which does not require the addition of additional sensors. The same data collection method can be used for equipment of different models, and the deployment is convenient. The collected data is more accurate and reliable, and the prediction of the equipment location and time requiring maintenance is more accurate. By utilizing augmented reality technology, the three-dimensional model inside the equipment is displayed, which facilitates maintenance personnel to more conveniently identify the cause of the failure and find the location of the faulty parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the interaction between the various parts of the system of the present invention;

[0026] Figure 2 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] Three-dimensional augmented reality technology is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing. It simulates computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and applies it to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.

[0028] Wi-Fi is wireless network access, the most widely used wireless network transmission technology today. Wi-Fi signals are usually the information carrier between the transmitter and the receiver. They can capture the radio frequency signals generated by the internal structure through the device shell and convert the signals into images as input data for the early warning management system.

[0029] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0030] The present invention provides a three-dimensional visualization system for equipment failure, including a radio frequency capture module, a three-dimensional modeling module, a mapping module and a display module.

[0031] The RF capture module uses RF to collect the internal vibration, structure and related spatial information of the device.

[0032] The 3D modeling module uses augmented reality technology to build a 3D model of the inside of the device through the internal vibration, structure and associated spatial information of the device and the prefabricated internal structure model diagram of the health device.

[0033] The mapping module identifies the actual location information of the internal structure and parts of the equipment, and establishes a real-time mapping relationship between the actual location information of the internal structure and parts of the equipment and the internal three-dimensional model of the equipment.

[0034] The display module uses real-time mapping to display the three-dimensional model inside the device.

[0035] The radio frequency capture module of the system of the present invention does not need to add additional sensors. The same data collection method can be used for different types of equipment. The deployment is convenient, and the collected data is more accurate and reliable. The three-dimensional modeling module uses augmented reality technology to display the three-dimensional model inside the equipment, and the prediction of the equipment part and time that need maintenance is more accurate, which is convenient for maintenance personnel to more conveniently identify the cause of the fault and find the location of the faulty parts.

[0036] In one embodiment of the system of the present invention, the RF capture module can be a 2.4GHz ISM band Wi-Fi signal MIMO system with two transmitting antennas and one receiving antenna, using signals in the Wi-Fi OFDM ISM band (2.4GHz) and typical Wi-Fi hardware, eliminating the flash reflected from the shell by performing MIMO zeroing scoring, and utilizing inverse SAR (synthetic aperture radar) technology to treat the object itself as an antenna array to track moving objects.

[0037] It can be understood that when collecting the spatial information of the device, in the first stage, the channel from each of the two transmitting antennas connected to its receiving antenna is measured. In the second stage, the two transmitting antennas use the channel measurement values ​​of the first stage to zero the signal at the receiving antenna. Since the wireless signal (including reflections) is linearly combined on the medium, only the reflections of objects moving between the two stages are captured in the second stage, and the reflections for static objects (including walls) will be zero. Eliminating interference to unnecessary receivers can also be used to eliminate reflections from static objects (including device housings and static objects inside the housing). Thereby more accurately obtaining the internal vibration, structure and associated spatial information of the device.

[0038] In another embodiment of the system of the present invention, the radio frequency capture module further includes a retrieval module for retrieving the three-dimensional model, obtaining the area in the three-dimensional model mapped with the actual position information, and obtaining the spatial information corresponding to the part.

[0039] The three-dimensional modeling module includes: an image acquisition module, which is used to receive the internal vibration, structure and associated spatial information of the device collected by the radio frequency capture module, as well as the internal structure model diagram of the health device; a modeling module, which is used to generate a corresponding model in the coordinate system based on the information received by the image acquisition unit, the internal structure model diagram of the health device, the relative coordinate system and the plane, and modeling is performed according to the generated model. Based on the principle of augmented reality, the three-dimensional modeling of the internal frame of the entire device is sent to the display module for display in the area to be displayed.

[0040] Taking coal mine ventilation equipment as an example, the vibration frequency of the internal structure of the equipment will be slightly different at different times, so the RF signal captured by the RF signal capture device on the outside of the coal mine ventilation equipment is also different, and the image presented is also different. The internal vibration, structure and associated spatial information of the equipment are different, so the faulty parts will be displayed on the 3D model through augmented reality technology. The model is deployed on the server in the production environment, and the RF signal capture device is used to connect to the server for real-time detection. Once a fault occurs, the 3D model diagram can be displayed in the display area, and the parts that may fail are marked. The specific process is as follows:

[0041] S1: Place the RF capture module outside the coal mine ventilation equipment to collect real-time working condition data of the coal mine ventilation equipment and collect the internal vibration, structure and associated spatial information of the equipment;

[0042] S2: The RF capture module transmits the internal vibration, structure and associated spatial information of the device to the mapping module via WiFi, and transmits the internal structure model diagram of the health device to the mapping module, and the mapping module maps the spatial positions of the internal parts;

[0043] S3: The radio frequency capture module transmits the internal vibration, structure and related spatial information of the device to the image acquisition unit of the 3D modeling module through WiFi, and the modeling unit uses augmented reality technology to perform 3D modeling;

[0044] S4: The modeling module transmits the 3D modeling information to the display module via Wi-Fi. The display module displays the augmented reality image in proportion in the display area, with emphasis on marking the parts that may fail.

[0045] The above-mentioned embodiment clarifies the functional modules inside the radio frequency capture module and the three-dimensional modeling module, making the visualization process inside the device clearer.

[0046] The present invention provides a three-dimensional visualization method for equipment failure, which uses radio frequency to collect the internal vibration, structure and related spatial information of the equipment.

[0047] The three-dimensional model of the interior of the device is established by using augmented reality technology through the internal vibration, structure and related spatial information of the device and the prefabricated internal structure model of the health device.

[0048] Identify the actual location information of the internal structure and parts of the equipment, and establish a real-time mapping relationship between the internal vibration, structure and associated spatial information of the equipment and the internal 3D model of the equipment.

[0049] Use real-time mapping to display the 3D model inside the device.

[0050] The method of the present invention utilizes real-time mapping relationships to collect actual position information of the parts at a certain location, such as image information, in real time, and identify whether the image contains the parts at that location. When identifying the included parts, the spatial information associated with the corresponding parts is sent according to the mapping relationship.

[0051] Taking coal mine ventilation equipment as an example, the vibration frequency of the internal structure of the equipment will be slightly different at different times, so the RF signal captured by the RF signal capture device on the outside of the coal mine ventilation equipment is also different, and the images presented are also different. The internal vibration, structure and related spatial information of the equipment are different, so the faulty parts will be displayed on the 3D model through augmented reality technology. The model is deployed on the server in the production environment, and the RF signal capture device is used to connect to the server for real-time detection. Once a fault occurs, the 3D model diagram can be displayed in the display area, and the parts that may fail are marked.

[0052] The above-mentioned information interaction, execution process and other contents are based on the same concept as the system embodiment of the present invention. For specific contents, please refer to the description in the system embodiment of the present invention and will not be repeated here.

[0053] In addition to the above method and system, the present invention also provides a three-dimensional visualization device for equipment failure, comprising: at least one memory and at least one processor;

[0054] The at least one memory is used to store a machine-readable program;

[0055] The at least one processor is used to call the machine-readable program to execute the three-dimensional visualization method for equipment failure.

[0056] A computer-readable medium is also provided, on which a computer instruction is stored, and when the computer instruction is executed by a processor, the processor executes the three-dimensional visualization method of equipment failure. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code for implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device is enabled to read and execute the program code stored in the storage medium.

[0057] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0058] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0059] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0060] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0061] It should be noted that not all steps and modules in the above-mentioned processes and system structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0062] In the above embodiments, the hardware unit can be realized by mechanical means or electrical means. For example, a hardware unit can include permanent dedicated circuits or logic (such as special processors, FPGA or ASIC) to complete the corresponding operation. The hardware unit can also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set by software to complete the corresponding operation. Concrete implementation (mechanical means or dedicated permanent circuits or temporarily set circuits) can be determined based on cost and time considerations.

[0063] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A three-dimensional visualization method for equipment failure, characterized by Radio frequency is used to collect the internal vibration, structure and related spatial information of the equipment. A multi-channel antenna system with two transmitting antennas and one receiving antenna is used to make the reflected signal of static objects zero, and inverse SAR technology is used to track moving objects. The three-dimensional model is also retrieved to obtain the area in the three-dimensional model mapped with the actual position information of the internal structure and parts of the equipment, and the internal vibration, structure and related spatial information of the equipment corresponding to the part is obtained. The three-dimensional model of the inside of the device is established by using augmented reality technology through the internal vibration, structure and related spatial information of the device and the pre-made internal structure model diagram of the healthy device. Image acquisition is performed to receive the internal vibration, structure and related spatial information of the device collected by radio frequency, as well as the internal structure model diagram of the healthy device. According to the information received by the image acquisition, the relative coordinate system and plane are used to generate the corresponding three-dimensional model of the inside of the device. Identify the actual location information of the internal structure and parts of the equipment, and establish a real-time mapping relationship between the internal vibration, structure and associated spatial information of the equipment and the internal 3D model of the equipment. Use real-time mapping to display the 3D model inside the device; For coal mine ventilation equipment, augmented reality technology is used to display it on a three-dimensional model. The specific process is as follows: S1: Collect real-time working condition data of coal mine ventilation equipment, and collect internal vibration, structure and related spatial information of the equipment; S2: Map the spatial positions of internal parts based on the internal vibration, structure and associated spatial information of the device and the internal structure model diagram of the health device; S3: 3D modeling using augmented reality technology based on the internal vibration, structure and associated spatial information of the equipment; S4: Display the 3D modeling information in the area to be displayed in an augmented reality image in proportion, with emphasis on marking the parts that may fail.

2. A three-dimensional visualization system for equipment failure, characterized by It includes a radio frequency capture module, a three-dimensional modeling module, a mapping module and a display module. The radio frequency capture module includes a multi-channel antenna system. The radio frequency capture module has two transmitting antennas and one receiving antenna, which makes the reflection signal of the static object zero and uses the inverse SAR technology to track the moving object. The radio frequency capture module uses radio frequency to collect the internal vibration, structure and related spatial information of the equipment. The radio frequency capture module also includes a retrieval module for retrieving the three-dimensional model, obtaining the area in the three-dimensional model mapped with the actual position information of the internal structure and parts of the equipment, and obtaining the internal vibration, structure and related spatial information of the equipment corresponding to the part. The three-dimensional modeling module uses augmented reality technology to establish a three-dimensional model of the inside of the device through the internal vibration, structure and associated spatial information of the device and the prefabricated internal structure model diagram of the health device. The three-dimensional modeling module includes an image acquisition module for receiving the internal vibration, structure and associated spatial information of the device and the internal structure model diagram of the health device acquired by the radio frequency capture module. The three-dimensional modeling module includes a modeling module for generating a corresponding three-dimensional model of the inside of the device using a relative coordinate system and a plane according to the information received by the image acquisition module. The mapping module identifies the actual location information of the internal structure and parts of the equipment, and establishes a real-time mapping relationship between the actual location information of the internal structure and parts of the equipment and the internal three-dimensional model of the equipment. The display module uses real-time mapping to display the 3D model inside the device. For coal mine ventilation equipment, the equipment fault three-dimensional visualization system is used to display it on a three-dimensional model through augmented reality technology. The specific process is as follows: S1: Place the RF capture module outside the coal mine ventilation equipment to collect real-time working condition data of the coal mine ventilation equipment and collect the internal vibration, structure and associated spatial information of the equipment; S2: The RF capture module transmits the internal vibration, structure and associated spatial information of the device to the mapping module via WiFi, and transmits the internal structure model diagram of the health device to the mapping module, and the mapping module maps the spatial positions of the internal parts; S3: The radio frequency capture module transmits the internal vibration, structure and related spatial information of the device to the image acquisition unit of the 3D modeling module through WiFi, and the modeling unit uses augmented reality technology to perform 3D modeling; S4: The modeling module transmits the 3D modeling information to the display module via Wi-Fi. The display module displays the augmented reality image in proportion in the display area, with emphasis on marking the parts that may fail.

3. A three-dimensional visualization device for equipment failure, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute the three-dimensional visualization method for equipment failure described in claim 1.

4. A computer-readable medium, characterized in that The computer-readable medium stores computer instructions, and when the computer instructions are executed by a processor, the processor executes the three-dimensional visualization method for equipment failure according to claim 1.

Images