Inspection method and equipment based on virtual reality mapping and storage medium

Abstract

The invention discloses an inspection method based on virtual reality mapping. The inspection method comprises the following steps: S1, constructing a three-dimensional space rectangular coordinate system; s2, manufacturing a virtual three-dimensional model; s3, importing the manufactured virtual three-dimensional model into a super-reality device, wherein the super-reality device is used for acquiring a real scene; s4, marking a projection position, calculating shooting coordinates of the real scene acquired by the super-reality equipment, calculating world coordinates of the super-reality equipment, marking feature information of the real scene, and calculating a conversion matrix of the shooting coordinates and the world coordinates according to the feature information so as to calculate the world coordinates of the real scene; and S5, enabling the virtual three-dimensional model to appear at the correct position of the real scene according to the position relationship between the real scene and the super-reality equipment, which is detected in real time. According to the invention, real-time and high-precision space positioning and environment perception can be realized, and the problem of alignment of virtual content and a real scene is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to an inspection method, device and storage medium based on virtual reality mapping. Background Art

[0002] At present, with the popularization of Internet of Things technology, virtual reality technology based on the Internet of Things has been widely used. This technology wirelessly transmits complex models and environmental data to VR glasses, allowing users to immerse themselves in highly simulated real scenes for interaction and operation. In areas such as facility inspection and distance education, the integration of VR technology has greatly enhanced the realism of the experience and operational efficiency, demonstrating the huge potential of the integration of the Internet of Things and virtual reality.

[0003] However, the current VR-based inspection application process still has the following problems: it relies on preset models and users manually adjusting glasses for spatial positioning, which is sensitive to environmental conditions. Specifically, changes in environmental factors, such as light intensity and interference from surrounding obstacles, may cause inaccurate positioning, resulting in positioning drift and precision errors during the inspection process.

[0004] This positioning instability caused by external environmental fluctuations limits the effective application of VR technology in precision inspections and other high-precision scenarios.

[0005] To this end, this application specifically proposes an inspection method based on virtual reality mapping to solve the above technical problems. Summary of the invention

[0006] The main purpose of the present invention is to provide an inspection method based on virtual reality mapping to solve the technical problems raised in the background technology.

[0007] The present invention adopts the following technical solutions to solve the above technical problems:

[0008] A virtual reality mapping-based inspection method includes the following specific operation steps:

[0009] S1. Construct a three-dimensional rectangular coordinate system;

[0010] S2. Create a virtual three-dimensional model, wherein the virtual three-dimensional model includes a ground model and a building model of the area to be projected;

[0011] S3. Importing the prepared virtual three-dimensional model into a super-reality device, the super-reality device is used to obtain a real scene;

[0012] S4. Mark the projection position, calculate the shooting coordinates of the real scene obtained by the super reality device, calculate the world coordinates of the super reality device, mark the feature information of the real scene, and calculate the conversion matrix of the shooting coordinates and the world coordinates according to the feature information for calculating the world coordinates of the real scene;

[0013] S5. Based on the positional relationship between the real scene and the super-reality device detected in real time, the virtual three-dimensional model appears in the correct position of the real scene;

[0014] S6. Build a virtual sandbox and store the virtual three-dimensional model and the coordinates of the hyper-reality device in the virtual sandbox to quickly reconstruct the positional relationship between the real scene and the hyper-reality device and perform auxiliary inspection operations.

[0015] Preferably, in step S2, the angle between each two edges in the ground model and the building model is the same as the angle between the corresponding two edges in the design drawing or the real space, and the length of each edge in the building model and the ground model is in a 1:1 ratio with the length of the corresponding edge in the design drawing or the real space.

[0016] Preferably, the super reality device in step S3 is composed of a Beidou positioning module, a gyroscope, an accelerometer, a magnetometer, an infrared sensor, a camera, and a holographic display screen, wherein:

[0017] The Beidou positioning module includes a chipset including an RF radio frequency chip, a baseband chip and a microprocessor. The Beidou positioning module is used to receive Beidou and GPS signals and calculate the three-dimensional position of the super reality device in real time;

[0018] The gyroscope and accelerometer are used to detect the position and orientation of the device;

[0019] The magnetometer is used to measure the angles between the device and the four directions of east, south, west and north;

[0020] The infrared sensor is used to detect the distance and position of the device;

[0021] The camera is used to shoot real scenes in real time. The camera shooting coordinates are the device shooting coordinates, and the camera shooting coordinates are a three-dimensional space coordinate system with the origin of the coordinates as the origin of the lens, the positive direction of the X axis is to the right, the Z axis is forward, which is the direction facing the lens, and the positive direction of the Y axis is upward, which is determined by the right-hand rule;

[0022] The holographic display screen is used to display the mapping of the virtual three-dimensional model superimposed on the real scene in real time.

[0023] Preferably, the specific operation steps of calculating the conversion matrix of the shooting coordinates and the world coordinates according to the feature information in step S4 for calculating the world coordinates of the real scene include:

[0024] S41. The roll angle, yaw angle, and pitch angle of the device during the preset shooting process are γ, β, and α respectively. There is a point P with coordinates in the world coordinate system of (X W , Y W , Z W ), the coordinates in the camera coordinate system are (X C , Y C , Z C ), then the rotation matrix R = JR1R2R3, where J is the attitude correction matrix, we have:

[0025]

[0026] Among them, j1, j2, and j3 are the attitude correction vector eigenvalues ​​of the roll angle, yaw angle, and pitch angle, respectively, and are averaged through multiple simulation experiments of the device;

[0027] S42. The translation matrix T can be obtained according to the rotation matrix R. Since the camera coordinate system and the world coordinate system are transformed through the camera's external parameter matrix, there is the following association rule between the rotation matrix R and the translation matrix T:

[0028]

[0029] S43. The points in the camera coordinate system can be transformed into the world coordinate system through the rotation matrix R and the translation matrix T, thereby obtaining the world coordinates of the real scene, which is expressed as:

[0030] P W =R -1 *(P c -T)

[0031] Among them, P c Represents the point in the camera coordinate system, P W Represents a point in the world coordinate system.

[0032] Preferably, in the real scene feature information: an infrared sensor is used to calculate the position and direction of the super display device relative to the detected area, the position of the super reality device in the three-dimensional rectangular coordinate system of space is calculated by the Beidou positioning module, and then the position and direction of the detected area are determined by calculating the transformation matrix of the three-dimensional rectangular coordinate system and the camera coordinate system, thereby realizing the positioning of the real scene.

[0033] Preferably, when making the virtual three-dimensional model appear at the correct position of the real scene in step S5, it is also necessary to determine the coordinate position of the three-dimensional model in the coordinate system corresponding to each viewing angle of the hyper-reality device according to the virtual three-dimensional model and the three-dimensional rectangular coordinates, including:

[0034] S51. According to the virtual three-dimensional model, determining the coordinate position of the feature information of the real scene in the coordinate system corresponding to the perspective of the super reality device;

[0035] S52 determines the transformation matrix of each viewing angle according to the same feature information number of the real scene and the collinear equation, and unifies them into a three-dimensional rectangular coordinate system.

[0036] Preferably, the specific operation process in step S52 includes:

[0037] L1. Based on the known image plane coordinates, world coordinates, and internal orientation elements of the control point, the six external orientation elements are solved using the collinear equation based on the principle of single image space resection, which are the coordinates of the photography center (X S , Y S , Z S ) and the attitude angle collinearity condition equation image point coordinate error equation is ( ω, μ), the collinearity condition equation image point coordinates are:

[0038]

[0039] Where (X, Y, Z) is the object point coordinates in the object space; (x, y) is the image point coordinates; (x0, y0) is the principal point coordinates; f is the principal distance of the image, a is i , b i , c i (i=1, 2, 3) are the 9 direction cosines composed of the 3 exterior azimuth elements of the image;

[0040] L2. The collinearity equation determines the strict relationship between the photographic center point, the image point and the object point. At this time, for a pair of points with the same name on the left and right images, if n images contain the same spatial point, the object point's object space coordinate value can be solved by the collinearity condition equation and unified into a three-dimensional rectangular coordinate system.

[0041] Preferably, in the step S6, after solving the object point and object space coordinate values ​​each time and unifying them into a three-dimensional rectangular coordinate system, a virtual sand table is set and mapped into the virtual sand table according to the coordinate values, and the specific steps include:

[0042] S61. According to the loaded three-dimensional virtual model, determine the minimum coordinate value x of the model boundary coordinate value on the three-dimensional vector min ,y min and z min , then adjust the coordinate center point of the virtual sandbox to (x min ,y min , z min );

[0043] S62. Specify the maximum value K of the three-dimensional coordinates of the virtual sandbox boundary;

[0044] S63. After each solution of the object point's object space coordinate value, according to the maximum value E of the newly entered coordinate data;

[0045] S64. If there is E - x min > K or E - y min > K or E - z min > K, then determine the sand table coordinate adjustment ratio Then update the iteration rule of the coordinate value setting in the three - dimensional rectangular coordinate system and iterate all coordinates in the virtual sand table:

[0046]

[0047] where i is the number of iterations;

[0048] S65. If there is E - x min < K and E - y min < K and E - z min < K, then determine the maximum coordinate values x max 、y max and z max , there is a sand table coordinate adjustment ratio After that, update K = MAX(x max , y max , z max ), then update the iteration rule of the coordinate value setting in the three - dimensional rectangular coordinate system and iterate all coordinates in the virtual sand table:

[0049]

[0050] where i is the number of iterations;

[0051] S66. During the continuous mapping of the coordinate value adjustment to the virtual sand table, loop the above steps S61 - S65.

[0052] On the other hand, the present invention also discloses a computer - readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the above - mentioned method.

[0053] On yet another aspect, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the above - mentioned method.

[0054] As can be seen from the above technical solutions, the present invention provides a patrol inspection method based on virtual reality mapping. Compared with the prior art, the present invention has the following advantages:

[0055] 1. The present invention integrates a variety of high-precision sensors such as the Beidou positioning module in the hyper-reality device, which can achieve real-time, high-precision spatial positioning and environmental perception, effectively solving the problem of aligning virtual content with real scenes, allowing inspection personnel to intuitively view the status of facilities in an intuitive holographic display interface, perform accurate fault detection and evaluation, and greatly improve the efficiency and quality of inspection work.

[0056] 2. The present invention integrates Beidou positioning, multiple motion sensors, and advanced vision and display technologies into super-reality equipment to achieve precise integration of virtual models and real environments, which can significantly improve the efficiency and accuracy of inspection operations.

[0057] 3. The present invention uses precise spatial positioning and posture perception, combined with real scene images captured by the camera and virtual three-dimensional models, and uses complex mathematical models and algorithm processing to accurately map the virtual model to the real scene, ensuring seamless connection between the virtual model and the physical space, thereby improving the accuracy of inspection operations.

[0058] 4. By constructing a virtual sandbox, the present invention can quickly reconstruct the digital position relationship information between the real scene and the super-realistic equipment, making the inspection operation intelligent and efficient, which not only reduces the labor cost but also improves the safety management level.

[0059] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easy to understand through the following description. Of course, it is not necessary to achieve all of the advantages described above simultaneously for any product implementing the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0061] Figure 1 It is an overall schematic diagram of the process of the method of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. In the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] In the embodiment, see Figure 1 .

[0064] In the embodiment, see Figure 1 .

[0065] like Figure 1 As shown. The inspection method based on virtual reality mapping proposed in the embodiment of the present invention can realize real-time, high-precision spatial positioning and environmental perception by integrating a variety of high-precision sensors such as the Beidou positioning module in the hyper-reality device, effectively solving the problem of aligning virtual content with the real scene, so that the inspection personnel can intuitively view the facility status in an intuitive holographic display interface, perform accurate fault detection and evaluation, and greatly improve the efficiency and quality of the inspection work.

[0066] The method includes the following specific steps:

[0067] S1. Construct a three-dimensional rectangular coordinate system;

[0068] The specific construction method is: take a certain plane as the origin, the extension line of the intersection of the three planes as the coordinate axis, the XY plane is the marking plane, the Z axis is parallel to the mark and upward, and the right-hand rule is used as the basis to determine the direction of the X and Y axes;

[0069] At this time, the world coordinate system uses the center of the square mark as the world origin, and the vertical mark plane outward as the positive direction of the Z axis;

[0070] S2. Create a virtual three-dimensional model, which includes a ground model and a building model of the area to be projected;

[0071] The angle between each two sides in the ground model and the building model is the same as the angle between the corresponding two sides in the design drawing or the real space, and the length of each side in the building model and the ground model is in a 1:1 ratio with the length of the corresponding side in the design drawing or the real space;

[0072] S3. Import the prepared virtual 3D model into the hyper-reality device, where the hyper-reality device is used to obtain the real scene and is composed of a Beidou positioning module, a gyroscope, an accelerometer, a magnetometer, an infrared sensor, a camera, a holographic display screen, etc., wherein:

[0073] The Beidou positioning module includes a chipset consisting of an RF radio frequency chip, a baseband chip and a microprocessor. The Beidou positioning module is used to receive Beidou and GPS signals through the chip and calculate the three-dimensional position of the hyper-reality device in real time;

[0074] Gyroscopes and accelerometers are used to detect the position and orientation of the device;

[0075] The magnetometer is used to measure the angle between the device and the four directions of east, south, west and north;

[0076] Infrared sensors are used to detect the distance and position of the device;

[0077] The camera is used to capture real scenes in real time. The camera shooting coordinates are the device shooting coordinates, and the camera shooting coordinates are a three-dimensional space coordinate system with the origin of the coordinate system as the origin of the lens. The positive direction of the X axis is to the right, the Z axis is forward, which is the direction facing the lens, and the positive direction of the Y axis is upward, which is determined by the right-hand rule.

[0078] The holographic display screen is used to display the mapping of the virtual three-dimensional model superimposed on the real scene in real time;

[0079] In the feature information of the real scene, the use effect of the components of the super reality device is: using the infrared sensor to calculate the position and direction of the super reality device with respect to the detected area, using the Beidou positioning module to calculate the position of the super reality device in the three-dimensional rectangular coordinate system of space, and then calculating the conversion matrix between the three-dimensional rectangular coordinate system and the camera coordinate system to determine the position and direction of the detected area, thereby realizing the positioning of the real scene;

[0080] In summary, a three-dimensional rectangular coordinate system is constructed, and then a virtual three-dimensional model that is highly matched with the actual environment is created to ensure that the geometric size and orientation of the model are consistent with the real world. At this time, by integrating Beidou positioning, multiple motion sensors, and advanced vision and display technologies into the super-reality device, the virtual model and the real environment are accurately integrated, which can significantly improve the efficiency and accuracy of the inspection operation.

[0081] S4. Mark the projection position, calculate the shooting coordinates of the super reality device to obtain the real scene, calculate the world coordinates of the super reality device, mark the feature information of the real scene, calculate the conversion matrix of the shooting coordinates and the world coordinates according to the feature information, and calculate the world coordinates of the real scene according to the conversion matrix;

[0082] Specifically, the specific operation steps of calculating the conversion matrix of the shooting coordinates and the world coordinates according to the feature information and using it to calculate the world coordinates of the real scene include:

[0083] S41. The roll angle, yaw angle, and pitch angle of the device during the preset shooting process are γ, β, and α respectively. There is a point P with coordinates in the world coordinate system of (X W , Y W , Z W ), the coordinates in the camera coordinate system are (X C , Y C , Z C ), then the rotation matrix R = JR1R2R3, where J is the attitude correction matrix, we have:

[0084]

[0085] Among them, j1, j2, and j3 are the attitude correction vector eigenvalues ​​of the roll angle, yaw angle, and pitch angle, respectively, and are averaged through multiple simulation experiments of the device;

[0086] S42. According to the rotation matrix R, a 3×1 translation matrix T can be obtained. Since the camera coordinate system and the world coordinate system are transformed through the camera's external parameter matrix, there is the following association rule between the rotation matrix R and the translation matrix T:

[0087]

[0088] S43. The points in the camera coordinate system can be transformed into the world coordinate system through the rotation matrix R and the translation matrix T, thereby obtaining the world coordinates of the real scene, which is expressed as:

[0089] P W =R -1 *(P c -T)

[0090] Among them, P c Represents the point in the camera coordinate system, P W Represents a point in the world coordinate system;

[0091] S5. Based on the positional relationship between the real scene and the super-reality device detected in real time, the virtual three-dimensional model appears in the correct position of the real scene;

[0092] Specifically, when making the virtual three-dimensional model appear at the correct position of the real scene, it is also necessary to determine the coordinate position of the three-dimensional model in the coordinate system corresponding to each viewing angle of the hyper-reality device according to the virtual three-dimensional model and the three-dimensional rectangular coordinates, including:

[0093] S51. According to the virtual three-dimensional model, determine the coordinate position of the feature information of the real scene in the coordinate system corresponding to the perspective of the super reality device;

[0094] S52 determines the transformation matrix of each viewing angle according to the feature information number of the same real scene and the collinear equation, and unifies them into a three-dimensional rectangular coordinate system:

[0095] The specific operation process at this time includes:

[0096] L1. Based on the known image plane coordinates, world coordinates, and internal orientation elements of the control point, the six external orientation elements are solved using the collinear equation based on the principle of single image space resection, which are the coordinates of the photography center (X S , Y S , Z S) and the attitude angle collinearity condition equation image point coordinate error equation is ( ω, μ), the collinearity condition equation image point coordinates are:

[0097]

[0098] Where (X, Y, Z) is the object point coordinates in the object space; (x, y) is the image point coordinates; (x0, y0) is the principal point coordinates; f is the principal distance of the image, a is i , b i , c i (i=1, 2, 3) are the 9 direction cosines composed of the 3 exterior azimuth elements of the image;

[0099] L2. The collinearity equation determines the strict relationship between the photographic center point, the image point, and the object point. From the collinearity equation, we can get:

[0100] (x-x0)[a3(X A -X S )+b3(Y A -Y S )+c3(Z A -Z S )]=-f[a1(XX S )+b1(YY S )+c1(ZZ S )]

[0101] (y-y0)[a3(X A -X S )+b3(Y A -Y S )+c3(Z A -Z S )]=-f[a2(XX S )+b2(YY s )+c2(ZZ s )]

[0102] After finishing, we can get:

[0103]

[0104] in:

[0105] l1=fa1+(x-x0)a3

[0106] l2=fb1+(x-x0)b3

[0107] l3=fc1+(x-x0)c3

[0108] l x =fa1X S +fb1YS +fc1Z S +(x-x0)a3X S +(x-x0)b3Y S +(x-x0)c3Z S

[0109] l4=fa2+(y-y0)a2

[0110] l5=fb2+(y-y0)b2

[0111] l6=fc2+(y-y0)c2

[0112] l y =fa2X S +fb2Y S +fc2Z S +(y-y0)a3X S +(y-y0)b3Y S +(y-y0)c3Z S

[0113] At this time, for a pair of points of the same name on the left and right images, four of the above linear equations can be listed, and the number of unknowns is 3, so the least squares method can be used to solve it;

[0114] If n images contain the same spatial point, the three unknown numbers X, Y, and Z can be solved by a total of 2n linear equations as the spatial coordinate values ​​of the object point and unified into a three-dimensional rectangular coordinate system;

[0115] In summary, through precise spatial positioning and posture perception, combined with the real scene images captured by the camera and the virtual 3D model, and using complex mathematical models and algorithm processing, such as collinear equations and least squares method, high-precision exterior orientation elements are calculated, so that the virtual model can be accurately mapped to the real scene at different viewing angles, ensuring the seamless connection between the virtual model and the physical space, thereby improving the accuracy of the inspection operation;

[0116] S6. Build a virtual sandbox, store the virtual 3D model and the coordinates of the hyper-reality device in the virtual sandbox, and use it to quickly reconstruct the positional relationship between the real scene and the hyper-reality device and perform inspection auxiliary operations;

[0117] Specifically, after solving the object point and object space coordinate values ​​each time and unifying them into a three-dimensional rectangular coordinate system, a virtual sand table is set and mapped into the virtual sand table according to the coordinate values. The specific steps include:

[0118] S61. According to the loaded three-dimensional virtual model, determine the minimum coordinate value x of the model boundary coordinate value on the three-dimensional vector min ,y min and zmin , then adjust the coordinate center point of the virtual sand table to (x min , y min , z min );

[0119] S62. Specify the maximum value K of the three-dimensional coordinates of the virtual sand table boundary;

[0120] S63. After each solution of the object point's object space coordinate value, according to the maximum value E of the newly entered coordinate data;

[0121] S64. If there exists E - x min > K or E - y min > K or E - z min > K, then determine the coordinate adjustment ratio of the sand table Then update the iteration rule of the coordinate values in the three-dimensional rectangular coordinate system and iterate all the coordinates in the virtual sand table:

[0122]

[0123] where i is the number of iterations;

[0124] S65. If there exists E - x min < K and E - y min < K and E - z min < K, then determine the maximum coordinate values x max , y max and z max of the model boundary coordinate values on the three-dimensional vector, there exists a coordinate adjustment ratio of the sand table After that, update K = MAX(x max , y max , z max ), then update the iteration rule of the coordinate values in the three-dimensional rectangular coordinate system and iterate all the coordinates in the virtual sand table:

[0125]

[0126] where i is the number of iterations;

[0127] S66. During the process of continuously mapping the coordinate values to the virtual sand table, loop the above steps S61 - S65;

[0128] In summary, by constructing a virtual sand table and using the equal-proportion mapping storage of the virtual three-dimensional model and the super-real device coordinates to the virtual sand table, the complex digital position relationship information between the real scene and the super-real device can be quickly reconstructed, providing strong support for subsequent inspection planning and auxiliary decision-making, making the inspection operation intelligent and efficient, reducing the labor cost and improving the safety management level.

[0129] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0130] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0131] In another embodiment provided by the present application, a computer program product including instructions is also provided. When the computer program product is executed on a computer, the computer executes any inspection method based on virtual reality mapping in the above embodiments.

[0132] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above method.

[0133] The embodiment of the present application also provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus.

[0134] Memory, used to store computer programs;

[0135] The processor is used to implement the above inspection method based on virtual reality mapping when executing the program stored in the memory.

[0136] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0137] The communication interface is used for communication between the above electronic device and other devices.

[0138] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0139] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0140] It should also be noted that electronic devices also include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal devices.

[0141] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

[0143] In addition, it should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0144] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or schemes that A and B meet at the same time. In addition, in the embodiments of the present invention, "multiple" refers to more than two. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A patrol inspection method based on virtual reality mapping, characterized in that: The specific steps are as follows: S1. Construct a three-dimensional rectangular coordinate system; S2. Create a virtual three-dimensional model, wherein the virtual three-dimensional model includes a ground model and a building model of the area to be projected; S3. Importing the prepared virtual three-dimensional model into a super-reality device, the super-reality device is used to obtain a real scene; S4. Mark the projection position, calculate the shooting coordinates of the real scene obtained by the super reality device, calculate the world coordinates of the super reality device, mark the feature information of the real scene, and calculate the conversion matrix of the shooting coordinates and the world coordinates according to the feature information for calculating the world coordinates of the real scene; S5. Based on the positional relationship between the real scene and the super-reality device detected in real time, the virtual three-dimensional model appears in the correct position of the real scene; S6. Build a virtual sandbox and store the virtual three-dimensional model and the coordinates of the hyper-reality device in the virtual sandbox to quickly reconstruct the positional relationship between the real scene and the hyper-reality device and perform auxiliary inspection operations.

2. The inspection method based on virtual reality mapping as claimed in claim 1, characterized in that: In step S2, the angle between each two sides in the ground model and the building model is the same as the angle between the corresponding two sides in the design drawing or the real space, and the length of each side in the building model and the ground model is in a 1:1 ratio with the length of the corresponding side in the design drawing or the real space.

3. The inspection method based on virtual reality mapping according to claim 1, characterized in that: The super reality device in step S3 is composed of a Beidou positioning module, a gyroscope, an accelerometer, a magnetometer, an infrared sensor, a camera, and a holographic display screen, wherein: The Beidou positioning module includes a chipset including an RF radio frequency chip, a baseband chip and a microprocessor. The Beidou positioning module is used to receive Beidou and GPS signals and calculate the three-dimensional position of the super reality device in real time; The gyroscope and accelerometer are used to detect the position and orientation of the device; The magnetometer is used to measure the angles between the device and the four directions of east, south, west and north; The infrared sensor is used to detect the distance and position of the device; The camera is used to shoot real scenes in real time. The camera shooting coordinates are the device shooting coordinates, and the camera shooting coordinates are a three-dimensional space coordinate system with the origin of the coordinates as the origin of the lens, the positive direction of the X axis is to the right, the Z axis is forward, which is the direction facing the lens, and the positive direction of the Y axis is upward, which is determined by the right-hand rule; The holographic display screen is used to display the mapping of the virtual three-dimensional model superimposed on the real scene in real time.

4. The inspection method based on virtual reality mapping as claimed in claim 3, characterized in that: The specific operation steps of calculating the conversion matrix of the shooting coordinates and the world coordinates according to the feature information in step S4 for calculating the world coordinates of the real scene include: S41. The roll angle, yaw angle, and pitch angle of the device during the preset shooting process are γ, β, and α respectively. There is a point P with coordinates in the world coordinate system of (X W , Y W , Z W ), the coordinates in the camera coordinate system are (X C , Y C , Z C ), then the rotation matrix R = JR1R2R3, where J is the attitude correction matrix, we have: Among them, j1, j2, and j3 are the attitude correction vector eigenvalues ​​of the roll angle, yaw angle, and pitch angle, respectively, and are averaged through multiple simulation experiments of the device; S42. The translation matrix T can be obtained according to the rotation matrix R. Since the camera coordinate system and the world coordinate system are transformed through the camera's external parameter matrix, there is the following association rule between the rotation matrix R and the translation matrix T: S43. The points in the camera coordinate system can be transformed into the world coordinate system through the rotation matrix R and the translation matrix T, thereby obtaining the world coordinates of the real scene, which is expressed as: P W =R -1 *(P c -T) Among them, P c Represents the point in the camera coordinate system, P W Represents a point in the world coordinate system.

5. The inspection method based on virtual reality mapping as claimed in claim 3, characterized in that: In the real scene feature information: an infrared sensor is used to calculate the position and direction of the super display device relative to the detected area, the position of the super reality device in the three-dimensional rectangular coordinate system of space is calculated by the Beidou positioning module, and then the position and direction of the detected area are determined by calculating the transformation matrix of the three-dimensional rectangular coordinate system and the camera coordinate system, thereby realizing the positioning of the real scene.

6. The inspection method based on virtual reality mapping as claimed in claim 5, characterized in that: When making the virtual three-dimensional model appear at the correct position of the real scene in step S5, it is also necessary to determine the coordinate position of the three-dimensional model in the coordinate system corresponding to each viewing angle of the super reality device according to the virtual three-dimensional model and the three-dimensional rectangular coordinates, including: S51. According to the virtual three-dimensional model, determining the coordinate position of the feature information of the real scene in the coordinate system corresponding to the perspective of the super reality device; S52 determines the transformation matrix of each viewing angle according to the same feature information number of the real scene and the collinear equation, and unifies them into a three-dimensional rectangular coordinate system.

7. The inspection method based on virtual reality mapping as claimed in claim 6, characterized in that: The specific operation process in step S52 includes: L1. Based on the known image plane coordinates, world coordinates, and internal orientation elements of the control point, the six external orientation elements are solved using the collinear equation based on the principle of single image space resection, which are the coordinates of the photography center (X S , Y S , Z S ) and the attitude angle collinearity condition equation, the image point coordinate error equation is The coordinates of the collinear condition equation image points are: Where (X, Y, Z) is the object point coordinates in the object space; (x, y) is the image point coordinates; (x0, y0) is the principal point coordinates; f is the principal distance of the image, a is i , b i , c i 9i=1, 2, 3) are 9 direction cosines composed of 3 exterior azimuth elements of the image respectively; L2. The collinearity equation determines the strict relationship between the photographic center point, the image point and the object point. At this time, for a pair of points with the same name on the left and right images, if n images contain the same spatial point, the object point's object space coordinate value can be solved by the collinearity condition equation and unified into a three-dimensional rectangular coordinate system.

8. The inspection method based on virtual reality mapping as claimed in claim 7, characterized in that: In the step S6, after solving the object point and object space coordinate values ​​each time and unifying them into a three-dimensional rectangular coordinate system, a virtual sand table is set and mapped into the virtual sand table according to the coordinate values. The specific steps include: S61. According to the loaded three-dimensional virtual model, determine the minimum coordinate value x of the model boundary coordinate value on the three-dimensional vector min ,y min and z min , then adjust the coordinate center point of the virtual sandbox to (x min ,y min , z min ); S62. Specify the maximum value K of the three-dimensional coordinates of the virtual sandbox boundary; S63. After solving the object point object space coordinate value each time, the maximum value E of the newly entered coordinate data is calculated; S64. If there is Ex min >K or Ey min >K or Ez min >K, then determine the sand table coordinate adjustment ratio Then update the iteration rules for setting coordinate values ​​in the three-dimensional rectangular coordinate system and iterate all coordinates in the virtual sandbox: Where i is the number of iterations; S65. If there exists E-x min <K and E-y min <K and E-z min <K, then determine the maximum coordinate values x max , y max and z max in the three-dimensional vector of the model boundary coordinate values. There exists a sand table coordinate adjustment ratio After that, update K = MAX(x max , y max , z max ), and then update the iteration rule of the coordinate values in the three-dimensional rectangular coordinate system and iterate all the coordinates in the virtual sand table: Where i is the number of iterations; S66. The coordinate value adjustment is continuously mapped to the virtual sandbox and the above steps S61-S65 are repeated.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

10. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

Images