Industrial scene-oriented augmented reality scene planning and display system and method

Abstract

This invention relates to an augmented reality scene planning and display system for large-scale industrial scenarios, comprising: a mobile terminal, a scene planning unit, and an augmented reality display unit. The mobile terminal is equipped with a camera. The scene planning unit includes a scene creation module and an AR content planning module. The scene creation module is used to read image data acquired by the camera and create a corresponding scene map based on the characteristics of the image data. The AR content planning module is used to save virtual models, place virtual models in the scene map, bind virtual models corresponding to AR tags, and save the virtual model parameters as an AR display content script file. The augmented reality display unit is used to load and parse the scene map data and the AR display content script file, and simultaneously relocate the scene map data using the currently read image data. Based on the relocated scene map data and the parsed AR display content data, the virtual models are displayed in the physical scene.

Description

Technical Field

[0001] This invention relates to an augmented reality scene planning and display system and method for large-scale industrial scenarios, belonging to the fields of augmented reality technology and intelligent manufacturing technology. Background Technology

[0002] With the strategic transformation of China's economic development and the introduction of the Industry 4.0 concept, intelligent and visual technologies for industrial scenarios have become one of the current research hotspots. Traditional industrial production often faces challenges such as complex factory environments, a large number of parts to assemble, and complicated operating procedures. Workers need extensive training and to read numerous assembly manuals before they can operate the equipment, which incurs significant time costs, leading to low efficiency and high error rates in large-scale production.

[0003] Augmented Reality (AR) is a technology that overlays virtual objects onto real-world scenes and is currently widely used in industrial production, machinery assembly, and remote maintenance. However, most existing AR systems suffer from limitations such as limited functionality, low positioning accuracy, and poor robustness in large-scale scenarios. Furthermore, different applications need to be developed for different scenarios, making it difficult to meet the needs of real-world industrial production. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes an augmented reality scene planning and display system and method for large-scale industrial scenarios. It establishes a scene planning unit and an augmented reality display unit, which can help users quickly build AR application systems and reduce the development difficulty of AR applications.

[0005] The technical solution of the present invention is as follows:

[0006] On one hand, this invention provides an augmented reality scene planning and display system for large-scale industrial scenarios, comprising:

[0007] The mobile terminal includes a scene planning unit and an augmented reality display unit. The mobile terminal is equipped with a camera, and the scene planning unit and the augmented reality display unit are mounted inside the mobile terminal.

[0008] The scene planning unit includes a scene creation module and an AR content planning module. The scene creation module is used to read image data collected by the camera and create a corresponding scene map based on the characteristics of the image data. The AR content planning module is used to save virtual models, place virtual models in the scene map and bind virtual models corresponding to AR tags, and save the parameters of the placed virtual models and the parameters of the virtual models corresponding to AR tags as AR display content script files.

[0009] The augmented reality display unit is used to load and parse scene map data and AR display content script files, while reading image data collected by the camera. It relocates the scene map data based on the currently read image data, and displays the virtual models placed in the physical scene and the virtual models corresponding to the AR tags based on the relocated scene map data and the data parsed from the AR display content script files.

[0010] In a preferred embodiment, the scene creation module includes an image reading module and a mapping module;

[0011] The image reading module includes a camera video reading unit, used to read camera video image frames captured by the camera;

[0012] The mapping module includes a feature extraction unit, a keyframe selection unit, a camera pose solving unit, and a scene map generation unit. The feature extraction unit extracts feature points from camera video image frames, calculates the descriptors of the feature points and their three-dimensional coordinates in the world coordinate system, and obtains local map data. The keyframe selection unit tracks each camera video image frame based on the extracted feature points, determines whether to insert a keyframe, and generates a keyframe sequence to be inserted into the map, while simultaneously calculating the feature information of the current keyframe. The camera pose solving unit solves for the real-time pose of the camera.

[0013] As the mobile terminal rotates and translates, the scene map generation unit continuously expands and acquires different local map data. Based on the acquired local map data, key frame sequences and key frame feature information, and the real-time pose of the camera, the local map data is fused to form a scene point cloud map, which is saved as a point cloud map script file. At the same time, the camera video image frames acquired when generating the scene map are saved as scene video files.

[0014] In a preferred embodiment, the image reading module further includes a scene video reading unit for reading scene video image frames from a scene video file;

[0015] The AR content planning module includes a model library, a plane detection unit, an interaction unit, an AR tag planning unit, and a script file generation unit. The model library is used to store virtual models and set parameter information for the virtual models. The plane detection unit reads scene video image frames and identifies planar regions in the current scene video image frames. The interaction unit is used to select virtual models and place and manipulate them in the identified planar regions. The AR tag planning unit is used to generate AR tags and bind the correspondence between specified AR tags and virtual models.

[0016] The script file generation unit saves the virtual model information placed in the interaction unit, the parameter information for operating the virtual model, the AR tag information generated by the AR tag planning unit, and the virtual model information corresponding to the AR tag as a display script file.

[0017] In a preferred embodiment, the augmented reality display unit includes a map import module, a positioning module, a script parsing module, an AR tag parsing module, and a display module;

[0018] The map import module is used to load the point cloud map script file and obtain the scene point cloud map; the positioning module is used to obtain camera video image frames captured by the camera, match key frames according to the camera video image frames, and reposition the camera according to the information of the matched key frames; the script parsing module is used to parse the display script file, obtain virtual model information, and set the parameter information of the virtual model in the physical scene accordingly; the display module is used to display the placed virtual model in the physical scene and parse AR tags and display the corresponding virtual models according to the content of the display script file.

[0019] In a preferred embodiment, the AR tag is a QR code.

[0020] On the other hand, the present invention provides an augmented reality scene planning and display method for large-scale industrial scenarios, implemented based on any embodiment of the augmented reality scene planning and display system for large-scale industrial scenarios, including the following steps:

[0021] The operator uses a mobile terminal to scan and photograph the industrial scene, and the camera video image frame is acquired in real time through the camera video reading module in the scene planning unit and transmitted to the mapping module;

[0022] The feature extraction unit in the mapping module extracts feature points from camera video image frames to obtain local map data. The key frame selection unit generates a key frame sequence and calculates the feature information of the current key frame. The camera pose solving unit solves the real-time pose of the camera. The scene map generation unit generates a point cloud map script file and a scene video file.

[0023] The AR content planning module loads the scene video file, parses the point cloud map script file and imports the scene point cloud map, plays the scene video file as a single frame image, uses the plane detection unit to detect the plane area of ​​the current scene video image frame, uses the interaction unit to set up the virtual model on the plane area of ​​the scene and operate on the virtual model, uses the AR tag planning unit to generate AR tags and bind the correspondence between the specified AR tags and the virtual model, and uses the script file generation unit to generate the display content script file.

[0024] The augmented reality display unit loads point cloud map script files and display content script files to generate scene point cloud maps and obtain parameter information of virtual models. The positioning module repositions the camera to obtain the camera pose. The display module displays the placed virtual models in the physical scene and parses AR tags to display the corresponding virtual models, thus realizing augmented reality display and registration.

[0025] In a preferred embodiment, the method of generating a keyframe sequence through a keyframe selection unit and simultaneously calculating the feature information of the current keyframe is as follows:

[0026] The feature points extracted by the feature extraction unit are used to track and calculate each frame of scene video image. The key frame determination function in the tracking process determines whether to insert a key frame and generates a key frame sequence to be inserted into the map.

[0027] Simultaneously, the feature information of the current keyframe, including BoW vectors and co-view relationships, is calculated, and the keyframe is inserted into the scene point cloud map;

[0028] Based on feature information, optimization operations are performed on keyframes, including 3D point fusion, local BA optimization, and keyframe removal.

[0029] In a preferred embodiment, the method for solving the real-time pose of the camera using the camera pose solving unit is specifically as follows:

[0030] After successfully tracking the previous scene video image frame, the camera is set to move at a constant speed. The feature points in the previous scene video image frame are matched with the corresponding point clouds in the scene point cloud map. The camera pose of the previous scene video image frame is used to optimize the current camera pose.

[0031] If the feature points extracted from the current scene video image frame are few, match them with the nearest keyframe and perform feature matching using the BoW vector dictionary to optimize the camera pose.

[0032] In a preferred embodiment, the method for repositioning the camera using the positioning module to obtain the camera's pose is specifically as follows:

[0033] In the keyframe sequence, find candidate keyframes that correspond to the current camera video image frame. Use the BoW vector dictionary to match the current camera video image frame with the keyframe. For each candidate keyframe, use the EPNP algorithm to estimate the pose to achieve camera relocalization.

[0034] In a preferred embodiment, the method for parsing AR tags and displaying the corresponding virtual models is as follows:

[0035] Scan the AR tag with a mobile terminal, and decode the AR tag using the QR code decoding module of the mobile terminal to obtain the information stored in the AR tag;

[0036] The system identifies the location information of AR tags, calculates the location information of the corresponding virtual model in the physical scene based on the location information of AR tags, and overlays the virtual model onto the physical scene based on the location information of the corresponding virtual model in the physical scene to complete the augmented reality registration.

[0037] The present invention has the following beneficial effects:

[0038] 1. This invention provides an augmented reality scene planning and display system for large-scale industrial scenarios. It establishes scene planning units and augmented reality display units, allowing users to quickly build AR application systems and reducing the difficulty of developing AR applications.

[0039] 2. The present invention provides an augmented reality scene planning and display system for large industrial scenarios. The scene planning unit can construct a global scene point cloud map of the large scene through the mapping module, and plan the AR display content through the display content planning module and save it as a script file. The augmented reality display unit loads the relevant script file and can realize the reconstruction of AR display content and AR content display based on camera repositioning. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the system framework of Embodiment 1 of the present invention;

[0041] Figure 2 This is a flowchart of the method in Embodiment 2 of the present invention;

[0042] Figure 3 This is an example diagram illustrating the calculation of the QR code position in an embodiment of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0045] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0046] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0047] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0048] Example 1:

[0049] See Figure 1 This embodiment provides an augmented reality scene planning and display system for large-scale industrial scenarios, including:

[0050] The mobile terminal includes a scene planning unit and an augmented reality display unit. Specifically, the mobile terminal is a mobile computer equipped with a camera for video capture. The scene planning unit and the augmented reality display unit are respectively installed in the mobile computer in the form of software.

[0051] The scene planning unit includes a scene creation module and an AR content planning module. The scene creation module is used to read image data collected by the camera and create a corresponding scene map based on the characteristics of the image data. The AR content planning module is used to save virtual models, place virtual models in the scene map and bind virtual models corresponding to AR tags, and save the parameters of the placed virtual models and the parameters of the virtual models corresponding to AR tags as AR display content script files.

[0052] The augmented reality display unit is used to load and parse scene map data and AR display content script files, while reading image data collected by the camera. It relocates the scene map data based on the currently read image data, and displays the virtual models placed in the physical scene and the virtual models corresponding to the AR tags based on the relocated scene map data and the data parsed from the AR display content script files.

[0053] In a preferred embodiment of this invention, the scene creation module includes an image reading module and a mapping module;

[0054] The image reading module includes a camera video reading unit, used to read camera video image frames captured by the camera;

[0055] The mapping module is used for physical acquisition of the scene, and includes a feature extraction unit, a keyframe selection unit, a camera pose solving unit, and a scene map generation unit. The feature extraction unit is used to extract ORB feature points from camera video image frames, calculate the descriptors of the feature points and their three-dimensional coordinates in the world coordinate system, and obtain local map data. The keyframe selection unit tracks each camera video image frame based on the extracted feature points, determines whether to insert a keyframe, and generates a keyframe sequence to be inserted into the map, while calculating the feature information of the current keyframe. The camera pose solving unit is used to solve the real-time pose of the camera.

[0056] As the mobile terminal rotates and translates, the scene map generation unit continuously expands and acquires point cloud data from different local maps. Based on the acquired point cloud data of each local map, key frame sequences and key frame feature information, and the real-time pose of the camera, the local map data is fused to form a global scene SLAM sparse point cloud map, which is saved as a point cloud map script file. At the same time, the camera video image frames acquired when generating the scene map are saved as scene video files.

[0057] In a preferred embodiment of this example, the image reading module further includes a scene video reading unit, used to read scene video image frames from a scene video file;

[0058] The AR content planning module includes a model library, a plane detection unit, an interaction unit, an AR tag planning unit, and a script file generation unit. The model library stores virtual models and sets their parameter information; it includes a 3D model library and an instrument interface library, among others. The plane detection unit reads scene video image frames and identifies planar regions within the current scene video image frame. The interaction unit selects virtual models and places and manipulates them within the identified planar regions, including operations such as translation, scaling, and rotation. The AR tag planning unit generates AR tags and binds the specified AR tags to the corresponding virtual models.

[0059] The script file generation unit saves the virtual model information placed in the interaction unit, the parameter information for operating the virtual model, the AR tag information generated by the AR tag planning unit, and the virtual model information corresponding to the AR tag as a display script file.

[0060] In a preferred embodiment of this invention, the augmented reality display unit includes a map import module, a positioning module, a script parsing module, an AR tag parsing module, and a display module.

[0061] The map import module is used to load the point cloud map script file and obtain the scene point cloud map; the positioning module is used to obtain camera video image frames captured by the camera, match key frames according to the camera video image frames, and reposition the camera according to the information of the matched key frames; the script parsing module is used to parse the display script file, obtain virtual model information, and set the parameter information of the virtual model in the physical scene accordingly; the display module is used to display the placed virtual model in the physical scene and parse AR tags and display the corresponding virtual models according to the content of the display script file.

[0062] In a preferred embodiment of this invention, the AR tag is a QR code.

[0063] Example 2:

[0064] See Figure 2 This embodiment provides an augmented reality scene planning and display method for large-scale industrial scenarios. Based on any embodiment of the augmented reality scene planning and display system for large-scale industrial scenarios described in this invention, the method includes the following steps:

[0065] S1. The operator uses a mobile terminal to scan and photograph the industrial scene, and the camera video image frame is acquired in real time through the camera video reading module in the scene planning unit and transmitted to the mapping module.

[0066] S2. The feature extraction unit in the mapping module extracts ORB feature points from the camera video image frames, calculates the descriptors of the feature points and their three-dimensional coordinates in the world coordinate system, and uses an image pyramid to achieve feature scale invariance. The pyramid has 8 layers and the scale factor of the pyramid image is 1.2 to obtain local map data. The keyframe selection unit generates a keyframe sequence and calculates the feature information of the current keyframe. The camera pose solving unit solves the real-time pose of the camera. The scene map generation unit generates a point cloud map script file and a scene video file.

[0067] S3. The scene video file is loaded through the AR content planning module, and the point cloud map script file is parsed and imported into the scene point cloud map. The scene video file is played in the form of single-frame images. During the video playback, the operator can control the video playback through the interactive unit, which can pause or slow down the video playback. The plane detection unit detects and processes the current scene video image frames. RANSAC (Random Sample Consensus Algorithm) is used to perform plane detection on the map point cloud corresponding to the current scene video image frame and calculate its model matrix. At this time, the virtual model is placed on the plane area of ​​the appropriate scene by clicking the "Place Model" button in the interactive unit. The virtual model can be rotated, translated, and scaled in the physical scene by keyboard interaction, mouse dragging, and other operations. The AR tag planning unit is used to generate AR tags and bind the correspondence between the specified AR tags and the virtual models. After the above settings are completed, the script file generation unit generates the display content script file by generating the virtual model, all parameter information of the virtual model, and AR tags and corresponding virtual model information.

[0068] S4. Using the augmented reality display unit, load the point cloud map script file and the display content script file to generate a scene point cloud map and obtain the parameter information of the virtual model; use the positioning module to reposition the camera and obtain the camera pose; use the display module to display the placed virtual model in the physical scene and parse the AR tag and display the corresponding virtual model to realize the display and registration of augmented reality.

[0069] In a preferred embodiment of this invention, the method of generating a key frame sequence through a key frame selection unit and simultaneously calculating the feature information of the current key frame is specifically as follows:

[0070] The feature points extracted by the feature extraction unit are used to track and calculate each frame of scene video image. The key frame determination function in the tracking process determines whether to insert a key frame and generates a key frame sequence to be inserted into the map.

[0071] Simultaneously, the feature information of the current keyframe, including BoW vectors and co-view relationships, is calculated, and the keyframe is inserted into the scene point cloud map;

[0072] Based on feature information, optimization operations are performed on keyframes, including 3D point fusion, local BA optimization, and keyframe removal.

[0073] In a preferred embodiment of this example, the method for solving the real-time pose of the camera using the camera pose solving unit is specifically as follows:

[0074] After successfully tracking the previous scene video image frame, the camera is set to move at a constant speed. The feature points in the previous scene video image frame are matched with the corresponding point clouds in the scene point cloud map. The camera pose of the previous scene video image frame is used to optimize the current camera pose.

[0075] If the feature points extracted from the current scene video image frame are few, match them with the nearest keyframe and perform feature matching using the BoW vector dictionary to optimize the camera pose.

[0076] The aforementioned feature points, keyframes, and camera poses are added to the local map. The local map is expanded by rotating and translating the computer. The local map is continuously merged to form a global map, and the information in the SLAM sparse point cloud map is saved as a binary SLAM script file.

[0077] In a preferred embodiment of this example, the method for repositioning the camera using the positioning module to obtain the camera's pose is specifically as follows:

[0078] In the keyframe sequence, find candidate keyframes that correspond to the current camera video image frame. Use the BoW vector dictionary to match the current camera video image frame with the keyframe. For each candidate keyframe, use the EPNP algorithm to estimate the pose to achieve camera relocalization.

[0079] See Figure 3 In a preferred embodiment of this invention, in step S4, the method for displaying and registering augmented reality by displaying the placed virtual model in the physical scene through the display module and parsing the AR tag and displaying the corresponding virtual model is as follows:

[0080] The display module parses the parameter information of the placed virtual model through the script parsing module and places the virtual model in the physical space; at the same time, the positioning module realizes the repositioning of the camera and sets the real-time pose of the camera in the physical space, thereby realizing augmented reality display.

[0081] Scan the QR code in the scene using a mobile terminal, and decode the QR code using the QR code decoding module of the mobile terminal to obtain the information stored in the QR code;

[0082] The camera of the mobile computer (coordinate system O) s -X s Y s Real-time recognition of the screen coordinates (2D coordinates) of the four corner points (i.e., feature points) in a QR code, denoted as Z. i (u i ,v i ); Let the feature point be in the camera coordinate system (O); c -X c Y c Z cThe three-dimensional coordinates under () are P i (x i ,y i ,z i The intrinsic parameter matrix K of the camera is calibrated using the MATLAB camera calibration toolbox, and then solved using the following formula:

[0083]

[0084] Suppose a point is in the world coordinate system (O) p -X p Y p Z p The coordinates of ) are P w (x w ,y w ,z w ), which corresponds to the camera coordinate system O. c -X c Y c Z c For P w (x c ,y c ,z c Their transformation relationship is as follows, where [R:t] is the transformation matrix:

[0085]

[0086] Given that the corner points of the QR code are in screen coordinate system O s -X s Y s and camera coordinate system O c -X c Y c Z c The coordinates are obtained by using the PnP algorithm and solving for [R:t] according to the above formula; the position of the virtual model in the world coordinate system is calculated based on [R:t], and the virtual model is superimposed on the position of the QR code to complete the augmented reality registration.

[0087] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An augmented reality scene planning and display system for large-scale industrial scenarios, characterized in that, include: The mobile terminal includes a scene planning unit and an augmented reality display unit. The mobile terminal is equipped with a camera, and the scene planning unit and the augmented reality display unit are mounted inside the mobile terminal. The scene planning unit includes a scene creation module and an AR content planning module. The scene creation module is used to read image data collected by the camera and create a corresponding scene map based on the characteristics of the image data. The AR content planning module is used to save virtual models, place virtual models in the scene map and bind virtual models corresponding to AR tags, and save the parameters of the placed virtual models and the parameters of the virtual models corresponding to AR tags as AR display content script files. The augmented reality display unit is used to load and parse scene map data and AR display content script file, while reading image data collected by the camera, relocating the scene map data using the currently read image data, and displaying the virtual model placed in the physical scene and the virtual model corresponding to the AR tag based on the relocated scene map data and the data parsed from the AR display content script file. The scene creation module includes an image reading module and a mapping module; The image reading module includes a camera video reading unit, used to read camera video image frames captured by the camera; The mapping module includes a feature extraction unit, a keyframe selection unit, a camera pose solving unit, and a scene map generation unit. The feature extraction unit extracts feature points from camera video image frames, calculates the descriptors of the feature points and their three-dimensional coordinates in the world coordinate system, and obtains local map data. The keyframe selection unit tracks each camera video image frame based on the extracted feature points, determines whether to insert a keyframe, and generates a keyframe sequence to be inserted into the map, while simultaneously calculating the feature information of the current keyframe. The camera pose solving unit solves for the real-time pose of the camera. As the mobile terminal rotates and translates, the scene map generation unit continuously expands and acquires different local map data. Based on the acquired local map data, key frame sequences and key frame feature information, and the real-time pose of the camera, the local map data is fused to form a scene point cloud map, which is saved as a point cloud map script file. At the same time, the camera video image frames acquired when generating the scene map are saved as scene video files.

2. The augmented reality scene planning and display system for large-scale industrial scenarios according to claim 1, characterized in that: The image reading module also includes a scene video reading unit, used to read scene video image frames from a scene video file; The AR content planning module includes a model library, a plane detection unit, an interaction unit, an AR tag planning unit, and a script file generation unit; The model library is used to store virtual models and set parameter information for the virtual models; the plane detection unit reads scene video image frames and identifies planar regions in the current scene video image frames; the interaction unit is used to select virtual models and place and manipulate virtual models in the identified planar regions; the AR tag planning unit is used to generate AR tags and bind the correspondence between specified AR tags and virtual models. The script file generation unit saves the virtual model information placed in the interaction unit, the parameter information for operating the virtual model, the AR tag information generated by the AR tag planning unit, and the virtual model information corresponding to the AR tag as a display script file.

3. The augmented reality scene planning and display system for large-scale industrial scenarios according to claim 2, characterized in that: The augmented reality display unit includes a map import module, a positioning module, a script parsing module, an AR tag parsing module, and a display module; The map import module is used to load the point cloud map script file and obtain the scene point cloud map; the positioning module is used to obtain camera video image frames captured by the camera, match key frames according to the camera video image frames, and reposition the camera according to the information of the matched key frames; the script parsing module is used to parse the display script file, obtain virtual model information, and set the parameter information of the virtual model in the physical scene accordingly. The display module is used to display the placed virtual models in the physical scene and to parse AR tags and display the corresponding virtual models according to the content of the display script file.

4. The augmented reality scene planning and display system for large-scale industrial scenarios according to claim 2, characterized in that: The AR tag is a QR code.

5. An augmented reality scene planning and display method for large-scale industrial scenarios, implemented based on the augmented reality scene planning and display system for large-scale industrial scenarios as described in any one of claims 1-4, characterized in that, Includes the following steps: The operator uses a mobile terminal to scan and photograph the industrial scene, and the camera video image frame is acquired in real time through the camera video reading module in the scene planning unit and transmitted to the mapping module; The feature extraction unit in the mapping module extracts feature points from camera video image frames to obtain local map data. The key frame selection unit generates a key frame sequence and calculates the feature information of the current key frame. The camera pose solving unit solves the real-time pose of the camera. The scene map generation unit generates a point cloud map script file and a scene video file. The AR content planning module loads the scene video file, parses the point cloud map script file and imports the scene point cloud map, plays the scene video file as a single frame image, uses the plane detection unit to detect the plane area of ​​the current scene video image frame, uses the interaction unit to set up the virtual model on the plane area of ​​the scene and operate on the virtual model, uses the AR tag planning unit to generate AR tags and bind the correspondence between the specified AR tags and the virtual model, and uses the script file generation unit to generate the display content script file. The augmented reality display unit loads point cloud map script files and display content script files to generate scene point cloud maps and obtain parameter information of virtual models. The positioning module repositions the camera to obtain the camera pose. The display module displays the placed virtual models in the physical scene and parses AR tags to display the corresponding virtual models, thus realizing augmented reality display and registration.

6. The augmented reality scene planning and display method for large-scale industrial scenarios according to claim 5, characterized in that, The method for generating a keyframe sequence through a keyframe selection unit and simultaneously calculating the feature information of the current keyframe is as follows: The feature points extracted by the feature extraction unit are used to track and calculate each frame of scene video image. The key frame determination function in the tracking process determines whether to insert a key frame and generates a key frame sequence to be inserted into the map. Simultaneously, the feature information of the current keyframe, including BoW vectors and co-view relationships, is calculated, and the keyframe is inserted into the scene point cloud map; Based on feature information, optimization operations are performed on keyframes, including 3D point fusion, local BA optimization, and keyframe removal.

7. The augmented reality scene planning and display method for large-scale industrial scenarios according to claim 5, characterized in that, The method for repositioning the camera and obtaining its pose using the positioning module is as follows: In the keyframe sequence, find candidate keyframes that correspond to the current camera video image frame. Use the BoW vector dictionary to match the current camera video image frame with the keyframe. For each candidate keyframe, use the EPNP algorithm to estimate the pose to achieve camera relocalization.

8. The augmented reality scene planning and display method for large-scale industrial scenarios according to claim 5, characterized in that, The method for parsing AR tags and displaying the corresponding virtual models is as follows: Scan the AR tag with a mobile terminal, and decode the AR tag using the QR code decoding module of the mobile terminal to obtain the information stored in the AR tag; The system identifies the location information of AR tags, calculates the location information of the corresponding virtual model in the physical scene based on the location information of AR tags, and overlays the virtual model onto the physical scene based on the location information of the corresponding virtual model in the physical scene to complete the augmented reality registration.

Images