Methods, systems, and computer-readable storage media based on augmented reality content

Abstract

The application provides a method, system and computer readable storage medium based on augmented reality content. The method comprises: obtaining visual map information of a target scene; obtaining video frame information of a virtual scene; configuring the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information; configuring placement scene image information and placement content information of augmented reality content based on the video frame information, the placement content information being suitable for displaying in the video frame information having a configuration relationship with the visual map information according to the specified placement scene image information; and generating data of the augmented reality content based on the placement scene image information and the placement content information. The application can realize the same augmented reality content as the experience of an offline real scene in a virtual scene.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and in particular to a method for generating augmented reality content data, a method for displaying augmented reality content data, a system for generating augmented reality content data, a system for displaying augmented reality content data, and a computer-readable storage medium. Background Technology

[0002] Augmented Reality (AR) aims to provide a seamless experience that blends the real and virtual worlds. Early AR systems required users to scan predefined markers or flat images to trigger predefined AR content. While this method had a large user base, it was small-scale and its integration with the real world felt unnatural. Therefore, recent developments in AR have focused on building large-scale 3D maps of real-world spaces and interacting with them through the device's camera to trigger true 3D, large-scale, real-time AR effects. This results in realistic AR effects and a superior user experience, and has been widely adopted in areas such as smart commerce in shopping malls, navigation and guidance in scenic areas, and AR multiplayer games.

[0003] Chinese patent application No. 202210575510.6 discloses an augmented reality scene planning and display system for large-scale industrial scenarios, comprising: a mobile terminal, a scene planning unit, and an augmented reality display unit. The mobile terminal is equipped with a camera. The scene planning unit includes a scene creation module and an AR content planning module. The scene creation module is used to read image data collected by the camera and create a corresponding scene map based on the characteristics of the image data. The AR content planning module is used to save virtual models, place virtual models in the scene map, bind virtual models corresponding to AR tags, and save the virtual model parameters as an AR display content script file. The augmented reality display unit is used to load and parse the scene map data and the AR display content script file, and simultaneously relocate the scene map data using the currently read image data. Based on the relocated scene map data and the parsed AR display content data, the virtual models are displayed in the physical scene.

[0004] Existing augmented reality scene planning and display systems rely on users to locate and experience augmented reality content in real-world scenarios (such as scenic spots and commercial districts) using devices like mobile phones and cameras. They cannot simultaneously provide augmented reality content for real-world scenarios within online virtual scenes. Summary of the Invention

[0005] The technical problem solved by the present invention is: how to achieve the same augmented reality experience in a virtual scene as in an offline real scene, so that users can obtain integrated hybrid augmented reality content in both offline real scenes and online virtual scenes.

[0006] To address the aforementioned technical problems, the present invention provides a method for generating augmented reality content data, comprising:

[0007] Obtain visual map information of the target scene;

[0008] Based on the visual map information, configure the first placement scene image information and the first placement content information of augmented reality content. The first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information.

[0009] Obtain video frame information of the virtual scene;

[0010] The video frame information is configured in the visual map information to form a configuration relationship between the video frame information and the visual map information;

[0011] Based on the video frame information, configure the second placement scene image information and the second placement content information of the augmented reality content. The second placement content information is suitable for displaying the virtual scene video that has a configuration relationship with the visual map information according to the specified second placement scene image information.

[0012] The augmented reality content data is generated based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information.

[0013] Optionally, the visual map information includes: a first image feature set and a first three-dimensional point cloud coordinate set corresponding to the target scene captured image.

[0014] Optionally, the first placement scene image information includes: first image positioning data associated with the first image feature set and the first three-dimensional point cloud coordinate set; the first placement content information is configured corresponding to the first image positioning data.

[0015] Optionally, the first placement scene image information includes: entity pose data; the first placement scene image information and the first placement content information configured based on the visual map information include:

[0016] Select the entity in the target scene where the first content to be placed needs to be displayed;

[0017] Obtain entity image features of the target scene based on the selected target scene entities;

[0018] The entity image feature set is matched with the first image feature set to obtain the matched first image feature set and the corresponding three-dimensional point cloud coordinate set;

[0019] Based on the entity image feature set and the corresponding 3D point cloud coordinate set, the entity pose of the content to be placed is obtained through the PNP algorithm.

[0020] Optionally, the first placement scene image information further includes: entity pose data; the first placement scene image information and the first placement content information configured based on the visual map information further include:

[0021] The rendering position of the content to be placed is determined based on the entity's pose.

[0022] Match the determined rendering position with the first placement content information;

[0023] The first placement content information is adapted to be displayed on the visual map at the matched rendering location.

[0024] Optionally, the video frame information includes: a second image feature set corresponding to the virtual scene video frame image; configuring the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information includes:

[0025] Configure the second image feature set based on the first image feature set to form a configuration relationship between the second image feature set and the first image feature set;

[0026] Based on the configuration relationship between the second image feature set and the first image feature set, the first three-dimensional point cloud coordinate set corresponding to the first image feature set is used as the second three-dimensional point cloud coordinate set.

[0027] Optionally, the second placement scene image information includes: second image positioning data associated with the first image feature set and the first three-dimensional point cloud coordinate set; the second placement content information is configured corresponding to the second image positioning data.

[0028] Optionally, the second image positioning data is derived from the first image positioning data.

[0029] Optionally, the first placement scene image information includes: an entity image feature set, an entity image 3D point cloud coordinate set, and first image positioning data; the second placement scene image information includes: a virtual entity image feature set, a virtual entity 3D point cloud coordinate set, and second image positioning data that have a configuration relationship with the first placement scene image information; the second placement content information is placed corresponding to the second image positioning data.

[0030] Optionally, obtaining the visual map information of the target scene includes:

[0031] Acquire images of the target scene;

[0032] Image features are extracted from the images captured for each target scene to obtain a first image feature set for each target scene image;

[0033] The extracted image features are processed using the structure-of-motion reconstruction algorithm to obtain the first three-dimensional point cloud coordinate set of each target scene image.

[0034] To address the aforementioned technical problems, the present invention also provides a method for generating augmented reality content data, comprising:

[0035] Obtain visual map information of the target scene;

[0036] Obtain video frame information of the virtual scene;

[0037] The video frame information is configured in the visual map information to form a configuration relationship between the video frame information and the visual map information;

[0038] Based on the video frame information, the placement scene image information and placement content information of augmented reality content are configured. The placement content information is adapted to be displayed in video frame information that has a configuration relationship with the visual map information according to the specified placement scene image information.

[0039] The augmented reality content data is generated based on the placement scene image information and placement content information.

[0040] To address the aforementioned technical problems, the present invention also provides a method for displaying augmented reality content, wherein the augmented reality content data generated based on the aforementioned augmented reality content data generation method includes:

[0041] Acquire captured image information;

[0042] The location information of the image capturing device in the target scene is determined based on the visual map information and the captured image information;

[0043] When the positioning information is associated with the first placement scene image information, the first placement content information is displayed according to the specified first scene image information;

[0044] Load video frame information of the virtual scene;

[0045] When the loaded video frame information is associated with the second placement scene image information, the second placement content information is displayed according to the specified second scene image information.

[0046] To address the aforementioned technical problems, the present invention also provides a method for displaying augmented reality content, wherein the augmented reality content data generated based on the aforementioned augmented reality content data generation method includes:

[0047] Load video frame information of the virtual scene;

[0048] When the loaded video frame information is associated with the placement scene image information, the placement content information is displayed according to the specified placement scene image information.

[0049] To address the aforementioned technical problems, the present invention also provides a data generation system for augmented reality content, comprising:

[0050] The first acquisition unit is suitable for acquiring visual map information of the target scene;

[0051] The first configuration unit is adapted to configure first placement scene image information and first placement content information of augmented reality content based on the visual map information, wherein the first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information.

[0052] The second acquisition unit is adapted to acquire video frame information of the virtual scene;

[0053] The second configuration unit is adapted to configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information.

[0054] The third configuration unit is adapted to configure second placement scene image information and second placement content information of augmented reality content based on the video frame information. The second placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified second placement scene image information.

[0055] The storage unit is adapted to generate data of the augmented reality content based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information.

[0056] To address the aforementioned technical problems, the present invention also provides a data generation system for augmented reality content, comprising:

[0057] The first acquisition unit is suitable for acquiring visual map information of the target scene;

[0058] The second acquisition unit is adapted to acquire video frame information of the virtual scene;

[0059] The first configuration unit is adapted to configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information.

[0060] The second configuration unit is adapted to configure placement scene image information and placement content information of augmented reality content based on the video frame information. The placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified placement scene image information.

[0061] The storage unit is adapted to generate data of the augmented reality content based on the placement scene image information and placement content information.

[0062] To address the aforementioned technical problems, the present invention also provides a data display system for augmented reality content, and a data generation system based on the augmented reality content described above, comprising:

[0063] The third acquisition unit is suitable for acquiring captured image information;

[0064] The positioning unit is adapted to determine the positioning information of the imaging device in the target scene based on the visual map information and the captured image information;

[0065] The first display unit is adapted to display the first placement content information according to the specified first scene image information when the positioning information is associated with the first placement scene image information;

[0066] The second display unit is suitable for loading video frame information of the virtual scene;

[0067] The third display unit is adapted to display the second placement content information according to the specified second scene image information when the loaded video frame information is associated with the second placement scene image information.

[0068] To address the aforementioned technical problems, the present invention also provides a data display system for augmented reality content, and a data generation system based on the augmented reality content described above, comprising:

[0069] The first display unit is suitable for loading video frame information of the virtual scene;

[0070] The second display unit is adapted to display the placement content information according to the specified placement scene image information when the loaded video frame information is associated with the placement scene image information.

[0071] To address the aforementioned technical problems, the present invention also provides a data display system for augmented reality content, comprising: a processor and a memory; the memory stores a computer program, and the processor executes the steps of the augmented reality content data generation method described above when executing the computer program.

[0072] To address the aforementioned technical problems, the present invention also provides a data generation system for augmented reality content, comprising: a processor and a memory; the memory stores a computer program, and the processor executes the steps of the data generation method for augmented reality content as described above when executing the computer program.

[0073] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the data generation method for augmented reality content as described above.

[0074] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for displaying augmented reality content as described above.

[0075] The beneficial effects of the technical solution of the present invention include at least the following:

[0076] The technical solution of this invention can construct integrated map data that combines offline real scenes and online virtual scenes for enhanced display. Based on the visual map information of the target scene, the video frame information and visual map information of the virtual scene are configured to have a relationship. Based on the placement scene image information and placement content of the real scene, the related placement scene image information and placement content in the virtual scene are established to generate data of integrated augmented reality content of offline real scenes and online virtual scenes.

[0077] The technical solution of this invention is also based on the data of augmented reality content that integrates offline real scenes and online virtual scenes, enabling users to achieve integrated display of augmented display content in both offline real scenes and online virtual scenes.

[0078] The technical solution of this invention further solves the problem that augmented reality systems in the prior art can only be experienced offline, while providing a consistent online and offline experience. It achieves synchronous online scene registration and rendering, allowing users to experience the same virtual-real fusion experience as in offline scenes without being at the target site or leaving home. Attached Figure Description

[0079] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0080] Figure 1 This is a schematic diagram of the framework structure of the inventive concept upon which the technical solution of this invention is based;

[0081] Figure 2A flowchart illustrating the steps of a data generation method for augmented reality content provided by the technical solution of this invention;

[0082] Figure 3 A flowchart illustrating the steps of another method for generating augmented reality content provided by the technical solution of the present invention;

[0083] Figure 4 A flowchart illustrating the steps of a method for displaying augmented reality content provided by the present invention.

[0084] Figure 5 A flowchart illustrating the steps of another method for displaying augmented reality content provided by the present invention. Detailed Implementation

[0085] To better illustrate the technical solution of the present invention, the present invention will be further described below with reference to the accompanying drawings.

[0086] Existing augmented reality (AR) systems for offline real-world scenarios can be exemplified by Huawei's Cyberverse technology. Cyberverse provides services in locations such as tourist attractions, museums, smart parks, airports, high-speed rail stations, and commercial spaces. This technology requires users to physically visit the actual location, such as a scenic area or shopping district, for location-based experience. While Aibee offers AR systems for both online and offline experiences, the online experience primarily relies on traditional VR methods, failing to achieve true integration of offline AR content with online mixed reality content.

[0087] This invention addresses the inconsistency in the online and offline display experience of augmented reality (AR) content by constructing augmented reality (AR) content data. Through the AR content data generation and display technology of this invention, users can experience AR content not only in offline target scenes but also in online virtual scenes by associating the same AR content with visual map data of the target scene, thus achieving a unified mixed reality effect both online and offline.

[0088] To achieve correlation and fusion between augmented reality content data from offline real-world scenes and augmented reality content data from online virtual scenes, this invention employs visual registration technology. By using pre-recorded video and image sequences and aligning them with a high-precision map through a high-precision registration service, the precise position and pose of each frame are obtained in advance. This allows the visual map based on the offline target scene to achieve visual registration and correlation with the video and image sequences of the virtual scene. Users can experience AR effects consistent with offline experiences without being physically present, by loading pre-recorded video and position / pose data and through rendering and interaction, thus solving the problem of limited users in large-scale AR experiences.

[0089] Figure 1 The diagram illustrates the framework of the invention's technical solution. The invention is developed based on the following framework components: high-precision map acquisition and construction, AR scene editing, offline visual positioning services and offline AR experience; panoramic experience video acquisition, scene registration, and online AR interaction. Among them:

[0090] High-precision map acquisition and construction refers to collecting video, image, laser, GPS, and other data of a scene using specialized acquisition tools, constructing a high-precision map using 3D reconstruction algorithms, and then using this map as input for visual positioning and AR scene editing. AR scene editing refers to designing virtual AR scenes and placing virtual AR content based on the high-precision map construction results, which can then be experienced both online and offline. Offline visual positioning and offline AR interaction refer to offline users acquiring image information using their mobile phone cameras, comparing it with the constructed high-precision map through a visual positioning system to determine their position and orientation relative to the map, and then loading the corresponding virtual AR content to achieve an interactive effect that combines the virtual and real worlds. Panoramic experience video acquisition refers to recording or shooting real scenes directly using panoramic cameras or mobile phones to create online virtual scene videos, or constructing virtual scenes using computers to create online virtual scene videos. Scene registration and online AR interaction refer to for online users, designing routes, acquiring panoramic video, and then registering each frame of the video to a high-precision map through visual registration to obtain the position and orientation of the panoramic image relative to the high-precision map. Users can load panoramic videos online, along with their synchronized position and posture, and then load AR scenes to experience the same AR effects as offline.

[0091] Based on the above inventive ideas, continue to refer to Figure 2 This embodiment provides a method for generating augmented reality content data to construct data that enables an integrated online and offline experience of augmented reality content. The method includes the following steps:

[0092] Step S100: Obtain visual map information of the target scene.

[0093] In step S100, the method for obtaining the visual map information of the target scene can employ existing visual map generation and localization techniques. The most common visual map generation technique is structure-from-motion (SFM) analysis, which mainly includes: capturing images of the scene, extracting and matching image features, triangulating to obtain spatial 3D points, and finally optimizing using bundle adjustment. The visual map generated using this technique consists of image feature data and corresponding 3D point cloud data. Specifically, the visual map information of the target scene in this step can be obtained in the following way:

[0094] Acquire images of the target scene;

[0095] Image features are extracted from the images captured for each target scene to obtain a first image feature set for each target scene image;

[0096] The extracted image features are processed using the structure-of-motion reconstruction algorithm to obtain the first three-dimensional point cloud coordinate set of each target scene image.

[0097] In this embodiment, the images of the target scene are denoted as P1, P2, P3…P m 1, 2, 3...m represents the sequence of images captured of the target scene, m represents the total number of images captured of the target scene, and m is a natural number greater than zero.

[0098] By capturing images P1, P2, P3...P of the target scene m Image features were extracted to obtain images P1, P2, P3…P corresponding to the target scene. m The first image feature set J1, J2, J3…J m The first image feature set J1, J2, J3…J m This can be further expressed as J1 = {j 11 ,j 12 …j 1N}、J2={j 21 ,j 22 …j 2N}、J3={j 31 ,j 32 …j 3N}…J m ={j m1 ,j m2 …j mN Each image feature set J1, J2, J3…J mIt includes at least N image features extracted from the corresponding target scene image, where N represents the total number of image features extracted from each target scene image. The value of N can vary within a range depending on the actual image feature extraction, generally between 1000 and 2000. A value between 3000 and 5000 can ensure better image feature preservation. N is a natural number greater than zero. It can be understood that a lower value of N can reduce the amount of image data without losing image features, thus better preserving the image features required for image localization; while a higher value of N can better preserve image features, but the relative amount of image data is also larger. The image features extracted from the corresponding target scene image region can include one or more of the following image features: color features, texture features, shape features, etc. of scene objects; image edges, corners, lines, curves, and other regions with special attributes, etc.

[0099] After processing the extracted image features using the structure-on-motion algorithm, it is possible to reconstruct each target scene image P1, P2, P3…P m The acquisition equipment parameters and the 3D structure of the scene are used to obtain 3D point cloud data S1, S2, S3…S m 3D point cloud data S1, S2, S3…S m The image features J1{j are listed above in sequence. 11 ,j 12 …j 1N}、J2{j 21 ,j 22 …j 2N}、J3{j 31 ,j 32 …j 3N}…J m {j m1 ,j m2 …j mN} Three-dimensional coordinate data placed in a preset three-dimensional coordinate system. S1 is a local feature J1{j 11 ,j 12 …j 1N Feature point j in} 11 ,j 12 …j 1N A set of three-dimensional coordinate data, S2 is a local feature J2{j 21 ,j 22 …j 2N Feature point j in} 21 ,j 22 …j 2N A set of three-dimensional coordinate data, S3 is a local feature J3{j 31 ,j 32 …j 3N Feature point j in} 31 ,j32 …j 3N The set of three-dimensional coordinate data, and so on, S m For local features J m J m {j m1 ,j m2 …j mN Feature point j in} m1 ,j m2 …j mN A set of three-dimensional coordinate data. Structure-of-motion (SOR) algorithms are a general term for techniques that recover a scene's three-dimensional sparse point cloud from a series of two-dimensional images. Their main steps include feature matching, point cloud triangulation, and basis-analysis (BA) optimization.

[0100] Therefore, the visual map information obtained based on the above steps of this embodiment includes: a first image feature set and a first three-dimensional point cloud coordinate set corresponding to the target scene captured images, that is, the target scene captured images P1, P2, P3...P m The visual map information is the first image feature set J1, J2, J3…J m And the first three-dimensional point cloud coordinate set S1, S2, S3…S m 3D point cloud data S1, S2, S3…S m The image feature sets J1{j are as follows: 11 ,j 12 …j 1N}、J2{j 21 ,j 22 …j 2N}、J3{j 31 ,j 32 …j 3N}…J m {j m1 ,j m2 …j mN} Three-dimensional coordinate data placed in a preset three-dimensional coordinate system.

[0101] Continue to refer to Figure 2 The data generation method for augmented reality content in this embodiment further includes:

[0102] Step S101: Configure first placement scene image information and first placement content information of augmented reality content based on the visual map information. The first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information.

[0103] In step S101, the first placement scene image information includes: first image positioning data associated with the first image feature set and the first three-dimensional point cloud coordinate set. The first image positioning data can be obtained in the following manner:

[0104] Select the entity in the target scene where the first content to be placed needs to be displayed;

[0105] Obtain the entity image features of the selected target scene entity;

[0106] The entity image feature set is matched with the first image feature set to obtain the matched first image feature set and the corresponding three-dimensional point cloud coordinate set;

[0107] Based on the entity image feature set and the corresponding 3D point cloud coordinate set, the entity pose of the content to be placed is obtained through the PNP algorithm.

[0108] In a specific embodiment, the entity in the target scene that needs to display the first content can be configured as y. Depending on the configuration, images of entity y from different positions can be obtained by taking photos or videos of entity y to obtain entity image features J', where J'{j'1,j'2…j'} N}, N can select the number of image features to be extracted within a preset range as needed. The entity image is placed in the visual map coordinate system to obtain the entity pose at different orientations; specifically, the entity image features J' are compared with the image feature set J1{j 11 ,j 12 …j 1N}、J2{j 21 ,j 22 …j 2N}、J3{j 31 ,j 32 …j 3N}…J m {j m1 ,j m2 …j mN Matching is performed to obtain the first image feature set J0' after matching, and based on the image feature set J0', the first three-dimensional point cloud coordinate set S1, S2, S3...S is used to obtain the first image feature set J0'. m Obtain the 3D point cloud coordinates S0' corresponding to the image features in the image feature set J0'. Based on the entity image features J' and the 3D point cloud coordinate set S0', the entity pose of the content to be placed is obtained using the PNP algorithm. Finally, the entity poses of the content to be placed corresponding to different orientations of entity y are obtained.

[0109] The Perspective-n-Point (PNP) algorithm is used to estimate the pose of a calibrated camera given n 3D points in a world reference frame and their corresponding 2D projections in an image. This PNP algorithm can be used to determine the pose of the capturing device based on image features J' (corresponding to the 2D projection) and a 3D point cloud coordinate set S' (relative to the 3D points). Since the PNP algorithm is widely used in existing technologies such as camera pose tracking, object pose tracking, AR / VR, robot manipulation, and initial pose value solving in SLAM, common solutions include the DLT algorithm, P3P algorithm, EPnP algorithm, and UPnP algorithm, which are very fundamental algorithms in all 3D vision fields. Therefore, the technical solution of this invention will not be elaborated further.

[0110] For entities in the target scene, this embodiment also uses a shooting device to capture entity images from different angles and orientations to obtain different entity poses for the entity images.

[0111] In this step, the first placement content information can be AR video stream data, which includes frame images and AR content. AR content can be used to visually indicate the position of an entity relative to the current field of view and can include at least one of the following: graphics, animation, video, text, and icons. When configuring augmented reality content based on the visual map information, the rendering position in the visual map can be further determined based on the entity pose to match the first placement content required by the system. When externally inputting positioning data, if the system detects that the positioning data matches a preset entity pose, the system can determine the first placement content to be placed based on the entity pose and display the first placement content on the visual map according to the rendering position corresponding to the entity pose. In a specific example of this embodiment, the entity pose can be directly preset to map the required AR video stream data and configure the rendering position.

[0112] Step S102: Obtain video frame information of the virtual scene.

[0113] In step S102, a panoramic camera can be used to capture panoramic video of a high-precision map to obtain video frame information of the virtual scene. Besides panoramic video, a mobile phone can also be used to shoot corresponding videos to obtain video frame information of the virtual scene.

[0114] Specifically, the video frame information includes: virtual scene video frame images X1, X2, X3…X R The corresponding second image feature set is defined as virtual scene video frame images X1, X2, X3…X… R The corresponding second image feature set is J x1 J x2 J x3 ...J x1RR is the number of video frame images, and R is a natural number greater than zero. J x1 J x2 J x3 ...J x1R The sets contain image feature elements {jx} 11 jx 12 …jx 1N}、{jx 21 jx 22 …jx 2N}、{jx 31 jx 32 …jx 3N}、…、{jx R1 jx R2 …jx RN N can be selected within a preset range as needed to extract the number of image features.

[0115] Continue to refer to Figure 2 The data generation method for augmented reality content in this embodiment further includes:

[0116] Step S103: Configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information.

[0117] In step S103, in order to match the frame image information of the virtual scene video with the visual map configuration so that the virtual scene in the frame image can be located on the visual map, it is necessary to first configure the image features of the frame image with the image features of the visual map one by one, and then map the three-dimensional point cloud data corresponding to the image features of the visual map with configuration relationship to the image features of the virtual scene frame image to obtain the map configuration data of the relevant virtual objects in the virtual scene video frame image on the visual map.

[0118] If the virtual scene video is captured based on entities in the target scene, then the configuration of the aforementioned image features can be image feature matching. If the virtual scene video is created dynamically by computer based on virtual objects, then the configuration of the aforementioned image features can be actively configured by the computer, that is, using the reference relationship relative to the visual map during creation, the image features of the virtual scene video frames are configured with the image features of the visual map by human intervention or computer intervention.

[0119] Based on the above configuration approach, step S103, which involves configuring the video frame information into the visual map information to form a configuration relationship between the video frame information and the visual map information, includes:

[0120] Configure the second image feature set based on the first image feature set to form a configuration relationship between the second image feature set and the first image feature set;

[0121] Based on the configuration relationship between the second image feature set and the first image feature set, the first three-dimensional point cloud coordinate set corresponding to the first image feature set is used as the second three-dimensional point cloud coordinate set.

[0122] Based on the first image feature set J1, J2, J3...J m The first three-dimensional point cloud coordinate set S1, S2, S3…S m and the second image feature set J x1 J x2 J x3 ...J x1R The second image feature set J can be obtained through configuration relationships. x1 J x2 J x3 ...J x1R J in each image feature element x1 {jx 11 jx 12 …jx 1N}、J x2 {jx 21 jx 22 …jx 2N}、J x3 {jx 31 jx 32 …jx 3N}、…、J x1R {jx R1 jx R2 …jx RN The first image feature set J1{j} corresponds to 11 ,j 12 …j 1N}、J2{j 21 ,j 22 …j 2N}、J3{j 31 ,j 32 …j 3N}…J m {j m1 ,j m2 …j mN Image feature elements J'1{j' in} 11 ,j' 12 …j' 1N}、J'2{j' 21 ,j' 22 …j' 2N}、J'3{j' 31 ,j' 32 …j' 3N}…J' R {j' R1,j' R2 …j' RN}, and from the first three-dimensional point cloud coordinate set S1, S2, S3…S m Image feature elements J'1{j' 11 ,j' 12 …j' 1N}、J'2{j' 21 ,j' 22 …j' 2N}、J'3{j' 31 ,j' 32 …j' 3N}…J' R {j' R1 ,j' R2 …j' RN The corresponding 3D point cloud coordinate set S'1, S'2, S'3…S' R As the second image feature set J x1 J x2 J x3 ...J x1R The second three-dimensional point cloud coordinate set S corresponding to the image feature elements in the image. x1 S x2 S x3 …S xR .

[0123] The above steps in this embodiment can construct the configuration relationship between the image features of video frame images and the image feature set in the visual map information, thereby obtaining the 3D point cloud data of the virtual scene video frame images in the visual map and constructing the positional configuration information of virtual scene objects in the visual map. The video frame information formed in this step not only includes the second image feature set J acquired based on the virtual scene video frame images. x1 J x2 J x3 ...J x1R It also includes a second three-dimensional point cloud coordinate set S generated based on the configuration relationship with visual map information. x1 S x2 S x3 …S xR .

[0124] Continue to refer to Figure 2 The data generation method for augmented reality content in this embodiment further includes:

[0125] Step S104: Configure the second placement scene image information and the second placement content information of the augmented reality content based on the video frame information. The second placement content information is adapted to be displayed in the virtual scene video that has a configuration relationship with the visual map information according to the specified second placement scene image information.

[0126] Specifically, the second placement scene image information includes: second image positioning data associated with the second image feature set and the second three-dimensional point cloud coordinate set; and the second placement content information is configured corresponding to the second image positioning data.

[0127] To achieve the integration of offline augmented reality content and online mixed reality content, in this embodiment, the second placement content information and the first placement content information can be identical. Since the first image positioning data of the first placement scene image information in the above steps of this embodiment includes a first image feature set J0' related to entity image feature J', corresponding 3D point cloud coordinates S0', and entity pose corresponding to the first placement content information, a preset configuration of the rendering position of the first placement content information according to different entity poses is obtained under external input positioning data. Because the second image feature set J0'... x1 J x2 J x3 ...J x1R With the first image feature set J1, J2, J3…J m With configuration relationships, virtual entity image feature set J x Based on the configuration relationship with the entity image feature J, the 3D point cloud coordinates S0' and the entity pose corresponding to the first placed content information are obtained, which are used as the virtual entity image feature set J. x The corresponding 3D point cloud coordinates S x0 ', and based on this, generate the virtual entity pose corresponding to the second placement content information and the video frame rendering position in the virtual scene.

[0128] Of course, in other embodiments, depending on the setup requirements, the second placement content information can be partially identical to the first placement content information, or frame images and AR content that are different from the first placement content information can be set separately as needed. In this case, it is necessary to obtain image feature sets J from different orientations of the virtual entity. x0 ', and similarly, the image feature set J x0 'With the second image feature set J x1 J x2 J x3 ...J x1R Perform matching to obtain the matched image feature set J x And from the second three-dimensional point cloud coordinate set S x1 S x2 S x3 …S xR Acquiring and Image Feature Sets J x The 3D point cloud coordinates S corresponding to the image features x0The PNP algorithm is used to obtain the virtual entity poses for different placement of the second content. Finally, the virtual entity poses for placing the second content in different orientations of the virtual entity are obtained.

[0129] In step S104, the second placement content information can be all or part of the first placement content information, or it can include content information that is different from the first placement content information.

[0130] The first placement content information is placed on the visual map based on the first image positioning data and external source positioning data. The first image positioning data includes: image features J' of the entity in different orientations, a three-dimensional point cloud coordinate set S0', and the entity pose, and the rendering position of the first placement content information can be determined based on the entity pose.

[0131] When the second placement content information is the same as the first placement content information, the second image positioning data of the second placement scene image information is obtained based on the first image positioning data according to the configuration relationship: the second placement content is placed in the virtual scene video frame according to the first image positioning data, the second image positioning data, and external input positioning data. The second image positioning data includes: image features J of virtual entities in different orientations. x0 (Obtained based on entity image features J), 3D point cloud coordinates S x0 'And the virtual entity pose, the second placement content information can determine the rendering position based on the virtual entity pose. Because the first image positioning data and the second image positioning data have a configuration relationship in image features and 3D point cloud data, the integration of offline augmented reality content and online mixed reality content can be realized.'

[0132] When the second placement content information is different from the first placement content information, the second placement scene image information corresponding to this part of the second placement content only contains the second image positioning data. The second placement content is placed in the virtual scene video frame according to the second image positioning data and external input positioning data to realize the personalization of online mixed content.

[0133] Continue to refer to Figure 2 The data generation method for augmented reality content in this embodiment further includes:

[0134] Step S105: Generate the augmented reality content data based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information.

[0135] Through the above steps S100 to S105, data capable of realizing an integrated online and offline augmented reality experience is constructed. The augmented reality content data constructed in this embodiment includes at least:

[0136] Images of the target scene P1, P2, P3…P mThe first image feature set J1, J2, J3…J m and the corresponding first three-dimensional point cloud coordinate set S1, S2, S3…S m ;

[0137] The first image localization data includes: entity image features J', corresponding 3D point cloud coordinates S0', and entity poses corresponding to different entity orientations of the content to be placed.

[0138] First placement of content information;

[0139] Virtual scene video frame images X1, X2, X3…X R The second image feature set J x1 J x2 J x3 ...J x1R , and the second image feature set J x1 J x2 J x3 ...J x1R The first image feature set J'1, J'2, J'3…J' with configuration relationship R ; and, the first image feature set J'1, J'2, J'3…J' R The corresponding first three-dimensional point cloud coordinate set S'1, S'2, S'3…S' R It also serves as the second three-dimensional point cloud coordinate set S x1 S x2 S x3 …S xR ;

[0140] The second image localization data includes: virtual entity image features J x0 ', 3D point cloud coordinates S x0 'and the pose of different virtual entities where the content needs to be placed;

[0141] The second part contains the content information.

[0142] When the second placement content information is the same as the first placement content information, the second image positioning data and the first image positioning data have a configuration relationship, specifically including: virtual entity image feature J' which has a configuration relationship with entity image feature J'. x0 ', 3D point cloud coordinates S x0 'And the poses of different virtual entities where the content needs to be placed, so as to achieve the integration of offline augmented reality content and online mixed reality content.'

[0143] Based on the above embodiments, the technical solution of the present invention provides a corresponding data generation system for augmented reality content, including:

[0144] The first acquisition unit is adapted to acquire visual map information of the target scene to execute step S100;

[0145] The first configuration unit is adapted to configure first placement scene image information and first placement content information of augmented reality content based on the visual map information. The first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information to execute step S101.

[0146] The second acquisition unit is adapted to acquire video frame information of the virtual scene in order to execute step S102;

[0147] The second configuration unit is adapted to configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information, so as to execute step S103.

[0148] The third configuration unit is adapted to configure the second placement scene image information and the second placement content information of the augmented reality content based on the video frame information. The second placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified second placement scene image information, so as to execute step S104.

[0149] The storage unit is adapted to generate data of the augmented reality content based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information, so as to execute step S105.

[0150] Based on the above embodiments, the present invention also provides another data generation system for augmented reality content, including: a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, it performs steps S100 to S105 to form the data generation method for augmented reality content.

[0151] Based on the above embodiments, the present invention also provides a corresponding computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of steps S100 to S105 of the data generation method for composing augmented reality content.

[0152] A variation of the above embodiments, such as Figure 3 As shown, the present invention also provides a method for generating augmented reality content data, comprising the following steps:

[0153] Step S200: Obtain visual map information of the target scene.

[0154] The visual map information in step S200 includes: the target image feature set and the target 3D point cloud coordinate set corresponding to the captured image of the target scene. The method for obtaining the target image feature set and the target 3D point cloud coordinate set in this step can refer to step S100.

[0155] Step S201: Obtain video frame information of the virtual scene.

[0156] In step S201, the method for acquiring video frame information of the virtual scene can be referred to in step S102, and will not be repeated here.

[0157] Step S202: Configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information.

[0158] The video frame information of the virtual scene includes: virtual scene video frame images, virtual image feature sets, and virtual 3D point cloud coordinate sets. By acquiring the virtual scene video frame image feature sets, the video frame information is configured within the visual map information to form a configuration relationship between the video frame information and the visual map information. Specifically, based on the configuration method in step 103, the target image feature set and the virtual image feature set can be configured to form a configuration relationship between the virtual image feature set and the target image feature set. Next, the target 3D point cloud coordinate set corresponding to the target image feature set that has a configuration relationship with the virtual image feature set can be used as the virtual 3D point cloud coordinate set. The above configuration method can be specifically referred to in step S103. Based on the configuration relationship between the virtual scene image feature set and the target image feature set, the target 3D point cloud coordinate set corresponding to the target image feature set is used as the virtual scene 3D point cloud coordinate set. Finally, a configuration relationship between the video frame information and the visual map information is formed.

[0159] Step S203: Configure the placement scene image information and placement content information of augmented reality content based on the video frame information. The placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified placement scene image information.

[0160] Similar to steps S101 and S104, the placement scene image information in this embodiment includes image positioning data, and the placement content information is configured corresponding to the virtual image positioning data.

[0161] Unlike steps S101 and S104, the augmented reality content data in this embodiment is based on the configuration relationship between visual map information and video frame information, and only general image positioning data and placement content information are configured. Referring to the above embodiments, the placement content information can be frame images of AR video stream data and AR content. The image positioning data is generated based on the target image feature set and the corresponding target 3D point cloud coordinate set. The object pose is obtained based on the image features of objects in different orientations of the target scene or virtual scene, and the placement content information for the corresponding rendering position is configured based on this pose. When an external image pose is input, content is placed at the corresponding rendering position when the input pose matches the object pose.

[0162] Step S204: Generate the augmented reality content data based on the placement scene image information and placement content information.

[0163] Through the above steps S200 to S204, data capable of realizing an integrated online and offline augmented reality experience is constructed. The augmented reality content data constructed in this embodiment includes at least:

[0164] The target image feature set and the target 3D point cloud coordinate set corresponding to the captured images of the target scene;

[0165] Virtual scene image feature set and 3D point cloud coordinate set of virtual scene image features from video frame images;

[0166] Placement scene image information, including: general image positioning data; and,

[0167] Place content information.

[0168] Corresponding to the above embodiments, the present invention also provides a data generation system for augmented reality content, comprising:

[0169] The first acquisition unit is adapted to acquire visual map information of the target scene in order to perform the above step S200;

[0170] The second acquisition unit is adapted to acquire video frame information of the virtual scene in order to perform the above step S201;

[0171] The first configuration unit is adapted to configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information, so as to perform the above step S202.

[0172] The second configuration unit is adapted to configure the placement scene image information and placement content information of augmented reality content based on the video frame information. The placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified placement scene image information, so as to perform the above step S203.

[0173] The storage unit is adapted to generate data of the augmented reality content based on the placement scene image information and placement content information, so as to perform the above step S204.

[0174] Corresponding to the above embodiments, the present invention also provides a data generation system for augmented reality content, including: a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, it executes the steps of the data generation method for augmented reality content consisting of steps S200 to S204.

[0175] Corresponding to the above embodiments, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the augmented reality content data generation method consisting of steps S200 to S204.

[0176] Based on the above embodiments, the technical solution of the present invention also provides, as follows: Figure 4 The method for displaying augmented reality content, shown below, is based on augmented reality content data provided by the method comprising steps S100 to S105, and includes the following steps:

[0177] Step S300: Acquire captured image information.

[0178] Let the acquired image information be P0. The system automatically collects the corresponding image features J for the acquired image information P0. p J p For {j p1 ,j p2 …j pN}

[0179] Step S301: Determine the location information of the image capturing device in the target scene based on the visual map information and the captured image information.

[0180] Based on the visual map information provided by the method consisting of steps S100 to S105, the captured image features J p The first image feature set J1{j with visual map information 11 ,j 12 …j 1N}、J2{j 21 ,j 22 …j 2N}、J3{j 31 ,j 32 …j 3N}…J m {j m1 ,j m2 …j mN} Perform matching to obtain the first image feature set and the captured image features J p Matching image feature set J p '.

[0181] Based on image feature set J p From the first three-dimensional point cloud coordinate set S1, S2, S3…S m Acquiring and Image Feature Set J p 'The 3D point cloud coordinates S corresponding to the image features' p Based on image feature set J p 'and the three-dimensional point cloud coordinates S p The pose of the image capturing device in the target scene is obtained through the PNP algorithm.

[0182] Step 302: When the positioning information is associated with the first placement scene image information, the first placement content information is displayed according to the specified first scene image information.

[0183] Since the first placement scene image information contains entity image features J' in the visual map, the corresponding three-dimensional point cloud coordinates S0', and entity poses corresponding to different entity orientations of the content to be placed, based on the pose data of the current capturing image device obtained in step S301, the system matches the device pose with the entity pose in the first placement scene image information. If the match is successful, it is considered that the positioning information is associated with the first placement scene image information. At this time, the system will load the corresponding first placement content information in the entity scene according to the rendering position corresponding to the entity pose.

[0184] Step S303: Load the video frame information of the virtual scene.

[0185] Since a virtual scene is a series of pre-shot video streams or panoramic videos, the system will load the video content of the virtual scene according to presets or user selections to form video frame information, including images of the virtual scene.

[0186] Step S304: When the loaded video frame information is associated with the second placement scene image information, the second placement content information is displayed according to the specified second scene image information.

[0187] When loading corresponding video frames according to presets or user selections, the loading of video frames involving virtual entity images is based on the configuration relationship between video frame information and visual map information in image features and 3D point cloud data. The system can automatically load the current virtual entity video frame image u and its image features J. u Based on the video frame information with configuration relationship to the visual map information provided by the method consisting of steps S100 to S105, the current virtual video frame image features J can be obtained. uThe pose in the visual map coordinate system. Specifically, the current virtual video frame image features J u With the second image feature set J x1 J x2 J x3 ...J x1R Perform matching to obtain the virtual entity image features J from the second image feature set. u Matching image feature set J u '.

[0188] Based on image feature set J u From the second three-dimensional point cloud coordinate set S x1 S x2 S x3 …S xR Obtain the image features J of the current virtual video frame u The corresponding 3D point cloud coordinate set S u Based on the current virtual video frame image features J u and the corresponding 3D point cloud coordinate set S u The pose of the current virtual video frame in the target scene is obtained by using the PNP algorithm.

[0189] Similar to step S302, after detecting the pose of the current virtual video frame in the target scene, the system matches the device pose with the virtual entity pose in the second placement scene image information. If the match is successful, the system considers the positioning information to be associated with the second placement scene image information. At this time, the system will load the corresponding second placement content information in the virtual scene according to the pose of the current virtual video frame in the virtual scene and based on the corresponding rendering position.

[0190] Corresponding to the above embodiments, the present invention also provides a data display system for augmented reality content, comprising:

[0191] The third acquisition unit is adapted to acquire captured image information in order to perform the above step S300;

[0192] The positioning unit is adapted to determine the positioning information of the image capturing device in the target scene based on the visual map information and the captured image information, so as to perform the above step S301;

[0193] The first display unit is adapted to display the first placement content information according to the specified first scene image information when the positioning information is associated with the first placement scene image information, so as to perform the above step S302;

[0194] The second display unit is adapted to load video frame information of the virtual scene in order to perform the above step S303;

[0195] The third display unit is adapted to display the second placement content information according to the specified second scene image information when the loaded video frame information is associated with the second placement scene image information, so as to perform the above step S304.

[0196] Corresponding to the above embodiments, the present invention also provides a data display system for augmented reality content, including: a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, it executes the steps of the data display method for augmented reality content consisting of steps S300 to S304.

[0197] Corresponding to the above embodiments, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the augmented reality content display method consisting of steps S300 to S304.

[0198] Based on the above embodiments, the technical solution of the present invention also provides, as follows: Figure 5 The illustrated method for displaying augmented reality content, based on augmented reality content data provided by the method comprising steps S200 to S204, includes the following steps:

[0199] Step S400: Load the video frame information of the virtual scene.

[0200] Step S401: When the loaded video frame information is associated with the placement scene image information, the placement content information is displayed according to the specified placement scene image information.

[0201] Unlike the augmented reality content data display method consisting of steps S300 to S304, steps S400 to S401 construct augmented reality content data based on the method consisting of steps S200 to S204, realizing the display of video augmented content in a virtual scene, and can be integrated with any offline real-world augmented content. When loading corresponding video frames according to presets or user selection, the loading of video frames involving virtual entity images, based on the configuration relationship between video frame information and visual map information in image features and 3D point cloud data, the system can automatically load the current virtual entity video frame image u and its image features J. u Based on the video frame information and general image positioning data provided by the method consisting of steps S200 to S204, which have a configuration relationship with the visual map information, the current virtual video frame image features J can be obtained. uThe pose in the visual map coordinate system. Similar to step S304, after detecting the pose of the current virtual video frame in the target scene, the system matches the pose of the current video frame with the preset virtual entity pose. If the match is successful, it is considered that the positioning information is associated with the placement scene image information. At this time, the system will load the corresponding placement content information in the virtual scene according to the pose of the current virtual video frame in the virtual scene and based on the corresponding rendering position.

[0202] Corresponding to the above embodiments, the present invention also provides a data display system for augmented reality content, comprising: a first display unit adapted to load video frame information of a virtual scene to execute the above step S400; and a second display unit adapted to display the placement content information according to the specified placement scene image information when the loaded video frame information is associated with the placement scene image information, to execute the above step S401.

[0203] Corresponding to the above embodiments, the present invention also provides a data display system for augmented reality content, including: a processor and a memory; the memory stores a computer program, and when the processor executes the computer program, it executes the steps of the data display method for augmented reality content consisting of steps S400 to S401.

[0204] Corresponding to the above embodiments, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the augmented reality content display method consisting of steps S400 to S401.

[0205] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for generating augmented reality content, characterized in that, include: Obtain visual map information of the real-world target scene; The visual map information includes: a first image feature set and a first three-dimensional point cloud coordinate set corresponding to the target scene captured image; Based on the visual map information, configure the first placement scene image information and the first placement content information of augmented reality content. The first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information. The first placement scene image information includes: first image positioning data associated with the first image feature set and the first three-dimensional point cloud coordinate set; the first placement content information is configured corresponding to the first image positioning data; The first placement scene image information further includes: entity pose data; the first placement scene image information and the first placement content information configured based on the visual map information further include: The rendering position of the content to be placed is determined based on the entity pose data; Match the determined rendering position with the first placement content information; The first placement content information is adapted to be displayed on the visual map according to the matched rendering position; Obtain video frame information from online virtual scenes; The video frame information is configured in the visual map information to form a configuration relationship between the video frame information and the visual map information; The video frame information includes: a second image feature set and a second three-dimensional point cloud coordinate set corresponding to the virtual scene video frame image; configuring the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information includes: Configure the second image feature set based on the first image feature set to form a configuration relationship between the second image feature set and the first image feature set; Based on the configuration relationship between the second image feature set and the first image feature set, the first three-dimensional point cloud coordinate set corresponding to the first image feature set is used as the second three-dimensional point cloud coordinate set; Based on the video frame information, configure the second placement scene image information and the second placement content information of the augmented reality content. The second placement content information is suitable for displaying the virtual scene video that has a configuration relationship with the visual map information according to the specified second placement scene image information. The augmented reality content data is generated based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information.

2. The data generation method for augmented reality content as described in claim 1, characterized in that, The first placement scene image information and the first placement content information configured based on the visual map information include: Select the entity in the target scene where the first content to be placed needs to be displayed; Obtain entity image features of the target scene based on the selected target scene entities; The entity image feature set is matched with the first image feature set to obtain the matched first image feature set and the corresponding three-dimensional point cloud coordinate set; Based on the entity image feature set and the corresponding 3D point cloud coordinate set, the entity pose data of the content to be placed is obtained through the PNP algorithm.

3. The data generation method for augmented reality content as described in claim 1, characterized in that, The second placement scene image information includes: second image positioning data associated with the second image feature set and the second three-dimensional point cloud coordinate set; the second placement content information is configured corresponding to the second image positioning data.

4. The data generation method for augmented reality content as described in claim 3, characterized in that, The second image positioning data is derived from the first image positioning data.

5. The data generation method for augmented reality content as described in claim 1, characterized in that, The first placement scene image information includes: entity image feature set, entity image three-dimensional point cloud coordinate set, and first image positioning data; the second placement scene image information includes: virtual entity image feature set, virtual entity three-dimensional point cloud coordinate set, and second image positioning data that have a configuration relationship with the first placement scene image information.

6. The data generation method for augmented reality content as described in claim 1, characterized in that, Obtaining visual map information of the target scene includes: Acquire images of the target scene; Image features are extracted from images captured for each target scene to obtain the first image feature set for each target scene image; The extracted image features are processed using the structure-of-motion algorithm to obtain the first three-dimensional point cloud coordinate set of each target scene image.

7. A method for displaying augmented reality content, based on augmented reality content data generated according to any one of claims 1 to 6, characterized in that, include: Acquire captured image information; The location information of the image capturing device in the target scene is determined based on the visual map information and the captured image information; When the positioning information is associated with the first placement scene image information, the first placement content information is displayed according to the specified first scene image information; Load video frame information of the virtual scene; When the loaded video frame information is associated with the second placement scene image information, the second placement content information is displayed according to the specified second scene image information.

8. A data generation system for augmented reality content, characterized in that, include: The first acquisition unit is suitable for acquiring visual map information of real offline target scenes; The visual map information includes: a first image feature set and a first three-dimensional point cloud coordinate set corresponding to the target scene captured image; The first configuration unit is adapted to configure first placement scene image information and first placement content information of augmented reality content based on the visual map information, wherein the first placement content information is adapted to be displayed on the visual map according to the specified first placement scene image information. The first placement scene image information includes: first image positioning data associated with the first image feature set and the first three-dimensional point cloud coordinate set; the first placement content information is configured corresponding to the first image positioning data; The first placement scene image information also includes: Entity pose data The first placement scene image information and the first placement content information configured based on the visual map information also include: The rendering position of the content to be placed is determined based on the entity pose data; Match the determined rendering position with the first placement content information; The first placement content information is adapted to be displayed on the visual map according to the matched rendering position; The second acquisition unit is suitable for acquiring video frame information of online virtual scenes; The second configuration unit is adapted to configure the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information. The video frame information includes: a second image feature set and a second three-dimensional point cloud coordinate set corresponding to the virtual scene video frame image; configuring the video frame information in the visual map information to form a configuration relationship between the video frame information and the visual map information includes: Configure the second image feature set based on the first image feature set to form a configuration relationship between the second image feature set and the first image feature set; Based on the configuration relationship between the second image feature set and the first image feature set, the first three-dimensional point cloud coordinate set corresponding to the first image feature set is used as the second three-dimensional point cloud coordinate set; The third configuration unit is adapted to configure second placement scene image information and second placement content information of augmented reality content based on the video frame information. The second placement content information is adapted to be displayed in the video frame information that has a configuration relationship with the visual map information according to the specified second placement scene image information. The storage unit is adapted to generate data of the augmented reality content based on the first placement scene image information and the first placement content information, the second placement scene image information and the second placement content information.

9. A data display system for augmented reality content, based on the data generation system for augmented reality content as described in claim 8, characterized in that, include: The third acquisition unit is suitable for acquiring captured image information; The positioning unit is adapted to determine the positioning information of the imaging device in the target scene based on the visual map information and the captured image information; The first display unit is adapted to display the first placement content information according to the specified first scene image information when the positioning information is associated with the first placement scene image information; The second display unit is suitable for loading video frame information of the virtual scene; The third display unit is adapted to display the second placement content information according to the specified second scene image information when the loaded video frame information is associated with the second placement scene image information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data generation method for augmented reality content as described in any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the augmented reality content display method as described in claim 7.

Images