A method and system for rendering large space AR content

Abstract

The application relates to a rendering method of large-space AR content. The large space is divided into space units in multiple levels. Real-time and predicted positions of a user are acquired to add state marks of an instant location, a potential reachable location and an unreachable location. A differentiated loading strategy under each unit is configured according to the state marks, and then large-space rendering is performed. Through hierarchical space management and a dynamic loading strategy, the method realizes accurate matching of virtual assets and space where the user is located and potential reachable space, significantly reduces the amount of redundant asset loading, reduces terminal resource consumption, avoids picture discontinuity, greatly improves the rendering efficiency and smoothness of large-space AR content, and enhances the immersion and interaction stability of virtual-real fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality, and in particular, to a large space AR content rendering method and system, a computer device, and a computer readable storage medium. BACKGROUND

[0002] With the rapid development of augmented reality (AR) technology, large space AR applications have shown great potential in commercial display, cultural tourism, education and training, etc. Such applications usually require the implementation of immersive experiences that integrate virtual and real spaces in physical spaces of hundreds to thousands of square meters, which poses a serious challenge to AR content rendering technology.

[0003] In related technologies, the current large space AR rendering scheme adopts a whole packaging method, that is, all virtual assets (including 3D models, maps, animations, etc.) in the scene are packaged into a complete running file. The advantage of this scheme is that it is simple to implement and all resources are loaded at once; however, the data volume is too large, and all resources need to be loaded into memory at runtime, resulting in high memory occupation, which seriously affects the performance of mobile devices, causing problems such as lag, heating, and even crashes.

[0004] In addition, a hot update mechanism can also be used for large space AR rendering. This scheme can optimize the running file by reducing the package size and dynamically loading and unloading resources. However, during the hot update process, the user cannot normally use or experience the system until the resources are downloaded; in addition, due to the lack of intelligent resource management strategies, the problems of "loading not in time" or "unloading too early" often occur.

[0005] Therefore, there is a need for a large space AR content rendering method that can achieve more refined space division and more intelligent resource scheduling. SUMMARY

[0006] Embodiments of the present application provide a large space AR content rendering method, device, system, computer device, and computer readable storage medium to at least solve the problem of low efficiency and high runtime memory occupation caused by managing the entire large space scene in related technologies.

[0007] In a first aspect, embodiments of the present application provide a large space AR content rendering method, characterized by being applied to AR content rendering in a large space, and the method comprises:

[0008] In the preprocessing process, the large space is segmented into multiple levels to obtain multiple levels of initial sub-space units, and the initial sub-space units are optimized based on the user's set experience area to obtain sub-space units and establish an association relationship between the sub-space units and their corresponding virtual resources, with the optimization objectives of improving spatial accuracy and minimizing the number of spaces.

[0009] applying an execution flow to add state marks to the sub-space units according to the real-time location and the predicted location of the user, the state marks including: an instant state, a potentially reachable state, and an unreachable state,

[0010] configuring differentiated loading and caching strategies for the sub-space units respectively according to the state marks, and rendering the AR content of the large space according to the loading and caching strategies and the virtual resources corresponding to the sub-space units.

[0011] In some embodiments, the multi-level segmentation of the large space to obtain the initial sub-space units of multiple levels includes:

[0012] identifying and parsing the CAD file corresponding to the large space through an intelligent platform to obtain the physical space hierarchical structure information of the large space;

[0013] preliminarily segmenting the large space according to the space hierarchical structure information to form the initial sub-space units of multiple levels;

[0014] In the process of the preliminary segmentation, the size of the initial sub-space unit is dynamically adjusted according to the space range occupied by the preset virtual assets in the initial sub-space unit.

[0015] In some embodiments, the multi-level segmentation of the large space to obtain the initial sub-space units of multiple levels includes:

[0016] preliminarily segmenting the large space by using a K-D tree algorithm to form the initial sub-space units of multiple levels, wherein each initial sub-space unit is a multi-dimensional asymmetric space;

[0017] obtaining the space overlap scenarios existing in the large space, and modifying the area range of the initial sub-space unit associated with the overlap scenarios according to the moving track of the user in the AR experience process.

[0018] In some embodiments, in the process of the multi-level segmentation of the large space to be processed, the method further includes:

[0019] marking the boundary range of each initial sub-space unit through a cubic framework based on a collision box;

[0020] After the sub-space units with the marked boundary range are displayed, the initial sub-space units are optimized according to the operation instruction of the user.

[0021] In some embodiments, the optimization of the sub-space units in combination with the set experience area of the user includes:

[0022] acquire a set action track or a passable area of the user in the large space;

[0023] According to the set action track or the passable area, and the intersection with the initial sub-space unit, the initial sub-space unit is adjusted to obtain a sub-space unit, wherein the adjustment includes merging adjacent sub-space units, deleting redundant sub-space units, and adjusting the boundary range of the sub-space unit.

[0024] The adjusted sub-space unit is associated with the corresponding virtual resource to form a hierarchical space set.

[0025] The optimization adjustment target includes reducing the loading error of virtual assets by improving the space segmentation accuracy, and reducing the total number of sub-space units to reduce the runtime memory occupation.

[0026] In some embodiments, the instant state is used to mark the sub-space unit currently occupied by the user, and the virtual resources in the currently occupied sub-space unit match the real-time loading strategy.

[0027] The potential reachable state is used to mark the sub-space unit that can be reached in the future within a preset period of time, and the virtual resources in the sub-space unit that can be reached in the future within a preset period of time match the preloading strategy.

[0028] The unreachable state is used to mark the sub-space unit that exceeds the current movement ability of the user, and the virtual resources in the sub-space unit that exceeds the current movement ability of the user match the delayed unloading strategy.

[0029] The dynamic update of the state marking is based on at least one of the following factors: real-time position coordinates of the user, movement speed vector of the user, field of view direction of the user, and topological connection relationship between sub-space units.

[0030] In some embodiments, according to the state marking, the loading strategy is configured for the sub-space unit respectively, including:

[0031] For the sub-space unit of the instant state, all virtual resources associated with it are loaded into the running memory, and the high-precision rendering state of the virtual resources is maintained, and the relative position relationship between the virtual resources and the user's perspective is updated in real time.

[0032] For the sub-space unit of the potential reachable state, the preloading priority is calculated according to the movement speed and direction of the user, and part of the virtual resources of the sub-space unit are loaded into the cache space according to the preloading priority.

[0033] For the sub-space unit in the unreachable state, the loaded virtual resource is marked as a to-be-released state, and the virtual resource is released according to the memory pressure condition.

[0034] In some embodiments, according to the loading and caching strategy and the virtual resource corresponding to the sub-space unit, rendering the large-space AR content comprises:

[0035] For the virtual assets in the sub-space unit, the virtual assets are dynamically sorted based on the priority of the virtual assets, and the virtual assets are loaded according to the dynamically sorted priority;

[0036] The priority of the virtual assets is based on at least one of the following factors: user interaction weight, visual saliency, and computing resource consumption.

[0037] In a second aspect, the embodiments of the present application provide a large-space AR content rendering system, which is applied to AR content rendering in a large space, and the system comprises a preprocessing module and an execution module, wherein:

[0038] The preprocessing module is configured to divide a large space into multiple levels to obtain initial sub-space units of multiple levels, and optimize the initial sub-space units based on a user's set experience area to obtain sub-space units, so as to improve spatial accuracy and minimize the number of spaces, and establish an association relationship between the sub-space units and corresponding virtual resources;

[0039] The execution module is configured to add state marks to the sub-space units according to a real-time position and a predicted position of a user, wherein the state marks comprise an instant state, a potential reachable state, and an unreachable state.

[0040] According to the state marks, the sub-space units are respectively configured with differentiated loading and caching strategies, and according to the loading and caching strategy and the virtual resource corresponding to the sub-space unit, the large-space AR content is rendered.

[0041] In a third aspect, the embodiments of the present application provide a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the method of the first aspect.

[0042] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the method of the first aspect.

[0043] Compared with the related art, the rendering method of the large-space AR content provided by the embodiment of the application performs multi-level segmentation on the large space to form space units, adds state marks of being in place, being potentially reachable and being unreachable by obtaining real-time and predicted positions of a user, configures a differentiated loading strategy under each unit, and then performs large-space rendering. Through hierarchical space management and a dynamic loading strategy, the method realizes accurate matching of virtual assets and spaces where the user is and is potentially reachable, significantly reduces the amount of redundant asset loading, reduces terminal resource consumption, avoids picture discontinuity, greatly improves the rendering efficiency and smoothness of the large-space AR content, and enhances the immersion and interactive stability of virtual-real fusion. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings used to provide further understanding of the application and form a part of the application, and the illustrative embodiments of the application and the description thereof are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0045] Figure 1 is a flowchart of a rendering method of a large-space AR content according to an embodiment of the application;

[0046] Figure 2 is a schematic diagram of a space division result based on a physical space according to an embodiment of the application;

[0047] Figure 3 is a schematic diagram of an optimized interval segmentation result according to an embodiment of the application;

[0048] Figure 4 is a schematic diagram of an optimized sub-space unit according to an embodiment of the application;

[0049] Figure 5 is a structural block diagram of a rendering system of a large-space AR content according to an embodiment of the application;

[0050] Figure 6 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the application more clear, the application is described and explained below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application. Based on the embodiments provided by the application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the application.

[0052] It is apparent that the drawings in the following description merely show some examples or embodiments of the present application, and for those skilled in the art, the present application can be applied to other similar situations without creative labor based on these drawings. In addition, it can be understood that although the efforts made in this development process can be complex and lengthy, some design, manufacture or production changes based on the technical content disclosed in the present application are only routine technical means for those skilled in the art related to the disclosure of the present application, and should not be understood as insufficient disclosure of the present application.

[0053] In the present application, the phrase "embodiment" means that the specific features, structures or properties described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in the present application should be understood as the usual meaning understood by those skilled in the art in the technical field to which the present application belongs. The terms "one", "a", "an", "the", and similar words involved in the present application do not represent quantity limitation, but can represent singular or plural. The terms "include", "contain", "have", and any variations thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but can also include steps or units not listed, or can also include other steps or units inherent to the process, method, product or device. The terms "connected", "connected", "coupled" and similar words involved in the present application are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" in the present application means two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third" and the like involved in the present application are only to distinguish similar objects, and do not represent a specific order for the objects.

[0055] The embodiment of the present application provides a rendering method of large space AR content, Figure 1 is a flowchart of a rendering method of large space AR content according to the embodiment of the present application, as shown in Figure 1As shown, the flow includes the following steps:

[0056] S101, in the preprocessing flow, the large space is segmented into multiple levels to obtain multiple levels of initial sub-space units, and based on the user's setting experience area, the sub-space unit is optimized to improve the spatial precision and minimize the space quantity as the optimization target, and the sub-space unit and its corresponding virtual resource are associated;

[0057] In one example embodiment, step S101 can provide a data basis for subsequent virtual asset dynamic management through space segmentation and optimization, and the core logic is to combine the space segmentation capability of K-D tree algorithm with the actual needs of user experience scene, to build a multi-level sub-space system that meets the spatial structure characteristics and adapts to user interaction trajectory.

[0058] Specifically, this step first uses the multi-dimensional space segmentation characteristics of K-D tree algorithm to disassemble the large space (such as building, supermarket, museum, etc.) as a whole into initial sub-space units with hierarchical relationship (such as "building-floor-public area-room" level), realizing the structured splitting of space;

[0059] On this basis, in view of the actual scenes such as space overlap and user trajectory limitation existing in AR experience process, taking "improving spatial precision" (ensuring that the sub-space can accurately match the user's visual range and interaction area) and "minimizing space quantity" (reducing resource management complexity) as the target, and combining the user's set experience area (such as passable range, key interaction area), the initial sub-space is adjusted, and finally the sub-space unit adapting to the actual experience scene is formed.

[0060] In step S101, first, based on the K-D tree algorithm, the large space is segmented into multiple levels to obtain multiple levels of initial sub-space units, including the following specific steps:

[0061] S1, using K-D tree algorithm to preliminarily segment the large space to form multiple levels of initial sub-space units, wherein each initial sub-space unit is a multi-dimensional asymmetric space;

[0062] Among them, K-D tree algorithm is a tree data structure for multi-dimensional space data division, and its core is to divide the space by recursively selecting coordinate axes (such as x, y, z axes in three-dimensional space), and finally form multiple initial sub-space units.

[0063] In the large space AR scene, the specific operation of this step is as follows:

[0064] Hierarchy setting: according to the physical structure of the large space and the AR experience demand, the hierarchy relationship of the subspaces is preset, for example, "Building-Level-Area-Room", each hierarchy corresponds to a different scale of space range (for example, "Building" is the largest hierarchy, and "Room" is the smallest hierarchy);

[0065] Multi-dimensional asymmetric partitioning: Unlike the symmetric partitioning of traditional K-D trees, this step combines the actual structure of the large space, such as irregular room layout and irregular public area, to perform asymmetric partitioning, i.e., the shape and size of the subspaces at the same hierarchy can be flexibly adjusted according to the physical space characteristics, for example, a narrow corridor and a square exhibition hall can be used as subspaces at the same hierarchy with different shapes;

[0066] Initial subspace generation: through recursive partitioning by the K-D tree algorithm, a plurality of initial subspace units with a hierarchy relationship are finally formed, each unit is a multi-dimensional (such as a three-dimensional space) asymmetric structure, and the hierarchy relationship of the subspace units corresponds to the physical / logical structure (such as the spatial hierarchy of the building) of the large space.

[0067] Figure 2 is a space partitioning result diagram based on a physical space according to an embodiment of the present application, as shown in Figure 2 Each partitioned space is independent of each other and has no overlapping area.

[0068] S2, obtain the space overlapping scenarios existing in the large space, and modify the area range of the subspace unit associated with the overlapping scenario according to the movement trajectory of the user in the AR experience process.

[0069] Considering that there are a large number of space overlapping scenarios in the AR experience (for example, the user can see part of the area of the adjacent room through the door and window in the corridor on the floor), the "independent non-overlapping subspace" of the traditional K-D tree partitioning cannot cover such scenarios. Therefore, in this embodiment, these overlapping scenarios are first identified, which can be determined by the scene mapping data or the pre-labeled interaction area to determine the position of the overlapping area;

[0070] Further, the subspace range associated with the overlapping scenario is modified based on the movement trajectory of the user in the AR experience process, wherein the trajectory can be determined by the pre-collected passable path, the user-set experience route, etc.

[0071] Figure 3 is an optimized interval partitioning result diagram according to an embodiment of the present application, as shown in Figure 3 If the user can see part of the content of the room (subspace S3) in the corridor (subspace S1), the range of S3 needs to be expanded to the overlapping area visible to the user in the corridor to form a new subspace S3' (containing the overlapping part of the original S3 and S1).

[0072] In another exemplary embodiment, the physical spatial hierarchy information inherent in the CAD file can also be utilized to construct a multi-level sub-space system that combines structural accuracy and user interaction adaptability, in combination with the actual needs of user experience.

[0073] Specifically, this step first automatically parses the CAD file corresponding to a large space (such as a building, a mall, a museum, etc.). By identifying and analyzing the layer information, block definition, and other structural data in the CAD file, the physical hierarchy relationship of the space is extracted, thereby preliminarily structuring the space and forming sub-space units that can reflect the actual building layout, such as the hierarchical structure of “building→floor→public area→room”.

[0074] On this basis, to solve the actual scene problems such as spatial overlap and user path limitation in augmented reality (AR) experience, the initial sub-space needs to be further optimized. The optimization goals are set as “improving spatial accuracy” (ensuring that the sub-space accurately matches the user’s viewable area and interactive area) and “reducing the number of spaces” (reducing resource management complexity). By integrating user-defined experience areas (such as navigable paths and key interactive areas), the initial sub-space is adjusted to finally generate optimized sub-space units that meet the actual experience needs.

[0075] In step S101, the large space is divided into multi-level sub-space units based on the CAD file, which includes the following steps:

[0076] S1: Use an intelligent platform to identify and parse the CAD file of a large space to obtain its physical spatial hierarchy information;

[0077] First, structural elements are identified from the layers and entities of the CAD file. For example, the outlines in the “walls (WALLS)” layer can be used to define the boundaries of rooms and corridors; the insertion of blocks in the “furniture” or “door” layer can be used to identify specific object and passage positions. Text annotations are usually used to name rooms or areas (such as “Exhibition Hall A” “Room 101”), and this information can be extracted to establish the logical hierarchy relationship of the space and form the initial structure of the space accordingly.

[0078] S2, according to the extracted hierarchy relationship, generate initial sub-space units. For example, the closed area defined by the wall polylines in the “1st floor (Floor1)” layer can be used as a “floor” level sub-space; each room with a name in the floor can be used as a “room” level sub-space. The set of multi-level initial sub-spaces formed in this way can directly correspond to the physical layout defined in the CAD drawing.

[0079] Optionally, in the initial division process, the size of the sub-space can be dynamically adjusted according to the space range occupied by the preset virtual assets in the initial sub-space unit. If a large sub-space (such as an open office) is designated to carry a large number of virtual assets, it can be further subdivided to avoid a single sub-space becoming a performance bottleneck in the rendering process. Further, the optimization of the sub-space unit is combined with the user's set experience area to improve the spatial accuracy and minimize the number of spaces, including:

[0080] S1, obtaining the set action trajectory or passable area of the user in the large space;

[0081] Firstly, the set action trajectory or passable area of the user is the core basis for optimization. Among them, the set trajectory can be a specific path or a passable area, which directly reflects the space range that the user can actually reach and see, providing a realistic reference for the adjustment of the sub-space.

[0082] S2, according to the set action trajectory or passable area, and the intersection of the sub-space unit, the sub-space unit is optimized and adjusted, wherein the optimization and adjustment includes merging adjacent sub-space units, deleting redundant sub-space units, and adjusting the boundary range of the sub-space unit;

[0083] Specifically, if the user's trajectory is limited by obstacles (such as support columns, island tables) (such as trajectory P1 cannot pass through a certain area), the associated sub-space range of the area needs to be adjusted (such as reduced or split into original sub-space) to ensure that the sub-space is consistent with the actual reachable and visible area of the user;

[0084] In this embodiment, the adjustment process needs to meet the requirements of "improving spatial accuracy" (the matching degree of the sub-space range and the actual visible / accessible area of the user) and "minimizing the number of spaces" (avoiding the management complexity caused by excessive splitting of sub-spaces), for example, by merging adjacent and similar sub-spaces, and deleting redundant sub-spaces that cannot be covered by the user's trajectory to achieve optimization.

[0085] Figure 4 is a schematic diagram of optimizing a sub-space unit according to an embodiment of the present application, as shown in Figure 4 When there are support columns and other non-passable areas in the space, the user's trajectory (such as P1) will be limited, and at this time the boundary of the associated sub-space (such as S3') needs to be adjusted to ensure that the sub-space range is consistent with the actual reachable and visible area of the user (improve accuracy); for redundant sub-spaces that the trajectory cannot pass through to generate interaction, they can be directly deleted, and adjacent sub-spaces that are commonly covered by the trajectory can be merged (reduce the number) to avoid excessive splitting of sub-spaces or deviation of the range from the actual experience scene.

[0086] S3, establishing a correlation between the optimized sub-space unit and its corresponding virtual resource to form a hierarchical space set;

[0087] Exemplarily, for a single hierarchical space, a virtual asset set relationship formula contained by the hierarchical space is marked as the nth hierarchical space contains m virtual assets: .

[0088] In this embodiment, the optimization goal is to improve the space precision and reduce the number of spaces as much as possible. It can be understood that improving the space precision can reduce the virtual asset loading error (only loading the assets that the user may contact in the subspace to avoid loading irrelevant assets); reducing the number of subspaces can reduce the runtime memory occupation (fewer subspaces mean that fewer asset sets need to be managed at the same time, reducing redundant loading), and finally solve the problems of large memory occupation and low performance in the traditional solution.

[0089] In addition, it should be noted that in different 3D engines (such as Unity3D, Unreal Engine, etc.), the segmented hierarchical spaces (such as buildings, floors, rooms, etc.) can be presented in an intuitive and visible manner by using geometric forms similar to collision boxes.

[0090] As can be understood by those skilled in the art, such visual display can enable the developer to clearly see the boundary range of each space, including whether the space is independent, whether there is an overlap (such as the overlapping area formed by the doorway and the room due to doors and windows), and the like, thereby more conveniently performing space editing and adjustment, such as optimizing the space boundary according to the moving track of the experimenter, ensuring the association relationship between the virtual asset and the corresponding space is accurate, and improving the operation efficiency during content development.

[0091] The above step S101 first preliminarily segments the large space to form a plurality of hierarchical, multi-dimensional asymmetric initial subspaces, and constructs a basic hierarchical structure; then, in combination with the space overlapping scene and the user moving track, the subspace range is adjusted to adapt to the actual visible and reachable area. This link realizes the accurate matching of the subspace to the user experience scene, improves the space precision, minimizes the number of spaces, lays a foundation for subsequent dynamic management of virtual assets for AR experience based on the user position, and improves the running performance and development efficiency.

[0092] S102, in the application execution process, according to the real-time position and the predicted position of the user, a state mark is added to the subspace unit, and the state mark includes: an instant state, a potential reachable state and an unreachable state;

[0093] The instant state is used to mark the subspace unit in which the user is currently located. In this embodiment, the virtual resources in the subspace unit in which the user is currently located match the real-time loading strategy.

[0094] Potential reachable state: used for marking a subspace unit that is predicted to be reachable within a preset period of time in the future and matches the current moving direction and speed of the user, and the virtual resource in the subspace unit predicted to be reachable within the preset period of time in the future matches the preloading strategy;

[0095] Unreachable state: used for marking a subspace unit that is beyond the current moving ability of the user, and the virtual resource in the subspace unit beyond the current moving ability of the user matches the delayed unloading strategy;

[0096] The dynamic update of the state marking is based on at least one of the following factors: real-time position coordinates of the user, a moving speed vector of the user, a head control pose (field of view direction) of the user, and a topological connection relationship between the subspace units.

[0097] It should be noted that the head control pose refers to the field of view direction of the user, and is used as a factor for dynamically updating the state marking of the subspace unit. Specifically, by the orientation of the user (such as a visual distance of 10 meters), the subspace directly facing the user can be preferentially marked as a "potential reachable state".

[0098] In the embodiment, the above mechanism determines the area that the user can enter through the intersection of a ray and space, and thus preloads the virtual resource in the area. The problem of lag caused by improper resource management in traditional AR rendering is solved, precise management of resources is achieved, and rendering efficiency and experience smoothness are improved.

[0099] In the embodiment, step S102 realizes precise management of virtual resources by dynamically marking the state of the subspace based on the subspace units optimized in the preprocessing stage, and solves the problems of "large runtime memory occupation" and "low loading efficiency" in the traditional solution.

[0100] This step is based on an executor (which can be an SDK, Software Development Kit) running on an AR host application. The executor captures the position and behavior data of the user in real time, combines the hierarchical relationship of the subspace units, marks each subspace with three states of "immediate location", "potential reach", and "unreachable", and matches different resource loading strategies (real-time loading, preloading, and delayed unloading) for different states. The core goal is to load resources only when necessary and release redundant resources in a timely manner, thereby reducing memory occupation and improving rendering smoothness, ensuring that the user does not experience lag and resource loading delay in a large-space AR experience.

[0101] Specifically, the immediate location state is a hierarchical space that the user "is passing through" in the AR experience, and can be a subspace unit directly covered by the current physical position or field of view range of the user.

[0102] The virtual resources in this state need to adopt a "real-time loading strategy", that is, immediately load and render all virtual assets in the subspace (such as virtual exhibits in the room and AR interactive elements on the wall). This strategy solves the problem of "resource loading lag caused by overall packaging" in traditional solutions and guarantees real-time interactive experience.

[0103] The potentially reachable state corresponds to the hierarchical space that the user "may pass through" in a short time. Specifically, based on the user's current movement direction, speed, and other data, the subspace unit that can be reached within a preset period (such as 5 seconds) is predicted. For example, when the user moves towards the end of the corridor, the next room in front is preloaded.

[0104] In this embodiment, the virtual resources in this state adopt a "preloading strategy" to preload resources (such as virtual models of the next exhibition hall) in advance before the user arrives, avoiding loading waiting when the user enters. The priority of preloading is determined by the user's movement trend (such as faster speed, further preloading range) and the direction of the field of view (such as loading the subspace in the direction of the user's gaze first), balancing resource loading efficiency and memory usage.

[0105] The unreachable state corresponds to the hierarchical space that "cannot be passed through". It refers to the subspace unit that is beyond the user's movement ability range (such as areas blocked by obstacles or areas completely deviating from the current trajectory direction).

[0106] In this embodiment, the virtual resources in this state adopt a "delayed unloading strategy" to hide the resources (reduce rendering burden) first, and if it is confirmed that the user cannot reach the subspace for a long time (such as more than 30 seconds), the resources are completely unloaded to release memory. This strategy avoids the problem of "invalid resources continuously occupying memory" in traditional solutions, further optimizing the running performance.

[0107] In addition, it should be noted that the state marking in this embodiment is dynamically updated according to multiple factors. Specifically, the factors include: real-time position coordinates to determine the "immediate state" of the subspace, movement speed vector to affect the prediction range of the "potentially reachable state", and field of view direction to mark the subspace in the direction of the user's gaze as potentially reachable, improving the accuracy of preloading. In addition, it also includes the topological connection relationship of the subspace, such as the connectivity of adjacent subspaces, which may affect the reachability judgment, such as a closed door making the corresponding subspace unreachable.

[0108] The above-mentioned multiple factors together ensure that the state marking is synchronized with the actual experience trajectory of the user, realizing dynamic and accurate management of resources.

[0109] Through the above step S102, according to the real-time and predicted positions of the user, the sub-space markers are marked in real-time as being in, potentially reachable, and unreachable states, and real-time loading, preloading, and delayed unloading strategies are matched respectively, and the states are updated dynamically based on the user position, speed, and the like. In subsequent execution links, the running memory occupation can be effectively reduced, the performance problems of traditional solutions can be solved, and the fluency and stability of large-space AR experience can be improved.

[0110] S103, according to the state marking, respectively configuring differentiated loading and caching strategies for the sub-space units, and rendering the large-space AR content according to the loading and caching strategies and the virtual resources corresponding to the sub-space units.

[0111] Among them, for the sub-space units in the being-in state, all virtual resources associated with them are loaded into the running memory, and the high-precision rendering state of the virtual resources is maintained, and the relative position relationship between the virtual resources and the user's perspective is updated in real time.

[0112] Specifically, for the space being passed through, the virtual-real fusion effect within the user's current field of view needs to be prioritized. In the execution link, the strategy in this state is:

[0113] All virtual resources associated with the sub-space are completely loaded into the running memory to ensure that the resources can be directly called. At the same time, the high-precision rendering state of the virtual resources (such as complete texture details and fine model structure) is maintained to avoid the impact of simplified presentation on experience. In addition, the relative position relationship between the virtual resources and the user's perspective needs to be updated in real time to ensure that the virtual content can accurately fit the physical space with the user's movements, turning, and other actions, and achieve a natural virtual-real fusion effect.

[0114] On this basis, in order to further optimize the instantaneous response and fluency of rendering, the virtual assets within the sub-space are dynamically sorted. The basis of this sorting is the preset priority of the virtual assets, which is determined by multiple factors, including user interaction weight, visual saliency, and computing resource consumption. For example, a virtual button that can be directly clicked by the user for interaction (high interaction weight) and a static decorative model far away will be given a higher priority, so that the former is preferentially loaded and rendered. Similarly, objects located in the center of the user's field of view or with strong color contrast (high visual saliency) will also be preferentially processed. Through this dynamic loading based on priority, the most critical and most eye-catching content can be presented first when the user enters a new space, greatly improving the immediacy and immersion of the experience.

[0115] For the sub-space units in the potentially reachable state, the preloading priority is calculated according to the user's moving speed and direction, and part of the virtual resources of the sub-unit space are loaded into the cache space according to the preloading priority.

[0116] Specifically, for the subspace that the user may arrive in the future preset period, the balance between resource preloading and system load needs to be achieved. In this embodiment, the specific strategy is to calculate the preloading priority in combination with the moving speed and moving direction of the user to avoid resource waste caused by indiscriminate loading.

[0117] It should be noted that the core of this step is not only to determine which subspaces to preload, but also to intelligently select which virtual assets in these subspaces to load. In this embodiment, the virtual assets are dynamically sorted and selectively loaded according to their priority. The priority evaluation criteria of the assets are also based on factors such as user interaction weight, visual saliency, and computing resource consumption. For example, when the user is walking towards an exhibition hall, the system will preferentially preload virtual assets such as signs and core exhibits in the exhibition hall that have high interaction weight and high visual saliency into the cache. Some special effects or background decorations that consume a lot of computing resources but are not core may be given a lower priority and not loaded or loaded in a low-precision version. Through this refined strategy, only part of the core virtual resources in the subspace are loaded into the cache space according to priority, which not only reserves key resources for the area that the user may enter, significantly reducing the loading delay when entering the space, but also avoids loading all resources of the entire subspace into the cache, thereby more effectively utilizing the limited cache space and not putting additional pressure on the currently running memory.

[0118] For the subspace units that cannot be reached, the loaded virtual resources are marked as to-be-released state, and the virtual resources are released according to the memory pressure situation.

[0119] Specifically, for the subspace that the user determines cannot be reached, this embodiment optimizes system performance by releasing redundant resources. The specific strategy is to mark the loaded virtual resources as to-be-released state, and first hide their rendering display to reduce real-time rendering pressure; then, according to the current memory pressure situation, gradually release these resources, completely empty the memory space occupied by them, and avoid problems such as running lag, heating, etc. caused by invalid resources long-term occupying system resources.

[0120] During the release process, the concept of priority can also be introduced. When the system memory pressure increases, virtual assets with lower priority, such as visually insignificant, non-interactive, and resource-intensive background objects, will be released first. This approach ensures that even when resources need to be released, it can retain as much as possible the cache of high-priority assets that may need to be quickly reloaded due to the user's sudden turn. This avoids problems such as running lag, heating, etc. caused by invalid resources long-term occupying system resources, and achieves more intelligent and efficient memory management.

[0121] It can be understood that S103 is the link from resource scheduling to final presentation of large space AR content, based on the state marking of the subspace unit, a differentiated loading and caching strategy is formulated and executed, and finally the accurate rendering of virtual content is completed. Through dynamically regulating the loading range, storage location (running memory / cache space) and rendering precision of virtual resources, it can not only ensure the smooth presentation of AR content that the user is currently and will soon contact, but also avoid invalid resource occupation of system memory, thereby solving the performance bottleneck caused by "non-discriminatory resource loading" in the traditional scheme, and realizing the efficiency and stability of large space AR experience.

[0122] Through the above steps S101 to S103, the optimization scheme is proposed to solve the problems of complex development, large installation package size, high runtime memory occupation and performance reduction caused by the traditional large space AR content development in units of entire scenes. The large space is divided into multi-level asymmetric subspaces such as "building-floor-public area-room", and the space is optimized in combination with the user AR experience moving track. The editor completes the space division and optimization, and the executor marks the subspace state according to the user real-time position and predicted position, and implements differentiated loading (real-time loading, preloading, delayed unloading) strategy for virtual assets. This scheme solves the defects of the traditional scheme through fine space management and dynamic resource scheduling, improves the development task decomposition granularity and efficiency, reduces the runtime memory occupation, and improves the rendering performance and experience smoothness of large space AR content.

[0123] The embodiment also provides a rendering system of large space AR content, which is used to implement the above-mentioned embodiments and preferred embodiments, and details are not repeated. As used below, the terms "module", "unit", "sub-unit" and the like can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0124] Figure 5 is a structural block diagram of a rendering system of large space AR content according to an embodiment of the present application, as Figure 5 shown, the system is applied to AR content rendering in large space, and includes a preprocessing module 50 and an execution module 51, where:

[0125] The preprocessing module 50 is configured to divide the large space into multiple levels to obtain multiple levels of initial subspace units, and to optimize the initial subspace units based on the set experience area of the user to obtain the subspace units, with the optimization target of improving the space precision and minimizing the space quantity.

[0126] The execution module 51 is configured to add state marks to the sub-space units according to the real-time position and the predicted position of the user, the state marks including an instant state, a potentially reachable state and an unreachable state,

[0127] According to the state marks, the sub-space units are respectively configured with differentiated loading and caching strategies, and the large-space AR content is rendered according to the loading and caching strategies and the virtual resources corresponding to the sub-space units.

[0128] Through the above system, an optimization scheme is proposed to solve the problems of complicated development, large installation package, high runtime memory occupation and reduced performance caused by taking the entire scene as a unit in traditional large-space AR content development. The core technical means are as follows: the large space is divided into a multi-level asymmetric sub-space of "building-floor-public area-room", the space is optimized in combination with the AR experience moving track of the user, the space division and optimization are completed by the editor, the execution device marks the state of the sub-space according to the real-time position and the predicted position of the user, and the differentiated loading strategy is implemented on the virtual assets. This scheme solves the defects of the traditional scheme through fine space management and dynamic resource scheduling, improves the development task decomposition granularity and efficiency, reduces the runtime memory occupation, and improves the rendering performance and experience fluency of the large-space AR content.

[0129] In one embodiment, Figure 6 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, as Figure 6 indicated, an electronic device is provided, which can be a server, and the internal structure diagram of the electronic device can be as Figure 6 indicated. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected by an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is configured to provide computing and control capability, the network interface is configured to communicate with an external terminal through network connection, the internal memory is configured to provide an environment for the operation of the operating system and the computer program, the computer program is executed by the processor to implement a large-space AR content rendering method, and the database is configured to store data.

[0130] Those skilled in the art can understand that Figure 6 the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0133] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent protection of the present application should be subject to the appended claims.

Claims

1. A method for rendering large-space AR content, characterized in that, The method for rendering AR content in large spaces includes: In the preprocessing process, the large space is divided into multiple levels to obtain multiple levels of initial subspace units. With the optimization goals of improving spatial accuracy and minimizing the number of spaces, the initial subspace units are optimized based on the user's set experience area to obtain subspace units, and the subspace units are associated with their corresponding virtual resources. In the initial division process, the size of the subspace is dynamically adjusted according to the space range occupied by the preset virtual assets within the initial subspace unit. With the optimization goals of improving spatial accuracy and minimizing the number of spaces, the initial subspace unit is optimized in conjunction with the user's defined experience area, including: The system acquires a user's set movement trajectory or passable area in the large space, where the set movement trajectory and passable area reflect the actual reachable and visible space range of the user; based on the intersection of the set movement trajectory or passable area with the initial subspace unit, the system optimizes and adjusts the initial subspace unit to obtain a subspace unit; wherein, the optimization and adjustment includes: merging adjacent subspace units, deleting redundant subspace units, and adjusting the boundary range of the subspace unit; The optimized subspace units are associated with their corresponding virtual resources to form a hierarchical space set. The optimization and adjustment objectives include: reducing virtual asset loading errors by improving spatial segmentation accuracy, and reducing runtime memory usage by reducing the total number of subspace units; During the application execution process, state markers are added to the subspace units based on the user's real-time and predicted locations. These state markers include: current location, potentially reachable, and unreachable states. Based on the status markers, differentiated loading and caching strategies are configured for each subspace unit, and large-space AR content is rendered based on the loading and caching strategies and the virtual resources corresponding to each subspace unit.

2. The method according to claim 1, characterized in that, Dividing a large space into multiple levels yields multiple levels of initial subspace units, including: The intelligent platform identifies and parses the CAD files corresponding to the large space to obtain the physical spatial hierarchy information of the large space. Based on the spatial hierarchy information, the large space is initially divided to form multiple levels of initial subspace units; During the initial segmentation process, the size of the initial subspace unit is dynamically adjusted according to the space occupied by the virtual assets within the initial subspace unit.

3. The method according to claim 1, characterized in that, The KD-tree algorithm is used to partition the large space into multiple levels, resulting in initial subspace units at multiple levels, including: Obtain the spatial overlapping scenes existing in the large space, and modify the area range of the initial subspace unit associated with the overlapping scenes according to the user's movement trajectory during the AR experience.

4. The method according to claim 3, characterized in that, In the process of dividing a large space into multiple levels, the method further includes: The boundary extent of each initial subspace unit is marked using a collision box-based cube framework; After the initial subspace units based on the marked boundary range are displayed, the initial subspace units are optimized according to the user's operation instructions.

5. The method according to claim 1, characterized in that, The real-time location status is used to mark the user's current subspace unit, and the virtual resources within the current subspace unit are matched with a real-time loading strategy. The potential reachability state is used to mark subspace units that are predicted to be reachable within a preset future time period and that match the user's current movement direction and speed. The virtual resources within the subspace units that are predicted to be reachable within the preset future time period are matched with a preloading strategy. The "unreachable state" is used to mark subspace units that are outside the user's current mobility range. Virtual resources within subspace units that are outside the user's current mobility range are matched with a delayed unloading strategy. The dynamic update of the status marker is based on at least one of the following factors: the user's real-time location coordinates, the user's movement speed vector, the user's field of view direction, and the topological connection relationship between subspace units.

6. The method according to claim 5, characterized in that, Based on the state flags, configuring loading strategies for the subspace units includes: For the subspace unit in the current state, load all associated virtual resources into the runtime memory, maintain the high-precision rendering state of the virtual resources, and update the relative positional relationship between the virtual resources and the user's viewpoint in real time. For the subspace unit of the potential reachable state, the preloading priority is calculated based on the user's movement speed and direction, and a portion of the virtual resources of the subspace unit is loaded into the cache space according to the preloading priority; For subspace units that are unreachable, the loaded virtual resources are marked as pending release, and the virtual resources are released according to memory pressure.

7. The method according to any one of claims 1, 5, and 6, characterized in that, Rendering large-space AR content, based on the loading and caching strategy and the virtual resources corresponding to the subspace unit, includes: The virtual assets within the subspace unit are dynamically sorted based on their priority, and the virtual assets are loaded according to the priority of the dynamic sorting. The priority of the virtual assets is based on at least one of the following factors: user interaction weight, visual salience, and computing resource consumption.

8. A rendering system for large-space AR content, characterized in that, For AR content rendering in large spaces, the system includes: a preprocessing module and an execution module, wherein: The preprocessing module divides the large space into multiple levels to obtain multiple levels of initial subspace units. With the optimization goals of improving spatial accuracy and minimizing the number of spaces, it optimizes the initial subspace units based on the user's defined experience area, obtaining subspace units and establishing associations between the subspace units and their corresponding virtual resources. During the initial partitioning process, the size of the subspace is dynamically adjusted according to the space occupied by the preset virtual assets within each initial subspace unit. With the optimization goals of improving spatial accuracy and minimizing the number of spaces, the initial subspace unit is optimized in conjunction with the user's defined experience area, including: The system obtains a user's set movement trajectory or passable area in the large space, where the set movement trajectory and passable area reflect the actual reachable and visible space range of the user; based on the intersection of the set movement trajectory or passable area with the initial subspace unit, the system optimizes and adjusts the initial subspace unit to obtain a subspace unit; wherein, the optimization and adjustment includes: merging adjacent subspace units, deleting redundant subspace units, and adjusting the boundary range of the subspace unit; The optimized subspace units are associated with their corresponding virtual resources to form a hierarchical space set. The optimization and adjustment objectives include: reducing virtual asset loading errors by improving spatial segmentation accuracy, and reducing runtime memory usage by reducing the total number of subspace units; The execution module is used to add status markers to the subspace units based on the user's real-time and predicted locations. These status markers include: current location status, potentially reachable status, and unreachable status. Based on the status markers, differentiated loading and caching strategies are configured for each subspace unit, and large-space AR content is rendered based on the loading and caching strategies and the virtual resources corresponding to each subspace unit.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images