Real-time refreshed data label loading and scheduling method in three-dimensional scene

Abstract

The invention relates to a real-time refreshed data label loading and scheduling method in a three-dimensional scene, and belongs to the technical field of three-dimensional visualization. The method comprises the following steps that S1, a virtual scene with three-dimensional models and entity labels is loaded, a plurality of three-dimensional models are associated with at least one entity label, and / or at least one entity label is associated with a spatial position point of the virtual scene; s2, loading the entity labels according to the following mode: loading all entity labels expected by a user or specified by a user project in the virtual scene according to user configuration, and creating an agent label according to a mode corresponding to all loaded entity labels; and S3, when the user views the virtual scene, determining whether to display the proxy label or the entity label according to the view direction, the view angle and the sight distance of the scene camera.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional visualization, and particularly to a method for loading and scheduling data signs with real-time refresh in a three-dimensional scene. Background Art

[0002] In the technical field of three-dimensional visualization, data signs, as a common display method, are widely used to present relevant data or information in virtual scenes. These data signs can be static texts, dynamically changing graphics, or even contain interactive components, aiming to provide users with intuitive and timely information display. With the continuous expansion of the application of three-dimensional scenes, from urban planning simulation to complex industrial environment monitoring, and then to immersive game experiences, how to efficiently load and schedule a large number of data signs has become a key technical challenge.

[0003] Existing data sign loading and scheduling methods mainly include several techniques based on object distance, visible range, scene information, and line-of-sight detection. For example, the method based on object distance determines whether to load by calculating the distance between the camera and the sign; the method based on the visible range considers the camera's frustum and only loads those signs within the visible area; the method based on scene information pre-analyzes the layout of the entire scene to optimize the loading order of the signs; and the method based on line-of-sight detection goes further, attempting to simulate the human visual mechanism and giving priority to loading the signs that the user is most likely to focus on.

[0004] For example, CN117197408A discloses a method, device, medium, and equipment for automatic avoidance of signs in an osgEarth 3D simulation environment. The method includes: obtaining a first circumscribed rectangle of a visible entity model in the osgEarth 3D simulation environment, and obtaining a second circumscribed rectangle of a target sign corresponding to the visible entity model in the osgEarth 3D simulation environment, where the target sign is a sign that displays the attributes of the visible entity model and faces the terminal screen; judging whether the visible entity model collides with the target sign according to the positional relationship between the first circumscribed rectangle and the second circumscribed rectangle; if so, determining a target position for the target sign to be redisplayed, and moving the target sign to the target position.

[0005] However, although these methods have their respective advantages, they are not sufficiently adaptable to the requirements in different business scenarios, resulting in a series of problems. First, in some cases, due to the failure to accurately predict the actual needs of users, the system may load unnecessary signs, causing a waste of computing resources. Second, when faced with large-scale complex scenarios, existing methods often struggle to ensure that all important information can be presented to users in a timely manner, thus affecting the user experience. In addition, over-reliance on a single strategy (such as only based on distance) may lead to obvious delays or lags under specific conditions (such as the perspective of a fast-moving camera), thereby reducing the system's response speed and smoothness. Therefore, there is an urgent need in this field for a method for loading and scheduling data signs with real-time refresh in a three-dimensional scene to solve the problems of slow loading speed and serious overlap when displaying a large number of data signs.

[0006] In addition, on the one hand, there are differences in the understanding of those skilled in the art; on the other hand, when the applicant made this invention, a large number of documents and patents were studied, but due to space limitations, all details and content were not listed in detail. However, this does not mean that this invention does not possess the features of these prior arts. On the contrary, this invention already possesses all the features of the prior arts, and the applicant reserves the right to add relevant prior arts in the background art. Summary of the Invention

[0007] Aiming at the deficiencies of the prior art, the present invention provides a method for loading and scheduling data signs with real-time refresh in a three-dimensional scene to solve at least some of the above technical problems.

[0008] In a virtual scene, such as in a digital twin chemical plant, the amount of sign data may reach tens of thousands or even hundreds of thousands of items. Especially when this data is in a state of dynamic update, such as displaying status data such as faults, pressures, and flow rates, the data communication pressure and rendering pressure may cause a significant increase in the three-dimensional rendering pressure at the user end, resulting in freezing or even program crashes. Especially for users logging in using mobile devices, they also face the influence of limited data channels or poor wireless data signal quality, and there is an urgent need to improve the user experience.

[0009] The current method adopted is to set the rendering priority. For example, render the three-dimensional model in the virtual scene first and then render the signs. At this time, there will be a problem of excessive time lag, which may even be more than ten minutes. When users urgently need to view the current status of a specific work unit on-site, it is difficult to meet the user's needs. Although there are already technologies for grading the rendering of three-dimensional objects according to the perspective and viewing distance for scene rendering, due to the variety of sign types and the diverse dynamic forms, these grading rendering technologies can only limitedly solve the problem and cannot well adapt to the situation of industrial on-site terminal devices, especially when the performance of terminal devices is limited and the memory is tense when running multiple APPs simultaneously, as well as the situation where the amount of sign data changes dynamically, special considerations are needed to meet the user's needs.

[0010] Based on this, the present invention discloses a method for loading and scheduling data signs with real-time refresh in a three-dimensional scene, which includes the following steps:

[0011] S1. Load a virtual scene with 3D models and physical signs, wherein a plurality of 3D models are respectively associated with at least one physical sign, and / or at least one physical sign is respectively associated with a spatial position point of the virtual scene;

[0012] S2. Load these physical signs in the following manner: load all the physical signs expected by the user or specified by the user project in the virtual scene according to the user configuration, and create proxy signs in a manner corresponding to all the loaded physical signs;

[0013] S3. When the user views the virtual scene, determine whether to display the proxy sign or the physical sign according to the viewing direction, viewing angle, and viewing distance of the scene camera.

[0014] In the method for loading and scheduling data signs with real-time refresh in the three-dimensional scene proposed by the present invention, step S1 not only establishes the basic structure of the virtual scene, but also provides necessary data support for subsequent optimization. First, by pre-defining the relationship between each physical sign and the 3D model, unnecessary calculations can be reduced during the loading phase. For example, if it is known that a certain sign will only be displayed under specific conditions, the loading of its resources can be postponed until the necessary moment. Second, this association mechanism ensures the accuracy of information display, enabling users to clearly understand the object pointed to by each sign. Finally, integrating all these elements into the virtual scene not only improves the initialization efficiency of the system, but also lays a solid foundation for dynamic adjustment in subsequent steps. This method effectively solves the problem of slow loading caused by the lack of preprocessing in the prior art, and significantly improves the response speed and user experience when initially entering the scene. Step S2 flexibly loads physical signs and creates proxy signs according to the user configuration, reducing the rendering and loading operation volume. Step S3 decides whether to display the proxy sign or the physical sign by intelligently judging the viewing direction, viewing angle, and viewing distance of the scene camera, achieving the optimal allocation of resources. This method ensures that important information is presented to the user immediately, while avoiding unnecessary rendering and loading, saving valuable computing resources, and enhancing the user's immersion and interaction experience. The above three steps complement each other and jointly improve the overall performance of the system in loading and scheduling signs in the three-dimensional scene. Through preprocessing, personalized configuration and proxy sign creation, and intelligent display selection, the present invention solves the problems caused by the lack of preprocessing, insufficient personalization, and excessive resource consumption in the prior art, significantly improves the response speed and user experience of the system, and at the same time reduces the pressure on the server side and the operating cost.

[0015] According to a preferred embodiment, the spatial position points associated with the physical sign include spatial position points above or near the 3D model and / or spatial position points in the airspace within the virtual scene to determine the associated object, wherein this association method is constructed by means of the lead line of the physical sign pointing to its associated object.

[0016] This solution further refines the spatial position points associated with the physical sign. This extensible association method is constructed by means of a lead line, enhancing the system's adaptability to different business scenarios. By introducing spatial position points in the airspace, the system can not only handle the information display requirements on tangible objects but also flexibly respond to the information indication of intangible but important areas (such as specific security alert areas). In addition, the presence of the lead line increases visual clarity, helping users locate the required information more quickly, especially useful when there are multiple similar items in the scene. The lead line can also be used as an additional information carrier, such as changing colors to represent different types of links or being dynamically updated to reflect real-time data changes. This method not only enriches the form of information display but also improves the user's interaction experience, thus solving the problems of unclear information pointing and visual chaos in traditional methods.

[0017] According to a preferred embodiment, when loading the virtual scene, the sign space area is divided according to the grouping situation of the physical signs, where all the physical signs within the virtual scene are grouped according to their positions within the virtual scene.

[0018] First of all, the grouping strategy helps to simplify the management logic, enabling developers to grasp the overall layout of the entire scene at a macroscopic level and quickly locate and adjust the content of specific areas. Secondly, the area division based on grouping can significantly reduce the complexity of visibility testing because only each group needs to be detected once instead of checking each sign one by one. This greatly reduces the computational overhead, especially more obvious when dealing with a large number of signs. Finally, reasonable grouping can also promote the effective application of the caching mechanism. When the user leaves an area, the relevant resources of that area can be temporarily retained so that the state can be quickly restored when the user visits again. This method not only improves the system's response speed but also reduces the pressure on the server side, effectively solving the performance bottleneck problem caused by frequent loading and unloading in the prior art.

[0019] According to a preferred embodiment, the created proxy sign refers to a pseudo-information display board shown within the virtual scene captured and displayed by the scene camera to reduce the rendering and loading computation amount, so that within the virtual scene, the proxy sign is used to replace the physical sign to represent the existence of the corresponding sign, but the physical sign is not actually rendered and displayed.

[0020] This design fundamentally solves the performance challenges brought by large-scale digital signage. First, proxy signage avoids the loading of a large amount of dynamic data, greatly reducing the Shader operation requirements and eliminating complex operations for the scene camera, especially operations such as rotating the signage matrix and calculating the normal vector direction. Second, proxy signage does not require texture mapping and vertex data verification. These optimization measures reduce the overall computation and communication volume by more than 90%. For large scenes containing hundreds of thousands of signage, this reduction can even reach 99%. By using proxy signage, the system can not only provide a smoother and more responsive user experience but also effectively control operating costs, which is a win-win situation for both enterprises and users. This method greatly alleviates the lag and latency phenomena caused by excessive resource consumption in the existing technology and significantly improves the interactivity and immersion of the 3D scene.

[0021] According to a preferred embodiment, the creation process of proxy signage includes:

[0022] Determine the position of the proxy signage: Based on the association relationship between the physical signage and the associated object, find the exact position of each physical signage in the virtual scene and determine the position where the proxy signage should be placed accordingly;

[0023] Set the attributes of the proxy signage: Assign the same geometric shape, color, and other necessary parameters to all proxy signage;

[0024] Establish a mapping relationship: Maintain a two-way mapping table from the proxy signage to the physical signage to record the corresponding relationship between each pair of proxy signage and physical signage.

[0025] First, by finding the exact position of each physical signage in the virtual scene based on the association relationship between the physical signage and the associated object and determining the position where the proxy signage should be placed accordingly, the accuracy and consistency of the proxy signage are ensured. Second, assigning the same geometric shape, color, and other necessary parameters to all proxy signage simplifies the development work while maintaining visual consistency, enabling users to obtain basic information guidance even at a distance. Finally, maintaining a two-way mapping table from the proxy signage to the physical signage to record the corresponding relationship between each pair of proxy signage and physical signage ensures that the system can easily find the corresponding signage for loading at the appropriate time. This method not only improves the flexibility and maintainability of the system but also ensures the accuracy and timeliness of information display, solving the problem of poor user experience caused by lagging information updates in the existing technology.

[0026] According to a preferred embodiment, for step S3, it includes the following sub-steps:

[0027] Determine the type of each signage space area in the virtual space according to the viewing direction, viewing angle, and viewing distance of the scene camera;

[0028] Determine whether to display a proxy sign or a physical sign according to the type of the sign space area, where

[0029] The types of the sign space area include a direct sign space area for displaying a physical sign and an indirect sign space area for displaying a proxy sign.

[0030] This method realizes the optimal allocation of resources by finely managing the display strategy of signs. First, by comprehensively considering the viewing direction, viewing angle, and viewing distance, it can accurately judge which signs are within the user's line of sight and which are not, thus avoiding unnecessary rendering and loading. Second, select an appropriate display method according to the type of the sign space area, which not only ensures the immediate presentation of important information but also saves valuable computing resources. This method not only improves the response speed of the system but also enhances the realism and immersion of the user experience, solving the problems of visual fatigue and interaction obstacles caused by information overload or insufficiency in the prior art.

[0031] According to a preferred embodiment, the direct sign space area is the sign space area that can be directly observed by the scene camera, or several sign space areas that can be viewed unobstructed under the current viewing direction, viewing angle, and preset viewing distance. These direct sign space areas are at least partially distributed in a fan-shaped area with the scene camera as the center, the viewing angle as the opening and closing angle, and the preset viewing distance as the radius.

[0032] The introduction of this concept clarifies the specific scope of the direct sign space area and provides a clear operation guide for the system. First, by setting the preset viewing distance, it ensures that important information is always within the visible range and avoids performance losses caused by unnecessary long-distance rendering. Second, the concept of the fan-shaped area is adopted, which not only takes into account the field of view width of the camera but also takes into account the user's viewing habits, enabling the system to more intelligently screen out the most relevant information for display. Finally, this method not only improves the information transmission efficiency but also enhances the user's sense of participation and interactivity, solving the problems of visual chaos and attention dispersion caused by unreasonable information distribution in the prior art.

[0033] According to a preferred embodiment, the indirect sign space area is the sign space area that can be directly observed only after the scene camera moves a certain distance, or several sign space areas that cannot be displayed or can only be partially displayed due to the interval occlusion of the direct sign space area under the current viewing direction, viewing angle, and preset viewing distance. These indirect sign space areas are at least partially distributed in an annular fan area with the scene camera as the center, within the viewing angle as the opening and closing angle but outside the preset viewing distance as the radius.

[0034] The introduction of this concept provides a more flexible management mechanism for the system. First, by distinguishing between direct and indirect signage space areas, the system can dynamically adjust the information display strategy according to the actual needs of users, avoiding information overload or omission. Second, using the definition of the fan-shaped ring area, it is possible to more accurately predict the user's next action and prepare the content of the area that the user may be interested in in advance, ensuring the coherence and integrity of information display. Finally, this method not only improves the intelligence level of the system, but also enhances the realism and immersion of the user experience, solving the interaction barrier problem caused by discontinuous or incomplete information display in the existing technology.

[0035] According to a preferred implementation, for the directly divided signage space areas, determine the priorities of these directly divided signage space areas according to the type of signage data, and adjust the field of view range of the scene camera according to the field of view width of the directly divided signage space area with a relatively higher priority, so as to preferentially display the directly divided signage space area with a relatively higher priority.

[0036] This method realizes the intelligent management of information display by introducing a priority mechanism. First, through the analysis of the signage data type, the information that the user is most concerned about can be accurately identified to ensure that this information is preferentially presented to the user. Second, adjusting the field of view width according to the priority not only ensures the immediate presentation of important information, but also avoids the interference of irrelevant information, improving the user's concentration and participation. Finally, this method not only improves the efficiency of information transmission, but also enhances the adaptive ability of the system, enabling the system to flexibly adjust the display strategy according to the needs of different application scenarios, solving the problem of poor user experience caused by disordered or redundant information display in the existing technology.

[0037] According to a preferred implementation, in addition to the direct signage space area and the indirect signage space area in the virtual scene, there can also be a distal signage space area and / or a blind area signage space area, where the distal signage space area is at least distributed in the fan-shaped ring area centered on the scene camera, within the opening and closing angle of the viewing angle but outside the radius of the maximum viewing distance; the blind area signage space area is other signage space areas that are not within the fan-shaped area centered on the scene camera, with an opening and closing angle of the viewing angle and an infinite radius.

[0038] This extensible definition provides a more comprehensive management framework for the system. First, by introducing the remote sign space area, the system can better handle the information display requirements over long distances, ensuring that important information is not overlooked due to excessive distance. Second, the concept of the blind spot sign space area helps identify areas that are completely out of the user's line of sight, thus avoiding unnecessary resource waste. Finally, this method not only improves the system's coverage and management accuracy but also enhances its flexibility and scalability, enabling the system to flexibly adjust the display strategy according to the requirements of different application scenarios, and solving the problem of poor user experience caused by limited information display range in the existing technology. Brief Description of the Drawings

[0039] Figure 1 is a schematic diagram of the division of the sign space area of a preferred embodiment provided by the present invention;

[0040] Figure 2 is a schematic diagram of constructing an association relationship between an entity sign and its corresponding associated object through a lead wire according to a preferred embodiment provided by the present invention;

[0041] Figure 3 is a flowchart of the steps of a method for loading and scheduling data signs according to a preferred embodiment provided by the present invention.

[0042] List of Reference Numerals

[0043] 100: Direct sign space area; 110: Entity sign; 200: Indirect sign space area; 210: Proxy sign; 300: Remote sign space area; 400: Blind spot sign space area; 500: Scene camera; 600: Associated object; 610: Lead wire. Detailed Embodiment

[0044] In the present invention, the "scene camera 500" refers to: a virtual viewfinder required for the user's display to present the current scene in the viewing direction, viewing angle, and viewing distance when the user views the virtual scene. In other words, the virtual scene presented by this virtual viewfinder will be displayed on the user's display as required by the user. The final image seen by the user is determined by the configuration of the user's display and can be a two-dimensional or three-dimensional image.

[0045] In the present invention, the "physical sign 110" refers to: within a virtual scene, a geometric body suitable for displaying information that is attached to display data related to certain objects, such as an information display board like a square frame or a circular frame. This information display board has transparency, a fixed size, and a display directionality. For example, a semi-transparent square information display board allows the information attached to it to be observed from both sides, and the distant view behind the information display board can be seen; while users can only observe the information displayed on an opaque circular information display board in a preset direction. In the present invention, in addition to the information display board, the physical sign 110 further includes: a lead line 610 pointing to its associated object 600. The starting point of this lead line 610 is located at a certain "spatial position point (such as a pixel point)" above or near the associated object 600; the ending point of this lead line 610 is located at a certain "spatial position point (such as a pixel point)" above or near this physical sign 110.

[0046] In the present invention, the "sign data" refers to: the data constitutes the information displayed by the physical sign 110, which can be numbers, letters, or graphics; it can also be a dynamically changing graphic; it can also be based on the data changes related to certain objects corresponding to this physical sign 110.

[0047] In the present invention, the "proxy sign 210" refers to: within the virtual scene captured and displayed by the scene camera 500, a pseudo-information display board displayed to reduce the rendering and loading computation amount. In the present invention, to reduce the computation amount, these proxy signs 210 can be pseudo-information display boards that are consistent with each other. As a pseudo-information display board, the proxy sign 210 has the minimum computation amount and loading computation amount, and preferably each has the same attributes, such as a default opaque gray square frame. These proxy signs 210 can replace the physical sign 110 within the virtual scene to indicate the existence of the corresponding sign, but do not actually render and display the physical sign 110.

[0048] In the present invention, the "data sign" refers to: the physical sign 110 and / or the proxy sign 210.

[0049] In the present invention, the "sign space area" refers to: the space area occupied by a group of physical signs 110 within the virtual scene, where all the physical signs 110 within the virtual scene are grouped according to their positions within this virtual scene. For example Figure 1 all the physical signs 110 are grouped in the way of a preset cube to obtain several sign space areas, and the physical signs 110 in different sign space areas can be replaced by proxy signs 210 or not directly display any data signs based on the parameter adjustment of the scene camera 500. The physical signs 110 are grouped based on the positions of the starting points of their respective lead lines 610 (see Figure 2 ) within the virtual scene.

[0050] The following will be described in detail with reference to the accompanying drawings.

[0051] As Figure 3 shown, the present invention discloses a method for loading and scheduling data signs with real-time refresh in a three-dimensional scene, which may include the following steps:

[0052] S1. Load a virtual scene with three-dimensional models and physical signs 110, wherein several three-dimensional models are respectively associated with at least one physical sign 110, and / or at least one physical sign 110 is respectively associated with a spatial position point of the virtual scene;

[0053] S2. Load these physical signs 110 and create proxy signs 210 in the following manner: Load all physical signs 110 expected by the user or specified by the user project in the virtual scene according to the user configuration, and create proxy signs 210 in a manner corresponding to all the loaded physical signs 110;

[0054] S3. When the user views the virtual scene, determine whether to display the proxy sign 210 or the physical sign 110 according to the viewing direction, viewing angle and viewing distance of the scene camera 500.

[0055] Preferably, for step S1, the loaded virtual scene is usually a three-dimensional scene, which includes a series of three-dimensional models. These three-dimensional models can be any object in the real world, such as buildings, devices, vehicles, people, etc., or can be completely fictional elements. These three-dimensional models can be created by professional 3D modeling software, such as Autodesk Maya, Blender or SolidWorks, etc. These software allow designers to precisely control the shape, size, color, material and other properties of the models according to requirements.

[0056] Preferably, in the loaded virtual scene, in addition to the three-dimensional models, it may also include physical signs 110 associated with these three-dimensional models, wherein a three-dimensional model may be associated with one or more physical signs 110. Further, the physical sign 110 may be associated with a spatial position point of the virtual scene. This spatial position point can be a certain "spatial position point" above or near the above-mentioned three-dimensional model, or can be only on a certain "spatial position point" in the virtual scene, that is, not pointing to any three-dimensional model. For example, it can be a "spatial position point" pointing to the airspace. As Figure 2 shown, this association method can be constructed by means of a lead line 610 pointing to its associated object 600. In other words, the "associated object 600" of the present invention can be either a three-dimensional model in the virtual scene or the airspace in the virtual scene. Herein, the "airspace" refers to the space in the virtual scene that is not occupied by entities, that is, the place where there is no three-dimensional model. Preferably, as Figure 2As shown, the starting point of the lead 610 is located at a "spatial position point (such as a pixel point)" above or near the associated object 600; the ending point of the lead 610 is located at a "spatial position point (such as a pixel point)" above or near the physical sign 110. Preferably, the setting of the lead 610 clearly indicates which physical sign 110 is associated with which associated object 600, which is particularly useful especially when there are multiple similar items in the scene. The presence of the lead 610 increases visual clarity and helps users locate the required information more quickly. At the same time, the lead 610 can also be used as an additional information carrier, such as changing colors to represent different types of links, or being dynamically updated to reflect changes in real-time data.

[0057] Preferably, when associating the physical sign 110 with the associated object 600, a logical connection between the two is actually established, so that the sign data on the physical sign 110 can accurately reflect the state or attributes of the associated object 600. This association can be implemented through programming languages to define where the physical sign 110 appears in the virtual scene and what sign data is displayed, etc.

[0058] Preferably, when loading the virtual scene, the sign space area can be divided according to the grouping of the physical signs 110, where all the physical signs 110 in the virtual scene are grouped according to their positions in the virtual scene. Further, the position of the physical sign 110 in the virtual scene refers to the position of the starting point of its corresponding lead 610, which is usually determined by the x, y, z values in a three-dimensional coordinate system, or for geographic information system (GIS) applications, latitude and longitude can be used to represent it. This positioning method provides an accurate spatial reference point for the signs. For situations where there may be a large number of physical signs 110 in a large area, such as there may be hundreds or even thousands of physical signs 110 per square kilometer, in order to improve management efficiency and rendering performance, the physical signs 110 can be hierarchically and regionally divided, where a predefined cube can be used as shown Figure 1 to divide the virtual scene into several such sub-regions.

[0059] Preferably, after dividing the virtual scene according to the above criteria, for each sub-region, the minimum bounding box of all the physical signs 110 in the region can be calculated. The bounding box is an axis-aligned rectangular body that can completely contain all the physical signs 110 in the region and is used for fast spatial queries and collision detection. In this way, the subsequent visibility test process can be greatly simplified. When loading the physical sign 110, in addition to recording the specific position of the sign, the corresponding bounding box information can also be generated and stored simultaneously.

[0060] Preferably, in step S1, "loading the virtual scene with the 3D model and the physical sign 110" mainly refers to the initialization stage. In this stage, the system loads the basic structure of the virtual environment, including all 3D models and the association information between these models and the physical sign 110. Here, the "physical sign 110" actually refers to a sign frame or placeholder that defines attributes such as position, size, and orientation, rather than immediately rendering a complete visual effect. That is to say, this step is mainly to ensure that all necessary data structures are ready, but it does not mean that all graphical resources (such as textures, fonts, etc.) have been fully loaded.

[0061] Furthermore, in step S2, the physical sign 110 is loaded in the following way: all physical signs 110 specified by the user's expectations or user projects within the virtual scene are loaded according to the user configuration. This step is a further refinement for specific application requirements. In this step, the system decides which specific physical signs 110 need to be activated or displayed according to the user's configuration file or project requirements. This involves loading additional graphical resources, such as specific images, text content, or other multimedia elements.

[0062] Preferably, the user configuration refers to a series of parameters set by developers or end-users through a graphical interface or other means. These parameters define which physical signs 110 should be loaded and how they should be displayed. For example, the user may only want to display physical signs 110 on specific types of buildings (such as hospitals, schools) or specific types of devices (such as distillation towers, storage tanks). Therefore, the system needs to parse these configuration files and extract specific requirements for the physical signs 110, including but not limited to attributes such as position, size, transparency, and directionality.

[0063] Furthermore, once the user requirements are clear, the corresponding physical signs 110 can be loaded according to these rules. Here, the "loading" not only means importing the pre-prepared 3D model into the virtual scene, but more importantly, ensuring that each physical sign 110 can be correctly associated with the corresponding associated object 600, so that when the associated object 600 moves, the physical sign 110 can update its relative position accordingly to maintain an accurate information reference.

[0064] Preferably, to optimize the rendering efficiency, the present invention proposes the concept of a proxy sign 210. The proxy sign 210 is a simplified version of an information display board used to quickly present to users at a long distance or with low resolution. Its main features are extremely low computational requirements and extremely short loading times. Since the proxy sign 210 does not require complex texture mapping, lighting effects, etc., the workload of the graphics processor is greatly reduced. When the user quickly browses a large area, the system can quickly switch to using the proxy sign 210, thus improving the interaction experience. Compared with the complete physical sign 110, the proxy sign 210 occupies less storage space, which helps to relieve the memory pressure in large projects.

[0065] Preferably, the proxy sign 210 can be created through the following process:

[0066] Determine the position of the proxy sign 210: For the physical signs 110 loaded in step S2, based on the association relationship between these physical signs 110 and the associated objects 600, find the exact position (such as coordinates) of each physical sign 110 in the virtual scene, and accordingly determine the position where the proxy sign 210 should be placed;

[0067] Set the attributes of the proxy sign 210: According to predefined criteria, assign the same geometric shape (such as a rectangle), color (such as opaque gray), and other necessary parameters (such as size) to all proxy signs 210;

[0068] Establish a mapping relationship: In order to be able to replace the physical sign 110 with the proxy sign 210 at the appropriate time, a two-way mapping table from the proxy sign 210 to the physical sign 110 needs to be maintained. This mapping table records the correspondence between each pair of proxy signs 210 and physical signs 110, enabling the system to easily find the corresponding sign for loading when necessary.

[0069] Furthermore, if the position of any physical sign 110 changes, the corresponding proxy sign 210 can be recreated through the above process to achieve an update.

[0070] Preferably, ideally, the position where the proxy sign 210 is placed should be as close as possible to the position of the corresponding physical sign 110, so that the user can roughly judge the position where the physical sign 110 is located based on the proxy sign 210 from a distance.

[0071] Preferably, setting the attributes of the proxy sign 210 not only simplifies the development work but also helps to maintain visual consistency.

[0072] Preferably, the proxy sign 210 can be represented by the following formula:

[0073] Billboard = {PositionP, Size(w, h), Normal Vector N},

[0074] where P is the central position of the proxy billboard 210, w and h are the width and height of the proxy billboard 210 respectively, and N is the normal vector (pointing to the scene camera 500) towards which the proxy billboard 210 faces.

[0075] Furthermore, the client device can achieve the replication of a large number of proxy billboards 210 by adjusting the normal vector N and vertex coordinates. In particular, the vertex representation (Vertices) can also be simply processed without pursuing absolute position accuracy. For example, a rectangular billboard is usually defined by four vertices (calculated based on the offset from P):

[0076]

[0077] where V 1 , V 2 , V 3 , V 4 are the four vertices of the rectangular billboard respectively.

[0078] In addition, the proxy billboard 210 of the present invention does not require complex matrix transformations either, such as:

[0079] rotaition matrix = lookAt(camera position , billboard position , up vecto r),

[0080] where rotation matrix is the calculated view matrix (View Matrix). The lookAt function constructs a 4×4 transformation matrix based on the given three parameters. This matrix can transform points in the world coordinate system to a new coordinate system centered on the scene camera 500. camera position is the position of the scene camera 500 in the world coordinate system, usually represented as a three-dimensional vector, which determines the exact location where the scene camera 500 is located. billboard position is the target position or the position of the object towards which the scene camera 500 faces. up vector is the vector defining the "up" direction, which depends on the coordinate system used to determine the rotation angle of the scene camera 500, ensuring that the generated view matrix not only points to the target position but also maintains the correct vertical direction.

[0081] The proxy sign 210 of the present invention does not need to calculate the unit vector N of the pointing direction based on the following formula according to the position C of the scene camera 500 and the center position P of the proxy sign 210:

[0082]

[0083] where N is the unit vector of the pointing direction, which represents the direction from the center position P of the proxy sign 210 to the position C of the scene camera 500.

[0084] Compared with the physical sign 110, the proxy sign 210 significantly reduces the calculation and communication burdens. First, the proxy sign 210 avoids loading a large amount of dynamic data, greatly reducing the Shader operation requirements, and omits complex operations for the scene camera 500, especially operations such as rotating the sign matrix and calculating the normal vector direction. In addition, the proxy sign 210 does not need to perform texture mapping and vertex data verification. These optimization measures reduce the overall calculation amount and communication amount by more than 90%, especially for large scenes containing hundreds of thousands of signs, and this reduction can even reach 99%. This performance improvement is crucial for the user experience, especially when dealing with large 3D scenes. For large enterprises that need to support a large number of client devices to view 3D scenes simultaneously, the significant performance improvement not only improves the user interaction experience but also means a significant reduction in the server investment cost, and the saved funds may reach millions or even tens of millions of yuan. By using the proxy sign 210, the system can not only provide a smoother and more responsive user experience but also effectively control the operation cost, which is a win-win situation for both enterprises and users.

[0085] Preferably, when performing step S2, multiple factors can be comprehensively considered to ensure the best effect, so as to not only effectively solve the performance challenges brought by a large number of data signs but also provide a richer and more interactive user experience. For example, for different types of client devices (desktop computers, mobile terminals, etc.), the display accuracy of the proxy sign 210 can be adjusted; for particularly large scenes, the area can be further segmented, and a multi-level proxy sign 210 structure can be adopted to balance performance and quality.

[0086] Preferably, as Figure 3 shown, for step S3, it may include one or more of the following sub-steps:

[0087] S3.1. When the user views the virtual space, determine the preset viewing direction, preset viewing angle, and preset viewing distance of the scene camera 500 according to the user's usage habits, the type of virtual scene loading request, and / or the characteristics of the client device, and thereby determine whether to pre-load the proxy sign 210 or the physical sign 110 for display;

[0088] S3.2. Determine the types of the space regions of each sign in the virtual space according to the viewing direction, viewing angle, and viewing distance of the scene camera 500;

[0089] S3.3. Determine whether to display the proxy sign 210 or the physical sign 110 according to the type of the sign space region.

[0090] Preferably, in step S3.1, the user usage habit refers to the specific behavior patterns and preferences formed by the user during the interaction with the virtual environment. These habits can be obtained in various ways, such as by analyzing the user's historical operation records, collecting real-time feedback data, or directly asking the user about their preference settings. Understanding the user's usage habits is crucial for optimizing the system performance because it can help predict the user's next actions and thus make preparations in advance. Historical data analysis refers to statistically analyzing the user's past behaviors to identify information such as which regions or objects the user most frequently accesses and which time periods the user is most active. Based on these analysis results, those regions with a high probability of being accessed can be preferentially processed during the corresponding time periods. The real-time feedback mechanism refers to using machine learning algorithms or other intelligent technologies to dynamically adjust its behavior at runtime to adapt to the user's immediate needs. For example, if it is detected that the user is quickly browsing the entire scene, the level of detail (LOD) can be temporarily reduced to speed up the rendering; conversely, when the user stays at a specific location, the details can be gradually increased to provide a clearer information display. The user preference settings refer to allowing the user to customize certain parameters, such as the default viewing angle, preferred color scheme, etc., which not only improves the degree of personalization but also can serve as an important reference for system optimization. For example, if the user prefers to use the first-person perspective, then the system can pre-adjust the angle of the scene camera 500 to match this preference.

[0091] Preferably, in step S3.1, the virtual scene loading request type refers to different types of loading instructions initiated by the user, which determine how the system should respond. Among them, the loading request types can include initial loading, incremental loading, and / or conditional loading. Initial loading refers to the loading process when the user first enters a new virtual scene. At this time, the system needs to present a complete visual experience to the user as quickly as possible, so usually a more conservative strategy is adopted, such as only loading necessary elements to avoid excessive data transmission and complex calculations. Incremental loading refers to that as the user moves or interacts in the scene, additional content loading may be triggered. In this case, the system should be able to respond flexibly and update the required data in a timely manner without affecting the stability of the existing content. For example, before the user approaches a specific area, start loading the high-resolution textures or model details within that area. Conditional loading refers to the loading events that occur under certain special conditions. Such loading is often uncertain, but corresponding resource management plans also need to be carefully planned in advance.

[0092] Preferably, in step S3.1, the client characteristics mainly refer to the type of the client device and its hardware configuration. The differences between different devices can be very large, ranging from high-performance workstations to ordinary laptops, and then to mobile terminals such as smartphones and tablets. Each device has its unique performance characteristics and limitations. Considering this, the system must be flexible enough to adjust its operation mode according to the needs of different devices.

[0093] Preferably, in step S3.1, the preset viewing direction is the direction that the scene camera 500 defaults to point to. Usually, an angle that can maximize the display of the core elements of the scene will be selected; the preset viewing angle is the field of view width of the scene camera 500, usually represented by the opening angle. Among them, a larger viewing angle allows users to see a wider area, but may cause distant objects to appear too small, while a smaller viewing angle can provide a more focused visual effect and is suitable for observing details at close range; the preset viewing distance is the effective observation distance of the scene camera 500, which determines how far the user can see. The selection of the preset viewing distance depends on the scene scale and user expectations, ensuring that important information is always within the visible range while avoiding performance losses caused by unnecessary long-distance rendering.

[0094] Preferably, for step S3.1, during the implementation of preloading, methods such as progressive loading, caching mechanism, predictive loading, and parallel processing can be used in combination and work together to ensure that the system can efficiently manage resources in different situations and provide a smooth user experience. Specifically, as a basic strategy, progressive loading first presents a preliminary visual effect by quickly loading low-resolution or simplified content, thus shortening the user's waiting time. As the user further explores the scene, the system gradually adds details, such as loading high-resolution textures or complex geometric models, to ensure the quality of the final displayed content. On this basis, the caching mechanism can avoid the situation of repeated loading of the same data. When the user leaves an area, the system does not immediately release all relevant resources, but temporarily retains these data in memory for a period of time. In this way, if the user returns to this area soon, the data can be directly read from the cache, greatly improving the response speed and reducing unnecessary network requests or disk I / O operations. At the same time, predictive loading uses the user's behavior patterns and historical data to prepare in advance the content of the areas that the user may be interested in. For example, the system can analyze the user's movement trend, judge their forward direction, and start loading the signs and models along the way a few seconds before the user arrives. This method can not only ensure the seamless transition of the scene but also effectively reduce the possible delay phenomenon when suddenly entering a new area, enhancing the immersion. Finally, in order to make full use of the powerful computing power of modern multi-core processors, parallel processing technology is applied to various loading tasks. This means that different loading steps (such as loading the model geometry and downloading the required texture files) can be carried out simultaneously on multiple threads. In this way, the system can complete more work in the same period of time, significantly improving the overall efficiency. In addition, parallel processing can also optimize resource allocation, so that even in the case of poor network conditions or limited hardware performance, a good loading speed and user experience can be maintained. Therefore, progressive loading provides instant feedback to the user, the caching mechanism ensures resource reuse, predictive loading prepares for future needs in advance, and parallel processing maximizes the use of hardware resources, thus efficiently completing the preloading process. Through this comprehensive preloading method, the system can not only quickly respond to the user's interaction requests but also provide a high-quality visual experience while maintaining high performance.

[0095] Preferably, the sign space area is preset in the virtual scene according to a preset geometric layout, and each physical sign 110 in the virtual scene can determine the sign space area to which it belongs according to the spatial position point where it is located. Preferably, in step S3.2, at least part of the sign space area is divided into a direct sign space area 100 and an indirect sign space area 200 according to the visibility at the current viewing direction, viewing angle, and viewing distance. Among them, the physical sign 110 can be displayed in the direct sign space area 100, and the proxy sign 210 can be displayed in the indirect sign space area 200.

[0096] Preferably, for step S3.2, as Figure 1 shown, the types of the sign space area may include a direct sign space area 100 and an indirect sign space area 200, and for the same sign space area, its type can be switched between the direct sign space area 100 and the indirect sign space area 200 according to the difference in visibility at the current viewing direction, viewing angle, and viewing distance. Preferably, the direct sign space area 100 is the sign space area that the scene camera 500 can directly observe, or several sign space areas that can be viewed unobstructed at the current viewing direction, viewing angle, and preset viewing distance. These direct sign space areas 100 are distributed or mainly distributed in a fan-shaped area with the scene camera 500 as the center, the viewing angle as the opening and closing angle, and the preset viewing distance as the radius. Further, "can be directly observed" means, for example: the sign space area directly adjacent to the scene camera 500 at the current viewing direction, viewing angle, and preset viewing distance, or the sign space area that is separated from the scene camera 500 by one sign space area but is not blocked by the 3D model in the separated sign space area at the current viewing direction, viewing angle, and preset viewing distance. Preferably, the indirect sign space area 200 is the sign space area that can be directly observed after the scene camera 500 moves a certain distance, or several sign space areas that cannot be displayed or can only be partially displayed due to the interval occlusion of the direct sign space area 100 at the current viewing direction, viewing angle, and preset viewing distance. These indirect sign space areas 200 are distributed or mainly distributed in an annular sector area with the scene camera 500 as the center, within the viewing angle as the opening and closing angle but outside the preset viewing distance as the radius. Further preferably, the indirect sign space area 200 is distributed or mainly distributed in an annular sector area with the scene camera 500 as the center, within the viewing angle as the opening and closing angle but outside the preset viewing distance as the radius and within the maximum viewing distance as the radius. The maximum viewing distance can be adjusted according to the configuration of the user device and the actual needs of the user to define the boundary of the annular sector area, where the maximum viewing distance is greater than the preset viewing distance.

[0097] Preferably, as Figure 1As shown, the types of the sign space regions may further include a distal sign space region 300, where the distal sign space region 300 is other sign space regions outside the direct sign space region 100 and the indirect sign space region 200 under the current viewing direction and viewing angle of the scene camera 500. In other words, the distal sign space region 300 is distributed or mainly distributed in the fan-shaped ring region centered on the scene camera 500, with the viewing angle as the opening and closing angle and the maximum viewing distance as the radius outside.

[0098] Preferably, as Figure 1 shown, the types of the sign space regions may further include a blind area sign space region 400, where the blind area sign space region 400 is other sign space regions not within the fan-shaped region centered on the scene camera 500 with the viewing angle as the opening and closing angle and an infinite radius.

[0099] Preferably, in step S3.2, for the obtained direct sign space region 100, the direct sign space region 100 with a high priority may be determined, where the priority is determined according to the type of the sign data. Further, the viewing field range (FOV: Field of View) of the scene camera 500 is adjusted according to the viewing field width of the direct sign space region 100 with a high priority to preferentially display the direct sign space region 100 with a high priority, where, in particular, the current viewing distance of the scene camera 500 is determined according to the priority order of the physical signs 110 determined according to the type of the sign data.

[0100] Preferably, in step S3.2, for the obtained indirect sign space region 200, the indirect sign space region 200 with a high priority is determined.

[0101] Preferably, in step S3.3, the type of the data sign to be displayed may be determined according to the type of the sign space region. For the direct sign space region 100, the physical sign 110 may be displayed; for the indirect sign space region 200, the proxy sign 210 may be displayed. Further, when there are still a distal sign space region 300 and / or a blind area sign space region 400 in the virtual scene, the proxy sign 210 may or may not be displayed in these two sign space regions, which is determined by the memory and display configuration of the client device.

[0102] According to a preferred embodiment, the present invention can provide a data sign loading and scheduling system for real-time refreshing in a three-dimensional scene, which includes a processor. Among them, the processor can be used to implement the data sign loading and scheduling method of the present invention. Preferably, the present invention can also provide a storage medium for storing an encoded program for running the data sign loading and scheduling method. Preferably, the processor can include a virtual scene loading module, an entity sign 110 recording module, an agent sign 210 creating module, and a display analysis module.

[0103] It should be noted that the above specific embodiments are exemplary. Those skilled in the art can come up with various solutions inspired by the disclosed content of the present invention, and these solutions also fall within the scope of the disclosure of the present invention and within the protection scope of the present invention. Those skilled in the art should understand that the description of the present invention and its drawings are illustrative and do not constitute a limitation on the claims. The protection scope of the present invention is defined by the claims and their equivalents. The description of the present invention contains multiple inventive concepts. Phrases such as "preferably" or "according to a preferred embodiment" indicate that the corresponding paragraphs disclose an independent inventive concept. The applicant reserves the right to file divisional applications based on each inventive concept. Throughout the text, the features guided by "preferably" are only optional and should not be construed as being required to be provided. Therefore, the applicant reserves the right to waive or delete relevant preferred features at any time.

Claims

1. A method for loading and scheduling data signs for real-time refresh in a three-dimensional scene, characterized in that: It includes the following steps: S1, loading a virtual scene with a three-dimensional model and a physical sign (110), wherein a plurality of three-dimensional models are respectively associated with at least one physical sign (110), and / or at least one physical sign (110) is respectively associated with a spatial position point of the virtual scene; S2, loading the entity signs (110) in the following manner: loading all entity signs (110) in the virtual scene that are desired by the user or specified by the user project according to the user configuration, and creating a proxy sign (210) in a manner corresponding to all the loaded entity signs (110); S3. When the user views the virtual scene, it is determined whether to display the proxy sign (210) or the physical sign (110) according to the viewing direction, viewing angle and viewing distance of the scene camera (500).

2. The method according to claim 1, characterized in that The spatial position points associated with the physical sign (110) include spatial position points on or near the three-dimensional model and / or spatial position points in the airspace within the virtual scene to determine the associated object (600), wherein this association is constructed with the aid of a lead line (610) pointing from the physical sign (110) to its associated object (600).

3. The method according to claim 1 or 2, characterized in that: When loading the virtual scene, the sign space area is divided according to the grouping of the physical signs (110), wherein all the physical signs (110) in the virtual scene are grouped according to their locations in the virtual scene.

4. The method according to any one of claims 1 to 3, characterized in that: The created proxy sign (210) refers to a pseudo information display board displayed in the virtual scene captured and displayed by the scene camera (500) in order to reduce the amount of rendering and loading calculations, so that the proxy sign (210) is used in the virtual scene instead of the physical sign (110) to indicate the existence of the corresponding sign, but the physical sign (110) is not actually rendered and displayed.

5. The method according to any one of claims 1 to 4, characterized in that: The creation process of the agent sign (210) includes: Determining the location of the proxy sign (210): based on the association relationship between the physical sign (110) and the associated object (600), finding the exact location of each physical sign (110) in the virtual scene, and determining the location where the proxy sign (210) should be placed accordingly; Setting the properties of the agent sign (210): giving all agent signs (210) the same geometric shape, color and other necessary parameters; Establishing a mapping relationship: maintaining a bidirectional mapping table from the agent sign (210) to the physical sign (110) to record the corresponding relationship between each pair of the agent sign (210) and the physical sign (110).

6. The method according to any one of claims 1 to 5, characterized in that: Step S3 includes the following sub-steps: Determining the type of each sign space area in the virtual space according to the viewing direction, viewing angle and viewing distance of the scene camera (500); Determining whether to display an agent sign (210) or a physical sign (110) according to the type of the sign space area, wherein: Types of signage space areas include direct signage space areas (100) for displaying physical signage (110) and indirect signage space areas (200) for displaying agency signage (210).

7. The method according to any one of claims 1 to 6, characterized in that: The direct sign space area (100) is a sign space area that can be directly observed by the scene camera (500), or a number of sign space areas that can be viewed without obstruction under the current viewing direction, viewing angle and preset viewing distance. These direct sign space areas (100) are at least partially distributed in a fan-shaped area with the scene camera (500) as the center, the viewing angle as the opening angle and the preset viewing distance as the radius.

8. The method according to any one of claims 1 to 7, characterized in that: The indirect sign space area (200) is a sign space area that can be directly observed only after the scene camera (500) moves a certain distance, or is a number of sign space areas that cannot be displayed or can only be partially displayed due to the interval occlusion of the direct sign space area (100) under the current viewing direction, viewing angle and preset viewing distance. These indirect sign space areas (200) are at least partially distributed in a fan ring area with the scene camera (500) as the center and the viewing angle as the opening and closing angle but outside the preset viewing distance as the radius.

9. The method according to any one of claims 1 to 8, characterized in that: For the direct signage space areas (100) obtained by division, the priorities of these direct signage space areas (100) are determined according to the type of signage data, and the field of view range of the scene camera (500) is adjusted according to the field of view width of the direct signage space areas (100) with relatively higher priorities, so as to preferentially display the direct signage space areas (100) with relatively higher priorities.

10. The method according to any one of claims 1 to 9, characterized in that: In addition to the direct sign space area (100) and the indirect sign space area (200) in the virtual scene, there can also be a remote sign space area (300) and / or a blind sign space area (400), wherein the remote sign space area (300) is at least distributed in a fan-shaped area with the scene camera (500) as the center, the viewing angle as the opening and closing angle but outside the maximum viewing distance as the radius; the blind sign space area (400) is other sign space areas that are not in the fan-shaped area with the scene camera (500) as the center, the viewing angle as the opening and closing angle and an infinite radius.

Images