Railway BIM large scene loading method and system based on edge-end cooperation

Abstract

The application provides a railway BIM large scene loading method and system based on edge-end cooperation, which comprises the following steps: an edge node of a node network acquires model request information sent by a user end, wherein the model request information comprises basic information of a requested model; model data of a corresponding model is loaded based on the model request information, and basic information of the model requested by the model request information is fed back to the user end in response to the model request information sent by the user end; camera parameter information sent by the user end is acquired, a BIM model is off-screen rendered based on the camera parameter information, and an image of the model in a camera visual angle is obtained; the color of each pixel point in the image is determined to determine a corresponding component id, and the geometric information and material information of the corresponding component id are determined based on the component id; the geometric information and material information of the corresponding component id are fed back to the user end, so that the user end loads a scene image based on the geometric information and material information of the corresponding component id.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of scene loading, in particular to a railway BIM large scene loading method and system based on edge-end cooperation. BACKGROUND

[0002] With the development of wireless network technology and Web3D technology, there are more and more demands for displaying three-dimensional large scenes in web browsers, such as the application of building information modeling (BIM) and geographic information system (GIS).

[0003] At present, according to the visualization mode, the three-dimensional scene rendering and loading strategy on the Web can include: 1. Taking the Web rendering as the core, the three-dimensional model data is loaded from the cloud to the browser or the local file is loaded by the Web, and the rendering is completed independently by the Web. In order to reduce the number of Web grid rendering, some culling techniques are proposed, and the combination of the view frustum culling technique based on hybrid space division and the level of detail technique is proposed to achieve good culling effect and improve the rendering frame rate to a certain extent; 2. Taking the cloud rendering as the core, the three-dimensional model data does not need to be transmitted from the cloud service, and the model rendering work is completed by the cloud, and the rendered image data is transmitted to the browser for display.

[0004] However, in the Web processing, the Web has limited computing resources, so the computing efficiency is low; in the cloud rendering processing, the cloud has rich computing resources, but the load of the cloud is large, and each data transmission with the Web needs to consume a large amount of transmission resources, and the processing efficiency of the processing mode of the prior art is low. SUMMARY

[0005] In view of this, the embodiments of the present application provide a railway BIM large scene loading method based on edge-end cooperation to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present application provides a railway BIM large scene loading method based on edge-end cooperation, which comprises the following steps:

[0007] The edge node of the node network acquires the model request information sent by the user end, and the model request information includes the basic information of the requested model;

[0008] Based on the model request information, the model data of the corresponding model is loaded, and in response to the model request information sent by the user end, the basic information of the model requested by the model request information is fed back to the user end;

[0009] Obtaining camera parameter information sent by the user terminal, performing off-screen rendering on the BIM model based on the camera parameter information to obtain an image of the model at a camera view angle;

[0010] Obtaining the color of each pixel point in the image to determine the corresponding component id, and determining the corresponding geometric information and material information based on the component id, and feeding back the geometric information and material information of the corresponding component id to the user terminal to enable the user terminal to load the scene image based on the geometric information and material information of the corresponding component id.

[0011] With the above scheme, the edge node of the node network first obtains model request information, and feeds back basic information of a model requested by the model request information to the user terminal, so that the user terminal obtains the basic architecture of the model, the user of the user terminal determines camera parameter information based on the basic architecture, and sends the camera parameter information to the edge node, and then the edge node performs off-screen rendering, determines the component id, determines the corresponding geometric information and material information, and sends them to the user terminal, so that the user terminal loads the scene image based on the geometric information and material information of the component id. The Web terminal does not need to bear a large amount of calculation, and since there are many edge nodes, the cloud terminal does not need to bear a large load.

[0012] In some embodiments of the present application, the node network further includes a cloud center node, and in the step of obtaining the model request information sent by the user terminal by the edge node of the node network, the edge node determines whether the model data corresponding to the model request information is pre-stored, and if not, initiates a loading request for the corresponding model data to the cloud center node, and obtains the model data from the cloud center node.

[0013] In some embodiments of the present application, in the step of feeding back the basic information of the model requested by the model request information to the user terminal in response to the model request information sent by the user terminal, the basic information is obtained from the model data of the corresponding model and fed back to the user terminal, so that the user terminal constructs a model organization structure tree based on the basic information.

[0014] In some embodiments of the present application, in the step of performing off-screen rendering on the BIM model based on the camera parameter information to obtain an image of the model at a camera view angle, the edge node performs off-screen rendering on the model to obtain an image of the model at a view angle corresponding to the camera parameter information.

[0015] In some embodiments of the present application, in the step of obtaining the color of each pixel point in the image to determine the corresponding component id, and determining the corresponding geometric information and material information based on the component id, the color of the pixel point is determined based on the pixel value of the pixel point, the corresponding component id is matched based on the color, and the corresponding geometric information and material information are matched in the model data based on the component id.

[0016] In some embodiments of the present application, in the step of feeding back the geometric information and material information corresponding to the component id to the user end, the number of pixel points corresponding to each component id is determined based on the image, a sending queue is constructed based on the number of pixel points corresponding to each component id, and the component id is sent in the model data based on the order of the sending queue to match the corresponding geometric information and material information.

[0017] In some embodiments of the present application, in the step of constructing a sending queue based on the number of pixel points corresponding to each component id, an effectiveness value is calculated based on the number of pixel points corresponding to the component id, the grid bounding box volume of the component corresponding to the component id, and the memory space value of the component corresponding to the component id, the effectiveness values corresponding to each component id are sorted according to their sizes, and a sending queue is obtained.

[0018] In some embodiments of the present application, in the step of calculating the effectiveness value based on the number of pixel points corresponding to the component id, the grid bounding box volume of the component corresponding to the component id, and the memory space value of the component corresponding to the component id, the effectiveness value is calculated according to the following formula:

[0019]

[0020] wherein Validity id represents the effectiveness value corresponding to the component id, S id represents the number of pixel points corresponding to the component id, V id represents the grid bounding box volume of the component corresponding to the component id, M id represents the memory space value of the component corresponding to the component id.

[0021] In some embodiments of the present application, in the step of feeding back the geometric information and material information corresponding to the component id to the user end, it is determined whether there is a record of feeding back the geometric information and material information of the same component id to the user end in the history record, and if there is, the geometric information and material information corresponding to the component id do not need to be sent again.

[0022] The second aspect of the present application also provides a railway BIM large scene loading system based on edge-end cooperation, which comprises a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, and the processor being configured to execute the computer instructions stored in the memory, so that the system implements the steps as described in the foregoing method.

[0023] The third aspect of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the aforementioned railway BIM large scene loading method based on edge-end cooperation.

[0024] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following detailed description and drawings in which

[0025] Those skilled in the art will appreciate that the objects and advantages of the application can be realized and attained by means illustrated in the accompanying drawings and detailed description, and thus, it is intended that the application not be limited as to the particular details of the description and the illustrative embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application.

[0027] Figure 1 A schematic diagram of an embodiment of the railway BIM large scene loading method based on edge-end cooperation of the present application;

[0028] Figure 2 A schematic diagram of the interaction between the edge node and the user end;

[0029] Figure 3 A schematic diagram of the processing of off-screen rendering. DETAILED DESCRIPTION

[0030] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but not as a limitation of the present application.

[0031] It should also be noted that, in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0032] As shown in Figure 1 and 2 The present application proposes a railway BIM large scene loading method based on edge-end cooperation, which comprises the following steps:

[0033] Step S100, the edge node of the node network acquires the model request information sent by the user end, and the model request information comprises the basic information of the requested model;

[0034] In practice, the node network includes a cloud center node and multiple edge nodes, with the cloud center node connected to the edge nodes.

[0035] In the specific implementation process, the user end is the Web end. The Web end sends model request information to the node network, and any edge node of the node network receives the model request information.

[0036] Step S200: Load the model data of the corresponding model based on the model request information, and respond to the model request information sent by the user terminal by feeding back the basic information of the model requested by the model request information to the user terminal.

[0037] In the specific implementation process, the model request information includes the basic information of the requested model, and the edge node determines the corresponding model based on the basic information of the model. The basic information of the model is the tree information of the three-dimensional model data.

[0038] Step S300: Obtain camera parameter information sent by the user terminal, and perform off-screen rendering on the BIM model based on the camera parameter information to obtain an image of the model from the camera's perspective;

[0039] In some embodiments of the present invention, after obtaining the basic information of the model, the user terminal selects the observation view of the model. The user terminal determines the corresponding camera parameter information based on the observation view selected by the user. The camera parameter information includes the user's frustum data, which includes the position of the frustum vertex and the frustum angle.

[0040] like Figure 3 As shown, in the specific implementation process, the edge nodes perform off-screen rendering of the model based on the camera parameter information sent again by the user. Specifically, this solution assigns the culling task to the edge nodes for execution. First, the component ID, a 10-bit integer with a unique identifier for the mesh, is converted to a 32-bit representation to construct color values ​​and transparency, using RGBA-4D vectors to describe vertex colors and transparency. Then, to reduce the number of GPU rendering operations and improve rendering efficiency, this solution applies frustum culling technology to the edge nodes, using the CPU to efficiently and quickly select visible mesh data under given scene and camera parameters. Finally, after frustum culling, a Framebuffer Object (FBO) is constructed at one-quarter of the final screen size, and the mesh within the frustum is rendered. The fragment shader renders the colors constructed from the component IDs into the FBO. The color values ​​in the FBO are the component ID values ​​of the mesh data finally presented on the user's screen, utilizing GPU capabilities to complete mesh occlusion culling.

[0041] In step S400, the color of each pixel point in the image is acquired to determine the corresponding component id, and the corresponding geometric information and material information are determined based on the component id, the geometric information and the material information of the corresponding component id are fed back to the user end to enable the user end to load the scene image based on the geometric information and the material information of the corresponding component id.

[0042] In the specific implementation process, the model data is provided with the geometric information and the material information corresponding to each component id, and in the process of loading the scene image by the user end based on the geometric information and the material information corresponding to the component id, the user end loads the corresponding geometric information and material information to the corresponding position based on the position corresponding to each component id.

[0043] By using the above scheme, the edge node of the node network first acquires model request information, and feeds back the basic information of the model requested by the model request information to the user end, so that the user end acquires the basic architecture of the model, the user of the user end determines the camera parameter information based on the basic architecture, and then sends the camera parameter information to the edge node, and then the edge node performs off-screen rendering, determines the geometric information and the material information corresponding to the component id through off-screen rendering, and sends the geometric information and the material information to the user end, so that the user end loads the scene image based on the geometric information and the material information of the component id. The Web end does not need to bear a large amount of calculation, and since there are many edge nodes, the cloud end does not need to bear a large load.

[0044] In some embodiments of the application, the node network further includes a cloud center node, in the step of acquiring the model request information sent by the user end by the edge node of the node network, the edge node determines whether the model data corresponding to the model request information is pre-stored, if not pre-stored, initiates a loading request for the corresponding model data to the cloud center node, and acquires the model data from the cloud center node.

[0045] In the specific implementation process, if the edge node pre-stores the corresponding model data, the model data is directly called.

[0046] In the specific implementation process, the edge node selects a preset number of models with the most requested times based on the requested times of the models, stores the corresponding model data, and increases the requested times once the model is requested once. The edge node stores a plurality of model data with high use frequency, which can ensure the use efficiency of the data, reduce the storage pressure of the edge node, and also reduce the calculation pressure of the cloud center node.

[0047] In some embodiments of the present application, in the step of feeding back the basic information of the model requested by the model request information to the user terminal in response to the model request information sent by the user terminal, the basic information is obtained from the model data corresponding to the model and fed back to the user terminal, so that the user terminal constructs a model organization structure tree based on the basic information.

[0048] In the specific implementation process, the user terminal obtains the basic structure of the model through the model organization structure tree for the user to select the observation perspective.

[0049] In some embodiments of the present application, in the step of off-screen rendering the BIM model based on the camera parameter information to obtain the image of the model at the camera perspective, the edge node off-screen renders the model to obtain the image of the model at the perspective corresponding to the camera parameter information.

[0050] In the specific implementation process, different colors are rendered for different component ids in the image of the model at the perspective corresponding to the camera parameter information.

[0051] In some embodiments of the present application, in the step of determining the corresponding component id based on the color of each pixel point in the image and determining the corresponding geometric information and material information based on the component id, the color of the pixel point is determined based on the pixel value of the pixel point, the corresponding component id is matched based on the color matching, and the corresponding geometric information and material information are matched in the model data based on the component id.

[0052] In some embodiments of the present application, in the step of feeding back the geometric information and material information corresponding to the component id to the user terminal, the number of pixel points corresponding to each component id is determined based on the component id corresponding to each pixel point of the image, a sending queue is constructed based on the number of pixel points corresponding to each component id, and the component id is sent by matching the corresponding geometric information and material information in the model data based on the order of the sending queue.

[0053] In some embodiments of the present application, the sending queue only sorts the component ids from high to low according to the number of pixel points corresponding to the component ids.

[0054] With the above scheme, the sending queue is first constructed based on the number of pixel points corresponding to the component ids, and the component ids with larger pixel point areas in the image are sorted in front in the sending queue. Therefore, the geometric information and material information corresponding to the component ids requiring larger computing resources can be sent first, so that the area corresponding to the component id can be loaded first at the user terminal, and the geometric information and material information corresponding to the component ids requiring less computing resources are sent one by one subsequently. Therefore, the present application improves the processing efficiency of the user terminal by constructing the sending queue, and ensures the overall loading efficiency.

[0055] In some embodiments of the present application, in the step of constructing the sending queue based on the number of pixel points corresponding to each component id, the validity value is calculated based on the number of pixel points corresponding to the component id, the grid bounding box volume of the component corresponding to the component id, and the memory space value of the component corresponding to the component id, the validity values corresponding to each component id are sorted in size to obtain the sending queue.

[0056] In the specific implementation process, in addition to occlusion culling, the geometric characteristics of the component itself are one of the factors affecting loading, such as the volume of the bounding box and the memory space value of the grid. The larger the bounding box, the easier it is to occupy a larger space on the screen, and it should be loaded as soon as possible; the smaller the memory space value of the grid, the faster the transmission, the smaller the overall loading time, the more it should be loaded as soon as possible. Specifically, more models that have a greater impact on user vision are transmitted within a certain time, the validity is greater, the loading priority is higher, and the user experience is better.

[0057] In some embodiments of the present application, the scheme calculates the validity value based on the number of pixel points corresponding to the component id, the grid bounding box volume of the component corresponding to the component id, and the memory space value of the component corresponding to the component id, and the sending queue sorts the component ids from high to low based on the validity value. Therefore, the scheme takes into account the influence of the three factors on the demand for computing resources, and the above scheme can determine the demand for computing resources of each component id corresponding area by combining multiple factors to ensure the overall loading efficiency at the user end.

[0058] In some embodiments of the present application, in the step of calculating the validity value based on the number of pixel points corresponding to the component id, the grid bounding box volume of the component corresponding to the component id, and the memory space value of the component corresponding to the component id, the validity value is calculated as follows:

[0059]

[0060] wherein Validity id represents the validity value corresponding to the component id, S id represents the number of pixel points corresponding to the component id, V id represents the grid bounding box volume of the component corresponding to the component id, M id represents the memory space value of the component corresponding to the component id.

[0061] In some embodiments of the present application, in the step of feeding back the geometric information and material information corresponding to the component id to the user end, it is determined whether the historical record exists to feed back the geometric information and material information of the same component id to the user end. If it exists, the geometric information and material information corresponding to the component id do not need to be sent again.

[0062] By adopting the scheme, the transmitted geometric information and material information are recorded, and do not need to be transmitted again, thereby improving transmission resource utilization efficiency and reducing transmission pressure.

[0063] Comparative example

[0064] With 1920x1080 as the final screen size, 100x100x100 closely connected unit cubes as the actual scene to be rendered, and 100mbps as the network bandwidth environment, the scheme is compared and analyzed with the Web mode and the cloud rendering mode in terms of initial loading time and 60s network total traffic.

[0065] Pure Web loading: At least the grid component id and grid geometric data need to be loaded in the Web mode, the component id is composed of 4 bytes, the component scene is 1 million cubic grids, it is assumed that each cubic grid data is composed of position coordinates, normal, mapping coordinates, color value and vertex index, the vertex is 8, each attribute is composed of 3 32-bit floating point numbers, 3 32-bit floating point numbers, 2 32-bit floating point numbers, and 1 32-bit floating point number, and the index is composed of 36 32-bit integers. The grid data amount S is calculated as follows: all : S all =1000000x[8x(3x4+3x4+2x4+3x4)+36x4+4], the S is calculated as about 492MB, and the network transmission time is calculated as about 39.36s, so the network traffic at 60s is about 492MB. all

[0066] Cloud rendering mode: under ideal conditions, the file IO time on the server is not taken into account, and only the rendering time of one frame and the image transmission time of one frame are calculated to represent the loading time of the cloud rendering mode. The server completes one frame of rendering in 16.7ms, the Web side completes one frame in 16.7ms, and the network transmission time of one frame of image (1920x1080x4) / 12.5x1000ms is recorded as the total time Time cloud : Time cloud is calculated as about 34.06ms. On the other hand, in order to ensure the smoothness of the Web rendering, the cloud rendering mode needs to transmit 25 image data to the Web side per second, and the transmission traffic per second is: Therefore, the transmission traffic at 60s is 11880MB. Considering the image compression, with a compression ratio of 10 times as a reference value, the transmission amount at 60s is 1188MB.

[0067] ​The scheme: no matter how to set the camera parameters, the number of visible cubes is at most 100x100x3, about 30,000, and according to the above calculation method, the initial loading time is about 16.7ms*2+39.36s / 100*3=1.21s, and the network transmission flow is about 14.76MB.

[0068] In summary: the scheme only uses about one tenth of the network transmission flow of the cloud rendering method to achieve the cloud rendering effect; and the initial loading time of the grid data loading is much smaller than that of the pure Web rendering method, about 1s, and about 30,000 components drawing will not affect the Web frame rate and interaction, basically meeting the user's visual demand and expectation.

[0069] The embodiment of the application also provides a railway BIM large scene loading system based on edge-end cooperation, which comprises a computer device, the computer device comprises a processor and a memory, the memory stores computer instructions, and the processor is used for executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system realizes the steps realized by the method as described above.

[0070] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps realized by the railway BIM large scene loading method based on edge-end cooperation. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0071] Those skilled in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software or a combination thereof. Whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or communication link.

[0072] It is to be expressly understood that the invention is not limited to the specific configurations and process described above and illustrated in the accompanying drawings. For the sake of clarity, detailed descriptions of known methods are omitted. In the above-described embodiments, several specific steps are described and illustrated as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and various changes, modifications and additions can be made thereto by one of ordinary skill in the art without departing from the spirit of the present invention, and the order of the steps can be changed.

[0073] In the present invention, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0074] The above description is merely illustrative of the application, and is not intended to limit the scope of the application. Various modifications and changes can be made by one of ordinary skill in the art without departing from the spirit and scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application should be included in the scope of the application.

Claims

1. A method for loading large-scale railway BIM scenes based on edge-end collaboration, characterized in that, The method includes the following steps: The edge nodes of the node network obtain model request information sent by the user terminal, the model request information including basic information of the requested model; Based on the model request information, the model data of the corresponding model is loaded, and in response to the model request information sent by the user, the basic information of the model requested by the model request information is fed back to the user. Obtain the camera parameter information sent by the user terminal, and perform off-screen rendering of the BIM model based on the camera parameter information to obtain an image of the model from the camera's perspective; The process involves obtaining the color of each pixel in the image to determine the corresponding component ID, and then determining the corresponding geometric and material information based on the component ID. The pixel color is determined based on the pixel value, and the corresponding component ID is matched against the color. The geometric and material information corresponding to the component ID is then matched against the model data. The geometric and material information corresponding to the component ID is fed back to the user. The number of pixels corresponding to each component ID is determined based on the component ID of each pixel in the image. A sending queue is constructed based on the number of pixels corresponding to each component ID. A validity value is calculated based on the number of pixels corresponding to each component ID, the bounding box volume of the component corresponding to the component ID, and the memory space occupied by the component ID. The validity values ​​corresponding to each component ID are sorted to obtain the sending queue. Based on the order of the sending queue, the geometric and material information corresponding to the component ID in the model data is matched against the component ID and sent, so that the user can load the scene image based on the geometric and material information of the corresponding component ID.

2. The railway BIM large-scene loading method based on edge-end collaboration according to claim 1, characterized in that, The node network also includes a cloud center node. In the step of obtaining model request information sent by the user terminal at the edge node of the node network, the edge node determines whether it has pre-stored the model data corresponding to the model request information. If it has not pre-stored, it initiates a loading request for the corresponding model data to the cloud center node and obtains the model data from the cloud center node.

3. The railway BIM large-scene loading method based on edge-end collaboration according to claim 1, characterized in that, In the step of responding to the model request information sent by the user terminal and feeding back the basic information of the model requested by the model request information to the user terminal, the basic information is obtained from the model data of the corresponding model and fed back to the user terminal so that the user terminal can construct a model organization structure tree based on the basic information.

4. The railway BIM large-scene loading method based on edge-end collaboration according to claim 1, characterized in that, In the step of performing off-screen rendering of the BIM model based on the camera parameter information to obtain an image of the model from the camera's perspective, edge nodes perform off-screen rendering of the model to obtain an image of the model from the perspective corresponding to the camera parameter information.

5. The railway BIM large-scene loading method based on edge-end collaboration according to claim 1, characterized in that, In the step of calculating the validity value based on the number of pixels corresponding to the component ID, the bounding box volume of the component corresponding to the component ID, and the memory space occupied by the component corresponding to the component ID, the validity value is calculated using the following formula: ； in, This represents the validity value corresponding to the component ID. This indicates the number of pixels corresponding to the component ID. This indicates the volume of the mesh bounding box corresponding to the component ID. This indicates the memory space occupied by the component corresponding to the component ID.

6. The railway BIM large-scene loading method based on edge-end collaboration according to claim 1, characterized in that, In the step of feeding back the geometric and material information of the corresponding component ID to the user, it is determined whether there is a record in the history of feeding back the geometric and material information of the same component ID to the user. If there is, it is not necessary to send the geometric and material information corresponding to the component ID again.

7. A railway BIM large-scene loading system based on edge-end collaboration, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 6.

Images