Panoramic engineering data generation methods, devices, electronic equipment, and readable storage media

Abstract

This disclosure relates to the field of data processing technology, and provides a method, apparatus, electronic device, and readable storage medium for generating panoramic engineering data. The method includes: fusing a collection of multi-view engineering images to obtain panoramic index data; extracting data from the panoramic index data based on preset state requirements to obtain an index subset and corresponding image block data; combining the index subset and image block data to obtain lightweight panoramic engineering data; performing local caching and stitching processing on the lightweight panoramic engineering data to obtain data to be rendered; and performing hierarchical rendering processing on the data to be rendered to obtain target panoramic engineering data. This improves data loading speed, enhances terminal display smoothness, increases panoramic engineering visualization efficiency, improves the comprehensiveness of information display, enhances interactivity, and increases the richness and accuracy of annotation information.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and readable storage medium for generating panoramic engineering data. Background Technology

[0002] Currently, real estate project displays mainly rely on flat images, text descriptions, and simple 3D models. However, because they cannot fully present the spatial layout and surrounding environment of the project, users find it difficult to form an intuitive understanding, and there is a lack of interactivity. When displayed on mobile devices, there are problems such as slow loading and poor image quality. In order to improve the display effect, high-precision 3D (Three-Dimensional) modeling is usually used to enhance the visual effect, but the production cost is high and the data volume is large. Alternatively, a dedicated mobile application can be developed to optimize the display, but cross-platform compatibility is poor and maintenance is complex.

[0003] It is evident that existing technologies suffer from problems such as a lack of overall spatial layout and surrounding environment display, strong constraints from terminal device performance and network environment, insufficient annotation methods, resulting in missing information display, poor interactivity, and difficulty in information annotation. Summary of the Invention

[0004] In view of this, the present disclosure provides a panoramic engineering data generation method, apparatus, electronic device and readable storage medium to solve the problems in the prior art that, due to the lack of overall spatial layout and surrounding environment display, strong constraints from terminal device performance and network environment, insufficient annotation information, resulting in missing information display, poor interactivity and difficulty in information annotation.

[0005] A first aspect of this disclosure provides a method for generating panoramic engineering data, comprising: fusing a collection of multi-view engineering images to obtain panoramic index data; extracting the panoramic index data based on preset state requirement conditions to obtain an index subset and image block data corresponding to the index subset; combining the index subset and image block data to obtain lightweight panoramic engineering data; performing local caching and stitching processing on the lightweight panoramic engineering data to obtain data to be rendered; and performing hierarchical rendering processing on the data to be rendered to obtain target panoramic engineering data, wherein the target panoramic engineering data can be displayed by a target terminal device.

[0006] In some embodiments, the data to be rendered is subjected to hierarchical rendering processing to obtain target panoramic engineering data, including: hierarchical identification processing of the data to be rendered based on preset terminal device attribute data to obtain terminal device adaptation conditions; multi-resolution layer segmentation processing of the data to be rendered based on the terminal device adaptation conditions to obtain layered rendering data; and dynamic compositing processing of the layered rendering data to obtain target panoramic engineering data.

[0007] In some embodiments, combining the index subset and image patch data to obtain lightweight panoramic engineering data includes: compressing and encoding the image patch data to obtain compressed image units; performing structured encapsulation processing on the index subset and compressed image units to obtain lightweight data packets; and performing metadata annotation processing on the lightweight data packets to obtain lightweight panoramic engineering data.

[0008] In some embodiments, the panoramic index data is extracted and processed based on preset state requirement conditions to obtain an index subset and corresponding image patch data, including: performing spatial region division processing on the panoramic index data to obtain a set of region labels; performing matching and filtering processing on the set of region labels based on preset state requirement conditions to obtain an index subset; and performing image patch extraction processing on the panoramic index data according to the index subset to obtain image patch data.

[0009] In some embodiments, the process of fusing a set of multi-view engineering images to obtain panoramic index data includes: aligning the set of multi-view engineering images to obtain image alignment parameters; stitching and fusing the set of multi-view engineering images based on the image alignment parameters to obtain an initial panoramic engineering image; and performing spatial index encoding on the initial panoramic engineering image to obtain panoramic index data.

[0010] In some embodiments, local caching and stitching processing is performed on lightweight panoramic engineering data to obtain data to be rendered, including: performing cache availability judgment processing on the lightweight panoramic engineering data to obtain cache status information; performing effective data filtering processing on the lightweight panoramic engineering data based on the cache status information to obtain a set of effective image block data; and performing spatial stitching processing on the set of effective image block data to obtain data to be rendered.

[0011] In some embodiments, dynamic synthesis processing is performed on the layered rendering data to obtain target panoramic engineering data, including: incremental decoding processing of the layered rendering data to obtain an incremental image layer; and inter-frame interpolation smoothing processing of the incremental image layer to obtain target panoramic engineering data.

[0012] A second aspect of this disclosure provides a panoramic engineering data generation apparatus, comprising: a first processing module for fusing a collection of multi-view engineering images to obtain panoramic index data; a second processing module for extracting the panoramic index data based on preset state requirement conditions to obtain an index subset and corresponding image block data; a third processing module for combining the index subset and image block data to obtain lightweight panoramic engineering data; a fourth processing module for locally caching and stitching the lightweight panoramic engineering data to obtain data to be rendered; and a fifth processing module for performing hierarchical rendering on the data to be rendered to obtain target panoramic engineering data, wherein the target panoramic engineering data is used for display on a target terminal device.

[0013] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0014] A fourth aspect of this disclosure provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of this embodiment compared with the prior art are as follows: Panoramic index data is obtained by fusing a collection of multi-view engineering images; the panoramic index data is extracted based on preset state requirements to obtain an index subset and corresponding image block data; the index subset and image block data are combined to generate lightweight panoramic engineering data; the lightweight panoramic engineering data is locally cached and stitched to form data to be rendered; the data to be rendered is sent to a hierarchical rendering process to output target panoramic engineering data that can be displayed on the target terminal device. This improves data loading speed, enhances terminal display smoothness, increases panoramic engineering visualization efficiency, improves the comprehensiveness of information display, enhances interactivity, and increases the richness and accuracy of annotation information. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure; Figure 2This is a flowchart illustrating a panoramic engineering data generation method provided in an embodiment of this disclosure; Figure 3 This is a flowchart illustrating another panoramic engineering data generation method provided in this embodiment of the disclosure; Figure 4 This is a schematic diagram of the structure of a panoramic engineering data generation device provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0019] It should be noted that the user information (including but not limited to terminal device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0020] A panoramic engineering data generation method and apparatus according to an embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 1, 2, and 3, server 4, and network 5.

[0022] Terminal devices 1, 2, and 3 can be hardware or software. When terminal devices 1, 2, and 3 are hardware, they can be various electronic devices with displays that support communication with server 4, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 1, 2, and 3 are software, they can be installed in the aforementioned electronic devices. Terminal devices 1, 2, and 3 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not impose any limitations on this. Furthermore, various applications can be installed on terminal devices 1, 2, and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0023] Server 4 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 4 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This disclosure embodiment does not limit this.

[0024] It should be noted that server 4 can be either hardware or software. When server 4 is hardware, it can be various electronic devices that provide various services to terminal devices 1, 2, and 3. When server 4 is software, it can be multiple software programs or software modules that provide various services to terminal devices 1, 2, and 3, or it can be a single software program or software module that provides various services to terminal devices 1, 2, and 3. This disclosure does not limit the scope of the embodiments.

[0025] Network 5 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), and Infrared. This disclosure does not limit the scope of the network.

[0026] Users can establish a communication connection with server 4 via network 5 through terminal devices 1, 2, and 3 to receive or send information. Specifically, server 4 can acquire a set of multi-view engineering images through terminal devices 1, 2, and 3, perform fusion processing on the multi-view engineering image set to obtain panoramic index data; extract the panoramic index data based on preset state requirement conditions to obtain an index subset and the corresponding image block data; perform combination processing on the index subset and the image block data to generate lightweight panoramic engineering data; perform local caching and stitching processing on the lightweight panoramic engineering data to form data to be rendered; and send the data to be rendered to the hierarchical rendering processing stage to output target panoramic engineering data that can be displayed on the target terminal device.

[0027] It should be noted that the specific types, quantities, and combinations of terminal devices 1, 2, and 3, server 4, and network 5 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any restrictions on this.

[0028] Figure 2 This is a flowchart illustrating a panoramic engineering data generation method provided in an embodiment of this disclosure. Figure 2 The method for generating panoramic engineering data can be derived from... Figure 1 The server executes this. For example... Figure 2 As shown, the method for generating panoramic engineering data includes: S201, perform fusion processing on the multi-view engineering image set to obtain panoramic index data.

[0029] Specifically, a multi-view engineering image set can be a collection of image data containing information about the main body of the project and its surrounding environment, collected from different angles and positions at the project site. The multi-view engineering image set can take the form of a series of images with overlapping areas in space, which together cover the all-round perspective of the target project. This multi-view engineering image set can be used to ensure that the generated panorama can present the overall appearance of the project completely and without omission. This multi-view engineering image set can be obtained by taking pictures of the target area from multiple angles and positions using image acquisition equipment deployed at the project site.

[0030] Fusion processing is a data processing procedure that spatially aligns, stitches, and optimizes individual images in a multi-view engineering image set to form a continuous and seamless panoramic image. This fusion processing can eliminate parallax and geometric deformation between images through computer vision algorithms, achieving pixel-level stitching and fusion. This allows the construction of a unified, large-scale visual scene, providing a basic image carrier for subsequent information extraction and indexing.

[0031] Furthermore, the fusion process can be implemented using a panoramic stitching algorithm. Feature points can be extracted from each input image, where they can be the center points of local regions with significant differences. Feature point matching can be performed between different images to determine corresponding feature point pairs in overlapping areas. Based on the successfully matched feature point pairs, the spatial transformation relationship between images can be calculated, and each image can be projected onto a unified coordinate system. Pixel fusion and seam optimization can be performed on the projected images to eliminate stitching artifacts and generate a visually coherent panoramic image.

[0032] Panoramic index data can be structured data extracted from panoramic images obtained through fusion processing, which can be used to associate and locate specific regions inside the panoramic image or external related information. The form of panoramic index data can be a collection of metadata such as location coordinates, feature descriptors, and association identifiers. This panoramic index data can be used to establish a mapping relationship between the spatial location of the panoramic image and specific engineering information.

[0033] In addition, the process of fusion processing to generate panoramic images can also include adjustments to the consistency of image color and brightness to ensure that the stitched panoramic image has a natural and unified visual effect without obvious color differences or brightness abrupt changes.

[0034] Furthermore, consistency adjustment can be achieved through histogram matching or color transfer processing, which can apply the color distribution characteristics of the reference image to the image to be stitched, thereby achieving overall color tone coordination.

[0035] In addition, after the panoramic image is generated, in order to construct panoramic index data, key information areas can be identified and labeled in the panoramic image. These key information areas may include, but are not limited to, entrances and exits of engineering projects, locations of key facilities, and / or prominent landmarks in the field of view.

[0036] For example, in a commercial complex construction project, the system uses drones equipped with high-definition lenses and panoramic cameras fixed on ground tripods to collect a set of multi-view engineering images of the main building and surrounding roads and green areas of the commercial complex from multiple preset flight waypoints and ground stations. These images include the four facades (east, west, south, and north) and a top-down view. Feature point detection and matching can be performed on these images to calculate the relative positions and orientations of each image. A spherical projection model can be used to fuse all the images into a complete 3D panoramic view of the construction site, allowing for a 360-degree surround view. From the generated panoramic view, key areas such as the main entrance, tower crane locations, material storage areas, and major road intersections within the field of view can be identified. For each key area, a panoramic index data entry is generated, containing its spherical coordinate range in the panoramic view, a visual feature description vector, and a unique area identifier (ID).

[0037] This application embodiment obtains panoramic index data by fusing a collection of multi-view engineering images. It merges discrete multi-view images into a unified panoramic view and extracts a structured index from it. By generating panoramic index data, the spatial location information in the panoramic images is transformed into structured data that can be processed by a computer, thereby improving processing and response efficiency. Through a standardized data processing flow, the original image collection is transformed into a standardized data carrier that can be used for further annotation and interaction. This provides a core data preprocessing step for the digital management and application of engineering data, improves data loading speed, enhances the smoothness of terminal display, and improves the visualization efficiency of panoramic engineering.

[0038] S202, based on preset state requirement conditions, extract and process the panoramic index data to obtain the index subset and the image block data corresponding to the index subset.

[0039] Specifically, the preset state requirement conditions can be data filtering rules predefined according to user intent or specific display requirements. These preset state requirement conditions can be used to quickly locate and filter data subsets related to the current display target in the panoramic index data. The preset state requirement conditions can also come from user interaction operations. For example, if a user clicks or selects a specific point of interest on the mobile interface, filtering conditions corresponding to the display requirements of that point of interest can be generated.

[0040] The extraction process can be a data processing procedure that traverses and matches the panoramic index data structure according to the filtering logic contained in the preset state requirements, and identifies the set of all image patch identifiers that meet the conditions. This extraction process can be used to achieve on-demand data filtering and avoid loading irrelevant data.

[0041] Furthermore, in the specific implementation of the extraction process, the spoken language analyzes the preset state requirement conditions, which may include spatial range coordinates, label type and / or keywords, etc., without limitation here. The preset state requirement conditions can be compared with the information recorded in each index entry in the panoramic index data. For example, it can check whether the center coordinates of a certain image patch fall within the specified spatial range, or whether the label information associated with the image patch contains the specified keywords. All image patch identifiers that pass the comparison can be collected to form an index subset.

[0042] In addition, the index subset can be a subset of the panoramic index data. This index subset can contain the index information of all image blocks that meet the preset state requirements. This index subset can be used to limit the range of data that needs to be loaded from the storage system. The image block data corresponding to the index subset can be the actual image data blocks pointed to by each identifier in the index subset. The image block data corresponding to the index subset can be the basic unit that constitutes the target display screen. In this way, it can be stitched and rendered in the mobile rendering engine to obtain a partial or complete panoramic view from the user's current perspective.

[0043] Furthermore, when acquiring image block data, a data request can be initiated to the data storage management system based on each identifier in the index subset. The storage management system can locate the corresponding image data block based on the identifier. The image data block can be stored in a compressed format and can be read and transmitted to the mobile device for display.

[0044] This application embodiment extracts panoramic index data by pre-set state requirement conditions, and filters relevant index subsets and image block data according to the user's real-time interaction intent, realizing on-demand data loading, reducing the data processing volume and network transmission burden of mobile devices. Since the index subset limits the data range, it improves the response speed of panoramic image rendering, enhances the on-demand loading speed, improves the smoothness of mobile device interaction, and improves data traffic utilization.

[0045] For example, in a mobile application showcasing a real estate project, when a user browses a panoramic view of a building using a mobile app, they can click on a hotspot icon corresponding to a children's playground within the complex. This generates a preset status requirement, which includes spatial coordinate information within a certain radius centered on the playground. Based on this preset status requirement, the panoramic index data of the building, stored on the server, can be extracted and processed. The panoramic index data records the identifiers, 3D coordinates, and associated annotations of each image patch into which the panoramic view of the building is divided. The extraction process compares this spatial coordinate requirement with the center coordinates of each image patch, filtering out all image patches whose coordinates fall within the specified circular area. For example, 50 relevant image patch identifiers might be obtained, and this set of identifiers constitutes a subset of the index.

[0046] S203 combines the index subset and image patch data to obtain lightweight panoramic engineering data.

[0047] Specifically, the combination processing can be a data processing process that associates and encapsulates the index subset with the corresponding image block data based on predetermined rules. Through combination processing, spatial positioning information (i.e., the index subset) and actual image content (i.e., image block data) can be organized together to form a structured data packet, which is convenient for network transmission and client parsing and rendering.

[0048] Furthermore, the combined processing can be to use a subset of the index as the header information of the data packet and the corresponding image block data as the main content, and then perform binary stitching and encapsulation. Alternatively, a lightweight project file format can be constructed, in which dedicated data segments are defined to store the index information and image block data respectively, and a mapping relationship between the two is established. In this way, after receiving the lightweight panoramic project data, the client can parse the index and locate and load the corresponding image block according to the index.

[0049] For example, in a mobile application for showcasing a real estate project, when a user swipes the screen to view the landscape on the east side of the community, the system can predict the user's next viewing point based on the direction of the swipe. Based on this, it can filter out index entries covering the eastern landscape area from the full panoramic image index, forming an index subset. Based on this index subset, it can read the corresponding compressed and encoded eastern landscape image block data from distributed storage. Through combined processing, the image block data and the index subset can be packaged according to a custom encapsulation format to generate a lightweight data package, namely lightweight panoramic engineering data.

[0050] In addition, lightweight panoramic engineering data can be a structured data package generated through combined processing, containing necessary index information and corresponding image content, or it can be a panoramic scene data format optimized for efficient transmission and rendering. This lightweight panoramic engineering data can serve as a standard data unit for transmission between the annotation end and the mobile display end, or between the server and the mobile client. This lightweight panoramic engineering data eliminates redundant data that is not necessary for the current view, ensuring that the mobile device can load and display panoramic content in a limited network environment.

[0051] Furthermore, lightweight panoramic engineering data can also include auxiliary information such as data version information, compression algorithm identifiers, and scene metadata to support correct decoding and rendering by the client.

[0052] This application embodiment pre-stores a full panoramic index and compressed image block data. When a user's viewpoint changes and triggers a request, it can extract a subset of the index for the corresponding region and retrieve matching image block data. The index subset is used as the header, and the image block data is used as the body. Based on predetermined rules, binary stitching and encapsulation are performed to construct a lightweight project file format. A mapping relationship between index information and image block data is established within the lightweight project file format. After adding data version information, compression algorithm identifier, and scene metadata, lightweight panoramic project data is generated. This improves transmission efficiency, enhances loading stability in weak network environments, increases panoramic rendering speed, improves the smoothness of mobile terminal display, enhances information accuracy, and improves user experience.

[0053] S204 performs local caching and stitching processing on lightweight panoramic engineering data to obtain the data to be rendered.

[0054] Specifically, local caching can be a specific area on the storage medium (such as flash memory) of mobile devices (such as mobile phones and tablets) that can be used to temporarily store data. Local caching can be used to store downloaded lightweight panoramic engineering data or its corresponding segments. When users browse the same or adjacent areas again, there is no need to repeatedly request all the data from the remote server, which reduces network latency and data traffic consumption and improves response speed.

[0055] Furthermore, local cache stitching processing can combine, connect, or decode multiple scattered data fragments stored in the local cache according to a predetermined logic and order to reconstruct complete or partially continuous data units. Local cache stitching processing can be used to dynamically combine lightweight panoramic engineering data that exists in the cache in a block or layered form, according to the user's current viewport or the area to be viewed, to obtain data packets that can be used for graphics rendering.

[0056] The data to be rendered can be a set of data that has been preprocessed and assembled, conforms to the rendering engine requirements, and contains all necessary information such as vertices, textures, materials, and marker positions, which can be used to generate the pixel image of the target screen. This data to be rendered can be obtained from the output of the stitching process, that is, generated after the format is adapted or the buffer is filled on the lightweight panoramic engineering data that has been stitched.

[0057] In addition, when receiving a user's browsing command or performing scene preloading, the system can query the local cache to see if the required lightweight panoramic engineering data exists. This query can be based on the project identifier, scene coordinates, or the unique hash value of the data block. If all or part of the required data exists in the local cache, it can be read from the cache.

[0058] Furthermore, the cache query strategy can be a multi-level index, which can check the fast cache area for frequently accessed data, and then check the general storage area to improve data retrieval efficiency.

[0059] In addition, the stitching process includes, but is not limited to, spatial alignment of data blocks, stitching of texture images, and transformation and integration of annotation information coordinates.

[0060] For example, when a user swipes the screen to switch perspectives, multiple texture tiles corresponding to the current field of view can be retrieved from the cache and stitched together based on the texture coordinates of the lightweight panoramic engineering data on the panoramic spherical model. The information of the annotation points located within the field of view (such as school and shopping mall icons and their attribute data) can be extracted from the cache and associated with the texture image and calibrated to form a composite data package containing a panoramic background layer and an annotation information layer, which is the data to be rendered.

[0061] This application embodiment uses project identifiers, scene coordinates, or unique hash values ​​of data blocks as keywords to query whether the required lightweight panoramic engineering data is already stored in the local cache; it performs local cache stitching processing on multiple block and layered lightweight panoramic engineering data in the cache, and completes the spatial position alignment of data blocks, texture image stitching, and coordinate transformation and integration of annotation information based on predetermined logic, dynamically combining them into a continuous and complete data package that conforms to the rendering engine format. Then, through format adaptation and buffer filling, it generates rendering data containing vertices, textures, materials, and annotation point positions. In this way, it improves the response speed of repeated access, enhances the data availability in weak network environments, improves bandwidth utilization, improves the efficiency of data preparation before rendering, and enhances the smoothness of user experience.

[0062] S205, the data to be rendered is processed by hierarchical rendering to obtain the target panoramic engineering data, which can be displayed by the target terminal device.

[0063] Specifically, hierarchical rendering is a data processing process that converts the data to be rendered into visual data with different levels of detail, based on the performance and display requirements of the target terminal device. The target panoramic engineering data can be a data format formed through hierarchical rendering that can be used for display on the terminal device. The target terminal device can be a device used to receive and display the target panoramic engineering data, and it can carry the display of panoramic content and interact with the user.

[0064] Furthermore, hierarchical rendering can divide the data to be rendered into multiple levels of detail based on the hardware performance parameters and network connectivity of the target terminal device. Hardware performance parameters can include the computing power of the graphics processing unit, memory capacity, and screen resolution. For high-performance terminal devices, high-level-of-detail rendering tasks can be assigned; for performance-constrained terminal devices, lower-level-of-detail rendering tasks can be assigned. Network connectivity can affect data transmission speed; under low bandwidth conditions, lower-level-of-detail rendering tasks with smaller data volumes can be prioritized. Based on the above division, the 3D model in the data to be rendered can be simplified, reducing the number of polygons; texture maps can be compressed to reduce resolution; and annotation information can be filtered, retaining only key information points.

[0065] Furthermore, the levels of detail can be categorized into high, medium, and low levels. High-level detail can contain models and textures with original precision, suitable for high-performance personal computers (PCs) or workstations; medium-level detail can simplify the model, and textures can be compressed, suitable for mainstream mobile devices; low-level detail can use highly simplified models and basic textures, suitable for mobile devices with lower performance or poor network environments.

[0066] In addition, the specific process of performing hierarchical rendering can be to identify the type and capabilities of the target terminal device, select the corresponding rendering strategy from the pre-established hierarchical rule base based on the identification results, and call the corresponding rendering engine according to the selected strategy. The rendering engine can read the data to be rendered and apply model simplification, texture compression and data clipping algorithms to generate target panoramic engineering data that matches the target terminal device and contains specific levels of detail.

[0067] Furthermore, the rendering process can use a progressive generation method to generate a basic low-detail panoramic framework, ensuring that the terminal device can display the initial image. Then, based on user interaction commands or preloading strategies, higher-detail data blocks can be gradually generated and transmitted in the background to refine and enhance the displayed image.

[0068] For example, at the sales office, a high-end workstation can be used to display a panoramic view of the community to customers. If the graphics processing unit has strong computing power, sufficient memory, and high screen resolution, a high level of detail strategy can be invoked to preserve the original precision of the 3D model and high-resolution texture, generating high-detail target panoramic engineering data.

[0069] This application embodiment identifies the hardware performance parameters and network connection status of the target terminal device, selects corresponding strategies from the hierarchical rule base, calls the rendering engine to perform model simplification, texture compression, and annotation information filtering on the data to be rendered, divides it into three levels of detail (high, medium, and low), generates target panoramic engineering data that matches the device, and sends out the low-detail panoramic framework in a progressive generation manner to ensure immediate display. Then, based on user interaction or preloading strategies, higher-detail data blocks are gradually uploaded in the background to complete the refinement. This improves cross-device compatibility, enhances availability in weak network environments, increases initial loading speed, improves bandwidth utilization, and enhances the continuity and smoothness of the user experience.

[0070] According to the technical solution provided in this disclosure, by fusing discrete multi-view images into a unified panoramic view and extracting structured indexes in real time, spatial location information is transformed into standardized data that can be processed by computers, thereby improving processing and response efficiency. On-demand loading of index subsets and image block data is achieved through preset state requirement conditions, reducing the amount of data processing and network transmission burden on mobile devices. Lightweight encapsulation and local caching improve the response speed for repeated accesses and enhance loading stability in weak network environments. Hierarchical rendering and progressive generation ensure cross-device compatibility, fast initial loading, and high bandwidth utilization, enhancing the smoothness, accuracy, and user experience of panoramic engineering data visualization.

[0071] In some embodiments, the data to be rendered is subjected to hierarchical rendering processing to obtain target panoramic engineering data, including: hierarchical identification processing of the data to be rendered based on preset terminal device attribute data to obtain terminal device adaptation conditions; multi-resolution layer segmentation processing of the data to be rendered based on the terminal device adaptation conditions to obtain layered rendering data; and dynamic compositing processing of the layered rendering data to obtain target panoramic engineering data.

[0072] Specifically, the preset terminal device attribute data can be a pre-configured set of data that can be used to characterize the hardware and software performance parameters of different types of mobile terminal devices, including but not limited to screen resolution, graphics processing unit (GPU) computing power, memory capacity, and operating system version. This preset terminal device attribute data can be used to provide a basis for hierarchical identification processing, so that the rendering strategy can be adapted to different device capabilities. This preset terminal device attribute data can be obtained by collecting the specifications of mainstream mobile devices on the market and establishing a device performance archive.

[0073] Furthermore, the hierarchical recognition processing can be a data processing process that determines the data level and format suitable for loading and rendering on the current terminal device by comparing the data volume and complexity of the data to be rendered with the preset performance parameter thresholds in the terminal device attribute data. Among them, the terminal device adaptation conditions can be the output results obtained from the hierarchical recognition processing. The form of the terminal device adaptation conditions can be a set of specific rendering strategy parameters, such as the maximum supported panoramic image resolution, the maximum number of annotation information displayed at the same time, whether to enable advanced lighting and shadow effects, etc., which are not limited here.

[0074] For example, in the mobile display scenario of a real estate project, the terminal device attribute data preset by the mobile phone can be obtained, such as identifying that the screen of the mobile phone is 1080 progressive scanning resolution (1080P) and the GPU model is mid-range. The rendering data of the real estate project can be classified and processed. By comparing the rendering data containing high-resolution panoramic images and surrounding facility annotation information with the obtained mobile phone performance parameters, terminal device adaptation conditions can be generated. The terminal device adaptation conditions may include limiting the highest resolution of the panoramic image to about 2000 pixels (2K resolution) and stipulating that no more than 20 annotation points are displayed on the same screen.

[0075] In addition, multi-resolution layer segmentation processing can be a data processing process that decomposes the original high-resolution panoramic image data into multiple image layers with different levels of detail, based on the resolution limitations in the terminal device adaptation conditions. For example, it can generate a low-resolution base layer that covers the entire image, and multiple high-resolution detail layers that cover local areas. This enables progressive loading and rendering, prioritizing the display of the overall image under limited device performance and network bandwidth, and then dynamically loading details as needed.

[0076] Layered rendering data can be a collection of image data blocks organized based on detail levels, obtained through multi-resolution layer segmentation. This layered rendering data can provide structured image materials for the compositing engine. It can be obtained by processing the panoramic image in the rendering data using image pyramid construction algorithms or tile cutting algorithms, based on the terminal device adaptation conditions. For example, based on the above application scenario, multi-resolution layer segmentation can be performed on the panoramic image of the real estate project according to the generated terminal device adaptation conditions. The original high-definition panoramic image can be cut into multiple layers of tiles. The base layer can be low-resolution tiles covering the entire scene, while higher layers can be high-resolution tiles corresponding to different areas of the scene, thus generating a set of layered rendering data. Annotation information data can also be filtered and grouped according to device adaptation conditions and associated with the corresponding image layers.

[0077] Furthermore, dynamic compositing processing allows for the real-time selection and assembly of image fragments and annotation information from layered rendering data, tailored to the current screen display, based on real-time user browsing actions (such as perspective changes and zoom levels). This ensures smooth user interaction, loading and rendering high-detail content only when needed, thus optimizing resource usage. For example, based on the above application scenario, when a user begins to swipe through the community environment of a real estate project, corresponding image tiles can be selected and stitched together in real-time from the layered rendering data according to the user's current perspective and zoom level. When the user zooms in to view the details of the greenery in the center of the community, a high-resolution tile layer of that area can be dynamically loaded and composited, along with relevant annotation information (such as vegetation type descriptions), thereby generating and presenting the target panoramic engineering data for the user to browse in real time.

[0078] According to the technical solution provided in this disclosure, by performing hierarchical identification based on terminal device attribute data, adaptation conditions are generated, and multi-resolution layer segmentation and dynamic synthesis are performed accordingly. This allows the rendering of panoramic engineering data to be adapted to mobile terminals with different performance levels, thereby optimizing the data loading speed and rendering smoothness of mobile terminals. Based on preset terminal device attribute data, hierarchical identification processing is performed on the data to be rendered to obtain terminal device adaptation conditions. According to the terminal device adaptation conditions, multi-resolution layer segmentation processing is performed on the data to be rendered to obtain layered rendering data composed of a low-resolution base layer and multiple local high-resolution detail layers. Through dynamic synthesis processing, corresponding image tiles and annotation information are selected and assembled from the layered rendering data in real time according to the user's real-time browsing operation to generate target panoramic engineering data. This improves the accuracy of cross-terminal differentiated adaptation, enhances the progressive loading stability under weak network and low-performance devices, and improves the screen continuity and rendering response speed during user interaction.

[0079] In some embodiments, combining the index subset and image patch data to obtain lightweight panoramic engineering data includes: compressing and encoding the image patch data to obtain compressed image units; performing structured encapsulation processing on the index subset and compressed image units to obtain lightweight data packets; and performing metadata annotation processing on the lightweight data packets to obtain lightweight panoramic engineering data.

[0080] Specifically, compression encoding is a data processing procedure that applies data compression to image block data to reduce its data volume, thereby reducing the data size of a single image unit and creating conditions for efficient transmission and loading on mobile devices. The compressed image unit can be an image data unit obtained through compression encoding. The data volume of the compressed image unit is smaller than that of the original image block data, but it maintains acceptable visual quality and can be used as an indexed and retrieved data entity in structured encapsulation.

[0081] Furthermore, standard compression algorithms suitable for image data can be used, such as the Joint Photographic Experts Group (JPEG) image format based on transform coding or the Web Picture format (WebP) based on predictive coding, to independently encode each image block data and generate a corresponding set of compressed image units.

[0082] In addition, structured encapsulation processing can be a data processing process that integrates index subsets and compressed image units into a logical whole according to a predefined data organization format, thereby generating a well-structured, easy-to-parse, and on-demand loaded composite data file; wherein, the lightweight data package can be a single data file or data stream obtained through structured encapsulation processing, which contains an index subset that can be used for spatial positioning and the corresponding compressed image unit data.

[0083] Furthermore, the encapsulation format can use index information as a header or metadata section, and compressed image units as data bodies arranged sequentially or interleaved. For example, it can be encapsulated in a custom binary format or through an existing container format (such as a custom archive file) to ensure the accuracy of the correspondence between the index subset and the compressed image units.

[0084] In addition, metadata annotation processing can refer to the data processing process of attaching descriptive information to lightweight data packets. The descriptive information includes, but is not limited to, data packet version, creation time, associated project identifier, original resolution of panoramic image, spatial coordinate system information and / or data integrity check code, etc., so as to provide necessary auxiliary information for the identification, verification, management and correct parsing of data packets.

[0085] Furthermore, metadata can be written to specific locations in the lightweight data package (such as the file header or footer) in the form of key-value pairs, or it can be generated as a separate configuration file associated with the lightweight data package.

[0086] For example, in the mobile panoramic display of real estate projects, all image block data can be compressed and encoded using the WebP format compression algorithm to generate a series of compressed image units. The selected index subset and the corresponding compressed image units can be encapsulated based on a custom binary structure. The index information can be placed at the beginning of the file, which can then be a continuous data block of compressed image units, forming a lightweight data packet. Metadata can be attached to this data packet, including the project identifier (ID), which can be "Project_A_Model_Room", the total size of the panoramic image, the spatial coordinate system used (e.g., spherical coordinate system), and the hash check value of the data packet. In this way, a lightweight panoramic engineering data file that can be distributed online can be generated.

[0087] According to the technical solution provided in this disclosure, image block data is compressed using a Joint Image Experts Group (JEP) image format based on transform coding or a network image format based on predictive coding to obtain compressed image units with smaller data volume and acceptable visual quality. The index subset and compressed image units are then structured and encapsulated with a header and data body to form a lightweight data packet. Metadata annotation, including project identifiers, total panoramic dimensions, spatial coordinate system, and hash checksums, is added in key-value pairs to generate lightweight panoramic engineering data that can be distributed over the network. This improves mobile network transmission efficiency, enhances the integrity and correctness verification capabilities of data packets, and improves loading speed and parsing stability in weak network environments.

[0088] In some embodiments, the panoramic index data is extracted and processed based on preset state requirement conditions to obtain an index subset and corresponding image patch data, including: performing spatial region division processing on the panoramic index data to obtain a set of region labels; performing matching and filtering processing on the set of region labels based on preset state requirement conditions to obtain an index subset; and performing image patch extraction processing on the panoramic index data according to the index subset to obtain image patch data.

[0089] Specifically, spatial region division processing can be a data processing process that divides the complete three-dimensional space covering the real estate project and its surrounding environment into segments based on predetermined geometric rules. This discretizes the continuous space into multiple logical regions with clear boundaries, facilitating subsequent information retrieval and management based on these regions.

[0090] Furthermore, spatial region division can be based on geographic coordinate grids, spatial cube division, or logical partitioning according to natural features such as roads and building boundaries in the actual scene. Each divided spatial region can be assigned a unique region label.

[0091] In addition, the preset state requirement conditions can be dynamic filtering instructions generated during mobile user interaction that can be used to limit the scope of information display. These preset state requirement conditions can be query conditions that include specific attribute keywords and / or spatial range constraints. This can locate the information category and spatial location that the user is currently focusing on, thereby enabling the filtering of massive panoramic data.

[0092] Matching and filtering can be a data processing process that compares the attribute keywords contained in the preset state requirement conditions with the metadata information associated with each tag in the set of regional tags, and determines whether the spatial range of the region meets the spatial constraints in the conditions. In this way, a subset of target regions that meet the user's current needs can be determined from all regions, i.e., the index subset.

[0093] In addition, image patch extraction processing can be a data processing procedure that locates and reads the corresponding data block from a database or file system that stores the original panoramic image data blocks based on the index identifier contained in the index subset; in this way, the image data units required to constitute the visualization content of the selected target area can be obtained, i.e., image patch data.

[0094] Furthermore, image block extraction processing can also include concurrently reading multiple data blocks from distributed storage nodes, and can adopt a progressive transmission strategy based on network conditions and terminal performance, prioritizing the transmission of low-resolution or critical area image blocks.

[0095] For example, when a consumer views a 3D panorama of a real estate project on a mobile device, spatial region segmentation can be performed on the stored panoramic index data. This panoramic index data has been acquired by a panoramic camera, processed by an image stitching algorithm, and associated with annotation information. After segmentation, a set of region labels associated with different types of labels such as "commercial area," "residential area," "educational area," and "medical area" can be obtained. Based on preset state requirement conditions, the region label set can be matched and filtered. The keyword "medical" in the preset state requirement conditions is matched with the labels and associated metadata of each region, and the region with a spatial range near the consumer's current viewpoint is filtered out. Thus, an index subset containing only the region where the relevant medical facilities are located is obtained. Based on this index subset, image block extraction processing can be performed from its image storage system to locate and read image block data that corresponds only to the region of the medical facility, and then transmit it to the mobile device.

[0096] According to the technical solution provided in this disclosure, discrete logical regions with region labels are obtained by spatially dividing the panoramic index data. Based on preset state requirement conditions including attribute keywords and / or spatial range constraints, a matching and filtering process is performed on the region label set to determine the index subset that meets the user's current focus requirements. Based on the index subset, the panoramic index data is processed to extract image blocks, and the corresponding image block data is read from the distributed storage in a concurrent or progressive manner. In this way, the retrieval speed of batch panoramic data is improved, the on-demand loading of mobile devices is enhanced in terms of targeting and real-time performance, and the efficiency of network bandwidth utilization and the smoothness of user interaction response are improved.

[0097] In some embodiments, the process of fusing a set of multi-view engineering images to obtain panoramic index data includes: aligning the set of multi-view engineering images to obtain image alignment parameters; stitching and fusing the set of multi-view engineering images based on the image alignment parameters to obtain an initial panoramic engineering image; and performing spatial index encoding on the initial panoramic engineering image to obtain panoramic index data.

[0098] Specifically, alignment processing can be a data processing process that geometrically calibrates each image in a multi-view engineering image set. This alignment processing can specifically be achieved by calculating the feature point matching relationship between images and solving for transformation parameters that can eliminate differences in shooting perspectives. This can eliminate position and angle deviations between images caused by different camera positions and angles. The multi-view engineering image set can be a set of original image data obtained by panoramic camera equipment taking surround shots of a target scene (such as the surrounding environment of a real estate project) from multiple preset points.

[0099] Furthermore, alignment processing can be based on feature point detection and matching algorithms, such as Scale-Invariant Feature Transform (SIFT) or OrientedFAST and Rotated BRIEF (ORB) algorithms, which can extract key feature points from each image and perform feature matching between different images to calculate image alignment parameters that can be used to characterize the transformation relationship between images. These image alignment parameters can include rotation, translation, and / or scale transformation information.

[0100] In addition, stitching and fusion processing can be a data processing process that merges multiple images obtained through geometric calibration into a continuous image covering a wider field of view based on their relative positional relationship in real space. This stitching and fusion processing can include image registration, resampling and fusion of overlapping areas, etc., so as to generate an initial panoramic engineering image that can be used to fully display the entire target scene.

[0101] Furthermore, the stitching and fusion processing can apply algorithms based on multi-band fusion or optimal seam search to perform weighted averaging of pixels in overlapping areas of the image or select the optimal stitching path to eliminate stitching gaps and lighting differences, thereby obtaining a visually coherent and naturally colored initial panoramic engineering image.

[0102] In addition, spatial indexing encoding can be a data processing procedure that assigns a unique identifier or code to each pixel or specific region in an image to represent its spatial location information. Specifically, it can establish a mapping relationship between the two-dimensional pixel coordinates of the image and the real three-dimensional geospatial coordinates, thereby converting the two-dimensional panoramic image into "panoramic index data" with spatial addressing capabilities.

[0103] Furthermore, spatial indexing encoding can calculate the latitude and longitude coordinates or three-dimensional coordinates in a unified coordinate system for each pixel in the initial panoramic engineering image based on the camera position parameters at the time of shooting and the pre-constructed scene 3D point cloud or model. The above coordinate information can be associated and stored with the image data in the form of an index table or mapping function to form panoramic index data.

[0104] For example, a panoramic camera can be used to capture images of a residential community to be showcased, obtaining a set of multi-view engineering images covering key areas such as the community entrance, internal roads, green landscapes, and surrounding shops. This set of multi-view engineering images can be aligned by detecting stable feature points such as building outlines, windows, and streetlights in the engineering images and performing cross-image matching to calculate the image alignment parameters required to calibrate all engineering images to the same virtual viewpoint. Based on the calculated image alignment parameters, multiple local images can be stitched together to generate a complete, seamless initial panoramic engineering image showcasing the community and its surrounding environment in 360 degrees. This initial panoramic engineering image can then undergo spatial indexing and encoding. Based on the camera's GPS location, orientation angle, and the scene's 3D structure information at the time of acquisition, each pixel region in the initial panoramic engineering image can be encoded with its corresponding geographic coordinates. For example, the portion representing the community gate can be encoded as a specific latitude and longitude range, and the portion representing a corner convenience store can be encoded as another set of coordinates, thereby generating panoramic index data containing the mapping relationship between image data and spatial location.

[0105] According to the technical solution provided in this disclosure, by performing alignment processing on a multi-view engineering image set based on scale-invariant feature transformation or directional fast rotation short descriptor, image alignment parameters containing rotation, translation, and / or scale transformation information are obtained. Based on these image alignment parameters, a stitching and fusion process including multi-band fusion or optimal seam search is performed to generate an initial panoramic engineering image. Then, based on GPS location, orientation angle, and scene 3D structure, the initial panoramic engineering image is spatially indexed and encoded, assigning corresponding latitude and longitude coordinates or 3D coordinates in a unified coordinate system to each pixel region, forming panoramic index data. In this way, the geometric accuracy and visual consistency of large-scene panoramic generation are improved, the addressability of pixel-level regions in the panoramic image and real geographic spatial coordinates is enhanced, and the response speed and data utilization efficiency of subsequent on-demand extraction, positioning, and interactive display are improved.

[0106] In some embodiments, local caching and stitching processing is performed on lightweight panoramic engineering data to obtain data to be rendered, including: performing cache availability judgment processing on the lightweight panoramic engineering data to obtain cache status information; performing effective data filtering processing on the lightweight panoramic engineering data based on the cache status information to obtain a set of effective image block data; and performing spatial stitching processing on the set of effective image block data to obtain data to be rendered.

[0107] Specifically, cache availability assessment can be a data processing process that checks whether each component of lightweight panoramic engineering data has been cached, is complete, and has expired by querying the local storage device. This can be used to assess the status of locally available data and provide a basis for subsequent data processing decisions. Among them, cache status information can be a set of data representing the status of each data block in the lightweight panoramic engineering data in the local cache. This cache status information can be generated by cache availability assessment and used to guide the selection of valid data.

[0108] Furthermore, it is possible to iterate through all image block identifiers contained in the lightweight panoramic engineering data, query the device's local file system or dedicated cache database to check if the corresponding file exists, if the file size meets expectations, and if the file's last modification time is within the validity period, and generate a structured status list, i.e. cache status information, by combining the above check results.

[0109] In addition, effective data filtering can be a data processing process that selects data blocks marked as "available" or "complete" from all potential data corresponding to lightweight panoramic engineering data based on cache status information, thereby filtering out invalid or missing data; among them, the effective image block data set can be a collection of image block data that has been confirmed as usable for subsequent image reconstruction after effective data filtering, and the effective image block data set can include panoramic image parts that can be directly loaded from the local environment in the current environment.

[0110] Furthermore, the cache status information can be parsed to identify image block identifiers with a status of "cached and valid". Based on the image block identifier, the corresponding image block file data can be read from the specified directory or database of the local cache. The image block file data can be organized based on its original logical order or spatial index to form a temporary data pool that can be used for merging operations, i.e., a set of valid image block data.

[0111] In addition, spatial stitching processing can be a data processing process that rearranges, aligns and combines scattered, independent image block data according to predefined spatial positional relationships (such as grid coordinates and viewpoint information) to form a larger and more continuous image, thereby restoring the data stored in blocks into a whole image that can be displayed.

[0112] Furthermore, each image block in the valid image block data set can be read, and each image block can be decoded to restore the original bitmap format. Based on the spatial coordinates recorded in the metadata of each image block (e.g., the longitude and latitude range in a spherical panorama, or the face index and texture coordinates (UV) coordinates in a cube map), the bitmap segments with the decoded spatial coordinates can be placed at the corresponding positions in a larger, blank target canvas or texture. For the seam areas between image blocks, pixel-level blending or feathering can be performed to eliminate visual discontinuities. The stitched image data can be converted into the texture format or framebuffer object specified by the rendering engine to generate the data to be rendered.

[0113] For example, in a mobile application scenario for showcasing real estate projects, lightweight panoramic engineering data can include index information that divides the panoramic image into hundreds of image blocks. It can perform cache availability checks to see if the aforementioned image block files are already cached in local storage, and whether the image block files are complete and not expired. It then generates a cache status information list, identifying available and missing image block files. Based on this cache status information, it can perform effective data filtering, loading only the image block files marked as available from local storage to form a set of effective image block data. Furthermore, it can perform spatial stitching on the successfully loaded set of effective image block data, reassembling the effective image block data into a complete panoramic image to be displayed, i.e., the data to be rendered, based on the spatial coordinates of each image block.

[0114] According to the technical solution provided in this disclosure, by performing cache availability judgment processing on lightweight panoramic engineering data, cache status information is obtained. Based on the cache status information, effective data filtering processing is performed, and local complete and unexpired image block files are imported into the effective image block data set. Then, based on the longitude and latitude range in the spherical panorama or the face index and texture coordinates in the cube map, pixel-level spatial stitching processing is performed on the effective image block data set to generate data to be rendered. In this way, the decision efficiency of skipping missing data and loading available data in real time is improved, the local cache hit rate and data integrity in weak network environment are enhanced, and the stitching speed of recombining segmented images into a whole panorama and the real-time performance of data preparation before rendering are improved.

[0115] In some embodiments, dynamic synthesis processing is performed on the layered rendering data to obtain target panoramic engineering data, including: incremental decoding processing of the layered rendering data to obtain an incremental image layer; and inter-frame interpolation smoothing processing of the incremental image layer to obtain target panoramic engineering data.

[0116] Specifically, incremental decoding can be a type of decoding process that decodes incremental image layer data. Since the amount of incremental layer data is smaller than that of a complete frame, it can reduce the computational load and decoding time required for a single decoding operation, enabling mobile devices to obtain scene-updated image information with limited computing resources.

[0117] Furthermore, this incremental decoding process can call the hardware decoder built into the mobile device or an optimized software decoding library to parse the header information of the incremental layer data packet and the compressed pixel data stream, restoring it into an incremental image layer that can be further processed. The incremental image layer can be a decoded image data block containing details of visual changes in a specific area or object in the scene. For example, it can obtain the corresponding layered rendering data packet based on the user's viewpoint rotation. Incremental decoding can be performed on the incremental layer part of the layered rendering data packet to obtain an incremental image layer representing the details of the newly appearing community green belt after the viewpoint rotation.

[0118] In addition, inter-frame interpolation smoothing can be used to generate a series of intermediate transition image frames between incremental image layers at consecutive time points or consecutive frames. When users perform rapid or continuous viewpoint switching operations, smooth intermediate frames can be inserted to simulate a more continuous visual motion trajectory, thereby eliminating the visual jump and improving the smoothness of browsing.

[0119] Furthermore, this inter-frame interpolation smoothing process can specifically analyze the pixel motion vectors between two consecutive incremental image layers. Based on the motion trajectory and temporal relationship, it calculates the position and color value that each pixel should have at the intermediate transition moment, thereby generating a smooth transition image sequence. For example, in the above application scenario, the user can continuously and slowly pan the viewpoint. After obtaining two incremental image layers with different viewpoints through incremental decoding, several frames of smoothly transitioning images can be generated between the two incremental layers through inter-frame interpolation smoothing, so that the panoramic panning process seen by the user on the screen is continuous and without lag.

[0120] In addition, the intermediate frame sequence generated by inter-frame interpolation smoothing can be fused with the decoded base layer and incremental layer keyframes. Based on the transparency, depth information and blending rules of each layer, all layers are superimposed and synthesized into a complete panoramic image frame corresponding to the user's current and transitional viewpoints. A series of continuous panoramic image frames constitute the target panoramic engineering data.

[0121] According to the technical solution provided in this disclosure, incremental decoding processing is performed on the layered rendering data to obtain an incremental image layer with a data volume smaller than that of a complete frame. Inter-frame interpolation smoothing processing based on pixel motion vectors is performed on the incremental image layers of the previous and next frames to generate a continuous transition image sequence. This sequence is then superimposed and fused with the key frames of the base layer and incremental layer based on transparency, depth information, and mixing rules to form a series of continuous panoramic image frames, i.e., target panoramic engineering data. In this way, the decoding efficiency under limited computing resources is improved and the reduction of single decoding load is increased. The smoothness of visual motion trajectory during continuous switching of viewpoints is enhanced, and the browsing smoothness and lag-free experience during rapid or continuous viewpoint changes on mobile devices are improved.

[0122] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0123] Figure 3 This is a schematic diagram of another panoramic engineering data generation method provided in this embodiment of the disclosure. Figure 3 As shown, the method for generating panoramic engineering data includes: 3D Panoramic Data Acquisition and Processing: Using panoramic camera equipment, comprehensive image acquisition is performed on the surrounding environment of the real estate project (such as community green spaces, surrounding roads, and supporting facilities), resulting in a multi-view engineering image set. During the acquisition process, the clarity, integrity, and accuracy of the images are ensured, covering every corner of the project and key surrounding areas.

[0124] An image stitching algorithm is used to seamlessly stitch together a large number of acquired images to generate a complete 3D panoramic image. By extracting and matching image features, the stitching positions are precisely adjusted to eliminate seams and ensure the visual continuity of the panoramic image. Simultaneously, image optimization algorithms are applied to perform color correction, brightness adjustment, and other processing on the stitched panoramic image to improve image quality.

[0125] PC-based information annotation system: A dedicated PC-based annotation system has been developed. This system is based on WebGraphics Library (WebGL) technology and features a simple and easy-to-use interface. Annotators can open 3D panoramic images using this software and annotate the image with project and surrounding information to obtain lightweight panoramic engineering data.

[0126] For surrounding information, the system marks the locations of supporting facilities such as schools, hospitals, shopping malls, and bus stops. Through a combination of map positioning technology and manual annotation, the system accurately marks the locations of these facilities in the panoramic image and adds relevant information, such as facility name, straight-line distance from the project, and time required to walk or drive. It can also mark surrounding road names, traffic flow conditions, and real-time traffic information (such as congested sections and uncongested sections).

[0127] The annotation system has information editing, modification, and management functions. Annotators can edit and modify the annotated information at any time to ensure its accuracy and timeliness. Furthermore, the annotated information can be categorized for easy retrieval and access later.

[0128] Data storage and management: The acquired and processed 3D panoramic data and annotation information are stored on high-performance servers. Distributed storage technology is employed, distributing data across multiple storage nodes to improve data security and retrieval speed. Simultaneously, a comprehensive database management system is established to effectively organize and manage the data.

[0129] The database stores 3D panoramic data and annotation information in a linked manner, achieving precise matching between the two through unique project identifiers and location coordinates. This allows for quick and accurate retrieval of panoramic images and annotation information relevant to the user's current browsing location when displayed on mobile devices.

[0130] Mobile Display System: Developed using HyperText Markup Language 5 (H5) to adapt to various mobile devices (such as smartphones and tablets). The application employs an optimized graphics rendering engine and is customized for the hardware performance and screen size of mobile devices to ensure smooth loading and display of 3D panoramic images on mobile devices.

[0131] The application features a simple and intuitive interface, allowing users to freely browse 3D panoramic images simply by touching the screen. Users can use swipes, zooms, and rotations to view the real estate project and its surrounding environment from all angles.

[0132] To improve loading speed and user experience on mobile devices, the application employs data caching technology and a progressive loading strategy.

[0133] For panoramic images, a progressive loading method is adopted, loading low-resolution images first, and then loading high-resolution images gradually according to the user's browsing operations, thereby reducing the initial loading time.

[0134] According to the technical solution provided in this disclosure, 3D panoramic technology allows for a comprehensive and multi-angle view of the surrounding environment of a real estate project. Combined with detailed information annotations, this provides a more complete and intuitive understanding of the project. Both the PC-based annotation system and the mobile-based display system offer rich interactive functions, allowing users to freely explore project details and query and filter information according to their needs, thus enhancing user engagement and experience. Optimized for mobile devices, the system employs a graphics rendering engine, data caching technology, and a progressive loading strategy to ensure fast loading and smooth display of 3D images on mobile devices, resolving the issue of poor display effects on existing mobile devices. The PC-based annotation system provides accurate annotation of the project and its surrounding information, including key information such as location, attributes, and distance, offering accurate and detailed reference materials. The web-based annotation tool eliminates the need for annotators to install dedicated software; they can log in to the annotation system through a browser, improving ease of operation and overcoming limitations imposed by operating systems and devices. In terms of graphics rendering, WebGL technology enables the rendering and display of 3D panoramic images in a web browser without the need to develop a dedicated mobile application, reducing development costs and improving the application's cross-platform compatibility.

[0135] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0136] Figure 4 This is a schematic diagram of a panoramic engineering data generation device provided in an embodiment of this disclosure. Figure 4 As shown, the panoramic engineering data generation device includes: The first processing module 401 is used to fuse the multi-view engineering image set to obtain panoramic index data; The second processing module 402 is used to extract and process the panoramic index data based on preset state requirement conditions to obtain an index subset and the image block data corresponding to the index subset. The third processing module 403 is used to combine the index subset and image block data to obtain lightweight panoramic engineering data. The fourth processing module 404 is used to perform local caching and stitching processing on lightweight panoramic engineering data to obtain data to be rendered; The fifth processing module 405 is used to perform hierarchical rendering processing on the data to be rendered to obtain target panoramic engineering data, which is used for display on the target terminal device.

[0137] According to the technical solution provided in this disclosure, by fusing discrete multi-view images into a unified panoramic view and extracting structured indexes in real time, spatial location information is transformed into standardized data that can be processed by computers, thereby improving processing and response efficiency. On-demand loading of index subsets and image block data is achieved through preset state requirement conditions, reducing the amount of data processing and network transmission burden on mobile devices. Lightweight encapsulation and local caching improve the response speed for repeated accesses and enhance loading stability in weak network environments. Hierarchical rendering and progressive generation ensure cross-device compatibility, fast initial loading, and high bandwidth utilization, enhancing the smoothness, accuracy, and user experience of panoramic engineering data visualization.

[0138] In some embodiments, the fifth processing module 405 is specifically used to: perform hierarchical identification processing on the data to be rendered based on preset terminal device attribute data to obtain terminal device adaptation conditions; perform multi-resolution layer segmentation processing on the data to be rendered based on the terminal device adaptation conditions to obtain layered rendering data; and perform dynamic compositing processing on the layered rendering data to obtain target panoramic engineering data.

[0139] In some embodiments, the third processing module 403 is specifically used to: perform compression encoding on the image block data to obtain compressed image units; perform structured encapsulation on the index subset and the compressed image units to obtain lightweight data packets; and perform metadata annotation on the lightweight data packets to obtain lightweight panoramic engineering data.

[0140] In some embodiments, the second processing module 402 is specifically used to: perform spatial region division processing on the panoramic index data to obtain a set of region labels; perform matching and filtering processing on the set of region labels based on preset state requirement conditions to obtain an index subset; and perform image block extraction processing on the panoramic index data according to the index subset to obtain image block data.

[0141] In some embodiments, the first processing module 401 is specifically used to: perform alignment processing on a set of multi-view engineering images to obtain image alignment parameters; perform stitching and fusion processing on the set of multi-view engineering images based on the image alignment parameters to obtain an initial panoramic engineering image; and perform spatial index encoding processing on the initial panoramic engineering image to obtain panoramic index data.

[0142] In some embodiments, the fourth processing module 404 is specifically used to: perform cache availability judgment processing on lightweight panoramic engineering data to obtain cache status information; perform effective data filtering processing on lightweight panoramic engineering data based on cache status information to obtain a set of effective image block data; and perform spatial stitching processing on the set of effective image block data to obtain data to be rendered.

[0143] In some embodiments, the dynamic synthesis processing of layered rendering data to obtain target panoramic engineering data specifically involves performing incremental decoding processing on the layered rendering data to obtain an incremental image layer; and performing inter-frame interpolation smoothing processing on the incremental image layer to obtain the target panoramic engineering data.

[0144] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0145] Figure 5 This is a schematic diagram of the electronic device 5 provided in an embodiment of this disclosure. Figure 5 As shown, the electronic device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in the various method embodiments described above. Alternatively, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in the various device embodiments described above.

[0146] Electronic device 5 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 5 may include, but is not limited to, processor 501 and memory 502. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown, or different components.

[0147] The processor 501 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0148] The memory 502 can be an internal storage unit of the electronic device 5, such as a hard disk or RAM of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 5. The memory 502 can also include both internal and external storage units of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0151] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A method for generating panoramic engineering data, characterized in that, include: A fusion process is performed on a collection of multi-view engineering images to obtain panoramic index data; Based on preset state requirement conditions, the panoramic index data is extracted and processed to obtain an index subset and the image block data corresponding to the index subset; The index subset and the image patch data are combined and processed to obtain lightweight panoramic engineering data; The lightweight panoramic engineering data is locally cached and stitched together to obtain the data to be rendered; The data to be rendered is subjected to hierarchical rendering processing to obtain target panoramic engineering data, which can be displayed by the target terminal device.

2. The panoramic engineering data generation method according to claim 1, characterized in that, The step of performing hierarchical rendering processing on the data to be rendered to obtain target panoramic engineering data includes: The data to be rendered is subjected to hierarchical identification processing based on preset terminal device attribute data to obtain terminal device adaptation conditions. Based on the terminal device adaptation conditions, the data to be rendered is processed by multi-resolution layer segmentation to obtain layered rendering data. The layered rendering data is dynamically synthesized to obtain the target panoramic engineering data.

3. The panoramic engineering data generation method according to claim 1, characterized in that, The step of combining the index subset and the image patch data to obtain lightweight panoramic engineering data includes: The image block data is compressed and encoded to obtain compressed image units; The index subset and the compressed image unit are structured and encapsulated to obtain a lightweight data packet; Metadata annotation processing is performed on the lightweight data package to obtain the lightweight panoramic engineering data.

4. The panoramic engineering data generation method according to claim 1, characterized in that, The process of extracting and processing the panoramic index data based on preset state requirement conditions to obtain an index subset and image patch data corresponding to the index subset includes: The panoramic index data is divided into spatial regions to obtain a set of region labels; Based on the preset state requirement conditions, the set of regional labels is matched and filtered to obtain the index subset; The panoramic index data is processed by extracting image patches based on the index subset to obtain the image patch data.

5. The panoramic engineering data generation method according to claim 1, characterized in that, The process of fusing multi-view engineering image sets to obtain panoramic index data includes: The multi-view engineering image set is aligned to obtain image alignment parameters; Based on the image alignment parameters, the multi-view engineering image set is stitched and fused to obtain an initial panoramic engineering image. The initial panoramic engineering image is subjected to spatial index encoding processing to obtain the panoramic index data.

6. The panoramic engineering data generation method according to claim 1, characterized in that, The process of locally caching and stitching the lightweight panoramic engineering data to obtain the data to be rendered includes: The lightweight panoramic engineering data is processed to determine cache availability, and cache status information is obtained. Based on the cached state information, the lightweight panoramic engineering data is effectively filtered to obtain a set of effective image block data. Spatial stitching is performed on the effective image block data set to obtain the data to be rendered.

7. The panoramic engineering data generation method according to claim 2, characterized in that, The dynamic compositing process of the layered rendering data to obtain the target panoramic engineering data includes: The layered rendering data is incrementally decoded to obtain an incremental image layer; The incremental image layer is subjected to inter-frame interpolation smoothing to obtain the target panoramic engineering data.

8. A panoramic engineering data generation device, characterized in that, include: The first processing module is used to fuse multi-view engineering image sets to obtain panoramic index data; The second processing module is used to extract and process the panoramic index data based on preset state requirement conditions to obtain an index subset and image block data corresponding to the index subset. The third processing module is used to combine the index subset and the image block data to obtain lightweight panoramic engineering data. The fourth processing module is used to perform local caching and stitching processing on the lightweight panoramic engineering data to obtain the data to be rendered; The fifth processing module is used to perform hierarchical rendering processing on the data to be rendered to obtain target panoramic engineering data, wherein the target panoramic engineering data is used for display on the target terminal device.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images