Online three-dimensional exhibition hall updating method and device, computer equipment and storage medium

Through AI video reconstruction technology combined with high-definition video data, the online three-dimensional exhibition hall model is automatically updated, which solves the problems of low update frequency and utilization, provides an immersive experience and reduces costs, and achieves efficient exhibition hall updates and monitoring data utilization.

CN118941716APending Publication Date: 2024-11-12ANCHUANG QIYUAN (HANGZHOU) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411046966.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing online three-dimensional exhibition hall model has low update frequency, high cost and low utilization rate, and has failed to fully utilize the potential value of high-definition video surveillance and 360-degree video surveillance systems.

Method used

The AI ​​video reconstruction model is used to combine high-definition video data to build and update the three-dimensional model of exhibits, obtain video data through the camera, extract and identify exhibits, generate sparse three-dimensional point cloud models, optimize and texture, and finally update the online exhibition hall model.

Benefits of technology

It realizes automatic update of online three-dimensional exhibition hall model, improves monitoring data utilization, provides users with an immersive experience, reduces update costs and time, and improves the timeliness and interactivity of exhibition halls.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941716A_ABST
    Figure CN118941716A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses an online three-dimensional exhibition hall updating method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring video data shot by a camera in an exhibition hall in each exhibition period or an idle period after the end of an exhibition theme; constructing an exhibit model by adopting an AI video reconstruction model in combination with the video data to obtain a reconstructed three-dimensional exhibit model; updating an online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; and sending the updated online three-dimensional exhibition hall model to a terminal for display. By implementing the method provided by the embodiment of the invention, the online three-dimensional exhibition hall model can be automatically updated through the video data, so that the utilization rate of the monitoring data is improved, and immersive experience is provided for the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an online exhibition hall processing method, and more specifically to an online three-dimensional exhibition hall updating method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of information technology, online three-dimensional exhibition halls are combined with traditional physical exhibition halls to become an important platform for displaying cultural relics, artworks, etc.

[0003] However, most online 3D exhibition halls are currently static and have low updating frequency. They are usually modeled manually based on a certain exhibition activity, and it is difficult to regularly update the changes in exhibits, which limits the diversity of the audience experience. Updating exhibits requires re-modeling manually, which is costly and slow. On the other hand, high-definition video surveillance and 360-degree video monitoring systems are widely used in exhibition halls, mainly for security and management, but their utilization rate is low and their potential value has not been fully utilized.

[0004] Therefore, it is necessary to design a new method to automatically update the online three-dimensional exhibition hall model through video data, which not only improves the utilization rate of monitoring data but also provides users with an immersive experience. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide an online three-dimensional exhibition hall updating method, device, computer equipment and storage medium.

[0006] To achieve the above purpose, the present invention adopts the following technical solution: an online three-dimensional exhibition hall updating method, comprising:

[0007] During the idle period after each exhibition period or exhibition theme, video data captured by cameras inside the exhibition hall are obtained;

[0008] Using an AI video reconstruction model in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model;

[0009] Updating the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model;

[0010] Send the updated online 3D exhibition hall model to the terminal for display.

[0011] A further technical solution is: before obtaining the video data shot by the camera inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends, it also includes:

[0012] When the three-dimensional scene of the exhibition hall is released for the first time, build a basic three-dimensional model of the exhibition hall, and restore the basic layout and decoration of the exhibition hall to obtain an online three-dimensional exhibition hall model.

[0013] A further technical solution is: the video data includes videos automatically shot by a camera and multi-angle videos shot manually.

[0014] A further technical solution is: the AI ​​video reconstruction model is combined with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model, including:

[0015] Extracting and identifying exhibits from the video data, performing positioning and segmentation detection to obtain segmented areas;

[0016] Match and extract the multi-view feature points of the exhibits according to the segmented areas to generate an initial sparse 3D point cloud model;

[0017] Reconstructing a dense 3D point cloud or mesh model from the initial sparse 3D point cloud model and the video data;

[0018] Optimize the dense 3D point cloud or mesh model to obtain the processed 3D exhibit model;

[0019] Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

[0020] A further technical solution is: reconstructing a dense three-dimensional point cloud or mesh model from the initial sparse three-dimensional point cloud model and the video data comprises:

[0021] A dense 3D point cloud or mesh model is reconstructed from the initial sparse 3D point cloud model and the video data using a multi-view stereo algorithm.

[0022] A further technical solution is: the optimization process of the dense three-dimensional point cloud or mesh model to obtain a processed three-dimensional exhibit model includes:

[0023] The dense 3D point cloud or mesh model is smoothed, denoised, and hole repaired to obtain a processed 3D exhibit model.

[0024] A further technical solution is: extracting texture data from the video data and mapping the texture data to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model, including:

[0025] Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model by using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

[0026] The present invention also provides an online three-dimensional exhibition hall updating device, comprising:

[0027] A video data acquisition unit, used to acquire video data captured by cameras inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends;

[0028] An exhibit model reconstruction unit, used to construct an exhibit model by using an AI video reconstruction model in combination with the video data to obtain a reconstructed three-dimensional exhibit model;

[0029] An updating unit, used for updating the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model;

[0030] The sending unit is used to send the updated online three-dimensional exhibition hall model to the terminal for display.

[0031] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0032] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0033] The beneficial effect of the present invention compared with the prior art is as follows: the present invention utilizes the video data captured by the cameras in the exhibition hall, combines it with AI video reconstruction technology to build a three-dimensional model of the exhibits, and then updates the online exhibition hall model for display, thereby automatically updating the online three-dimensional exhibition hall model through video data, which not only improves the utilization rate of monitoring data, but also provides users with an immersive experience.

[0034] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0036] Figure 1 A schematic diagram of an application scenario of the online three-dimensional exhibition hall updating method provided by an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of a flow chart of an online three-dimensional exhibition hall updating method provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a sub-process of an online three-dimensional exhibition hall updating method provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a flow chart of an online three-dimensional exhibition hall updating method provided by another embodiment of the present invention;

[0040] Figure 5 A schematic block diagram of an online three-dimensional exhibition hall updating device provided by an embodiment of the present invention;

[0041] Figure 6 A schematic block diagram of an exhibit model reconstruction unit of an online three-dimensional exhibition hall updating device provided in an embodiment of the present invention;

[0042] Figure 7 A schematic block diagram of an online three-dimensional exhibition hall updating device provided by another embodiment of the present invention; and

[0043] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0046] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0047] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0048] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the online three-dimensional exhibition hall updating method provided in an embodiment of the present invention. Figure 2A schematic flow chart of an online three-dimensional exhibition hall update method provided in an embodiment of the present invention. The online three-dimensional exhibition hall update method is applied to a server. The server interacts with a camera, a terminal or a display screen, and uses the exhibition hall's high-definition video surveillance data and the full-angle exhibit video data taken manually offline, combined with an AI video visual analysis algorithm, to dynamically generate a three-dimensional model and texture of the exhibits. It can not only automatically generate a three-dimensional model of the exhibits, but also dynamically update the model of the online exhibition hall according to the theme and time of the exhibition hall. In this way, users can deeply experience the different exhibits and artworks of previous exhibition halls according to their personal preferences and needs. Through the automatically generated three-dimensional exhibit model, the online exhibition hall can be quickly updated to retain and present the wonderful style of previous exhibition halls. This method improves the timeliness of the online exhibition hall update, while effectively utilizing the monitoring data to provide an immersive experience. It not only helps to promote cultural dissemination and popularization of knowledge, but also permanently preserves the real-time status of the previous exhibits in the exhibition hall, providing users with a new exhibit browsing and learning experience.

[0049] Figure 2 FIG. 1 is a flow chart of an online three-dimensional exhibition hall updating method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S140.

[0050] S110, during the idle period after each exhibition period or exhibition theme ends, obtaining video data shot by cameras inside the exhibition hall.

[0051] In this embodiment, the video data includes videos automatically shot by a camera and multi-angle videos shot manually.

[0052] Specifically, after each exhibition period or exhibition theme, the exhibition hall will use surveillance data collection to prepare for updating the exhibit models of each booth in the online exhibition hall. This work includes using multiple high-definition cameras inside the exhibition hall to collect all-round high-definition video recordings of each booth. However, considering that the exhibition hall’s cameras may not be able to fully capture the details of all exhibits, manual supplementary shooting is also very important. This means that manual supplementary shooting will be performed on the parts of the exhibits that cannot be captured by the camera from multiple angles and without blind spots, in order to improve the accuracy and restoration of AI video remodeling.

[0053] This approach ensures the authenticity of online exhibits and cultural relics. The combination of the exhibition hall's high-definition cameras and manual reshoots not only provides comprehensive video data support, but also ensures the quality and authenticity of the exhibit models. This systematic approach not only improves the efficiency and quality of online exhibition hall updates, but also brings users a more immersive and realistic cultural relic browsing experience.

[0054] S120 , constructing an exhibit model by using an AI video reconstruction model in combination with the video data to obtain a reconstructed three-dimensional exhibit model.

[0055] In this embodiment, the reconstructed three-dimensional exhibit model refers to a model constructed using the AI ​​video reconstruction model using video data.

[0056] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S125.

[0057] S121, extracting and identifying exhibits from the video data, performing positioning and segmentation detection to obtain segmented areas.

[0058] In this embodiment, the segmented area refers to the result obtained after dividing the area according to the locations of different exhibits.

[0059] By analyzing video data, computer vision technology is used to extract and identify exhibits. Then positioning and segmentation detection is performed, that is, the position of the exhibits in the video is determined and the exhibit area is segmented; AI technology can automatically identify and locate exhibits from the video, avoiding the complexity and time-consuming manual operation. The accuracy and consistency of the algorithm ensure the accuracy of the exhibit location and segmentation, providing a good foundation for subsequent steps.

[0060] Specifically, computer vision techniques such as feature extraction algorithms (such as SIFT, SURF, ORB, etc.) are used to identify exhibits in images; deep learning models such as convolutional neural networks (CNN) can be used for object detection and recognition, such as using models such as YOLO and Faster R-CNN.

[0061] Once the exhibits are identified, the next step is to locate the exhibits in the image and perform segmentation detection. Semantic segmentation models (such as FCN and U-Net) can be used to achieve accurate segmentation of the exhibits and separate them from the background. Edge detection algorithms (such as Canny edge detection) can be combined in the segmentation detection process to improve the accuracy and precision of the segmentation.

[0062] Based on the segmentation results, the area of ​​the exhibit is extracted; the exhibit area can be further processed or analyzed, such as measuring the size, calculating the area, identifying the attributes of the exhibit, etc.; the segmented exhibit image or data is output for subsequent exhibit management, identification or display.

[0063] When implementing the above steps, commonly used tools and libraries include OpenCV, TensorFlow, PyTorch, etc., which provide rich image processing and deep learning functions and are suitable for application development of exhibit recognition and segmentation.

[0064] S122, performing multi-view feature point matching and extraction of the exhibit according to the segmented areas to generate an initial sparse three-dimensional point cloud model.

[0065] In this embodiment, the segmented exhibit area is used to match and extract feature points from multiple angles of the exhibit. These feature points are used to generate an initial sparse 3D point cloud model, which represents the 3D spatial shape of the exhibit in the form of points. The comprehensive use of data from multiple angles can improve the accuracy and integrity of 3D reconstruction. Generating a sparse 3D point cloud model is the basis for subsequent reconstruction steps and provides a starting point for further refinement of the model.

[0066] Specifically, the segmented area image or data of the exhibit obtained in the above steps is used as input; ensuring that the segmented area is as accurate as possible so that the subsequent feature point matching and three-dimensional reconstruction steps can be carried out effectively.

[0067] Next, feature points are extracted from each segmented area of ​​the exhibit. Common feature point detection algorithms include SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc. Each feature point has its position in the image and a descriptor, which is used to describe the characteristics of the image area around the point, such as color, gradient, etc.

[0068] Furthermore, in images from different perspectives, the descriptors of the feature points are used for matching. This is usually achieved by measuring the distance or similarity between the feature points; the RANSAC (Random Sample Consensus) algorithm can be used for robust feature point matching to exclude incorrect matching pairs, thereby improving the accuracy and robustness of the matching.

[0069] Finally, based on the matched feature point pairs, the camera's geometric information (such as intrinsic parameters, extrinsic parameters) and disparity information are used to calculate the initial position of the 3D point; these calculated 3D points are combined into a sparse 3D point cloud model; each point has its position information in the world coordinate system or the camera coordinate system.

[0070] In addition, optimization and reconstruction can also be performed. For example, the generated sparse point cloud can be optimized and reconstructed to improve its accuracy and completeness; the optimization method includes bundle adjustment-based technology to adjust camera parameters and 3D point positions to reduce reprojection errors; more complex 3D reconstruction algorithms, such as structured light and stereo matching, can be used to further improve the quality and density of the 3D point cloud model.

[0071] The final output is an initial sparse 3D point cloud model, which contains the approximate shape and location information of the exhibits. This model can be used for subsequent 3D reconstruction, virtual display, real-time tracking and other applications.

[0072] When implementing these steps, computer vision and image processing libraries (such as OpenCV) and 3D reconstruction and point cloud processing libraries (such as PointCloud Library) are usually used to help implement and optimize the algorithms at each stage.

[0073] S123: Reconstruct a dense three-dimensional point cloud or mesh model from the initial sparse three-dimensional point cloud model and the video data.

[0074] In this embodiment, a multi-view stereo algorithm is used to reconstruct a dense 3D point cloud or mesh model from the initial sparse 3D point cloud model and the video data.

[0075] Using multi-view stereo algorithms, a dense 3D point cloud or mesh model is further reconstructed from the initial sparse 3D point cloud model and video data. This process can fill and refine the initial model to make it more accurate and detailed. Through algorithmic processing, a denser and more accurate 3D model is generated from the initial point cloud model. Using multi-view information, it ensures that all angles and details of the model are fully expressed, improving the visual effect and realism.

[0076] Specifically, first, the feature points in the initial sparse 3D point cloud are tracked using consecutive frames in the video sequence. This can be achieved by an optical flow algorithm (such as the Lucas-Kanade optical flow algorithm) or a tracking method based on feature point descriptors (such as the KLT algorithm). Ensure that the tracked feature points remain stable between different frames so that the subsequent reconstruction step can utilize these continuous feature point information.

[0077] Next, the depth information in the video sequence is used or a stereo matching algorithm (such as Semi-Global Matching, SGM) is used to estimate the depth or disparity of each feature point. Depth estimation can be achieved by calculating the disparity of the feature points in different perspectives and then combining camera parameters (such as baseline distance) to calculate the precise three-dimensional position of each point.

[0078] Furthermore, the depth information estimated from the video sequence is applied to the feature points of the sparse 3D point cloud to generate dense 3D point clouds; these point clouds will contain more details and precise spatial information because they not only rely on the feature points in the initial point cloud, but also take into account the motion and depth information between consecutive frames.

[0079] Then, using the generated dense 3D point cloud data, a closed mesh model can be generated through surface reconstruction algorithms (such as Poisson reconstruction algorithm, Marching Cubes algorithm); these algorithms are able to convert discrete point cloud data into a smooth surface representation, thereby forming a complete 3D model.

[0080] Finally, the resulting mesh model is optimized and post-processed to improve the quality and accuracy of the model. This may include removing unwanted noise or artifacts, smoothing surfaces, or repairing missing areas.

[0081] The final output is a dense 3D point cloud or mesh model that can be used in various applications such as virtual reality, augmented reality, numerical simulation, etc.

[0082] When implementing these steps, it is usually necessary to use computer vision libraries (such as OpenCV), 3D reconstruction libraries (such as PCL), deep learning frameworks (such as PyTorch or TensorFlow), and specialized 3D reconstruction software to help implement and optimize algorithms at each stage. The entire process requires a deep understanding and application of image processing, computational geometry, and optimization techniques.

[0083] S124. Optimize the dense three-dimensional point cloud or mesh model to obtain a processed three-dimensional exhibit model.

[0084] In this embodiment, the dense three-dimensional point cloud or mesh model is smoothed, denoised, and hole repaired to obtain a processed three-dimensional exhibit model.

[0085] Specifically, the generated dense 3D point cloud or mesh model is optimized, including smoothing, denoising and hole repair operations, to obtain a more complete and high-quality 3D exhibit model; through optimization, the noise and irregularities in the model are eliminated, making it more realistic and suitable for display. This improves the user's online exhibition experience and allows them to experience a more realistic and detailed exhibit display effect.

[0086] Specifically, the steps include:

[0087] Smoothing:

[0088] Gaussian Smoothing: Applicable to point clouds or mesh models, it smoothes the local surface by applying a Gaussian filter to remove noise and roughness. The intensity of the smoothing can be controlled by adjusting the size and parameters of the filter.

[0089] Averaging method: The neighborhood of each point in the point cloud is averaged to reduce mutations and sharp edges and make the surface more uniform.

[0090] Denoising:

[0091] Statistical filtering: For example, using statistical methods (such as statistical local geometry analysis, PCA analysis) to identify and remove outliers or abnormal values ​​to eliminate noise in the model.

[0092] Distance-based denoising: remove points that are far away from their neighbors, as they may be the result of noise or false matches.

[0093] Cavity repair treatment:

[0094] Mesh filling: For mesh models, filling algorithms (such as Poisson reconstruction) can be used to fill holes and missing parts in the model to generate closed surfaces.

[0095] Voxelization and Reconstruction: Convert point clouds or mesh models to voxel representations, identify and fill holes in voxelized data.

[0096] Local surface repair:

[0097] Local reconstruction: When necessary, for damaged or partially missing areas, local data or local topological information can be used for local reconstruction to repair the local geometric structure of the surface.

[0098] Optimization and post-processing:

[0099] Normal Smoothing: Ensures normals are continuous and consistent across the model to improve lighting and rendering.

[0100] Detail preservation: When performing smoothing and denoising, try to preserve the important details and features of the model to avoid loss of details caused by over-processing.

[0101] Final Verification: Perform visual and geometric verification on the processed model to ensure it meets the design requirements and presentation standards.

[0102] S125 , extracting texture data from the video data, and mapping the texture data to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

[0103] In this embodiment, texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

[0104] Specifically, the last step is to extract texture information from the video data and use visual analysis and texture mapping technology to map the texture data onto the surface of the processed 3D exhibit model to complete the reconstructed 3D exhibit model. Adding real texture data can enhance the visual authenticity and viewing effect of the model. Mapping the texture onto the model surface makes the online display of the exhibit more complete and attractive.

[0105] Specifically, first, appropriate frame images are extracted from the video. These frames should cover all angles and surfaces of the exhibits; this step can be completed by video processing software (such as FFmpeg) or specific video frame extraction tools; texture extraction is performed on each selected frame image. This usually includes extracting the color information (RGB values) and surface details (such as texture, pattern, etc.) of the image.

[0106] Next, create a UV map for the 3D model. UV mapping is the process of unfolding the surface of a 3D model into a 2D plane for subsequent texture mapping.

[0107] Map the texture data extracted from the video onto the UV map of the 3D model. This can be done with 3D modeling software (such as Blender, Maya) or dedicated texture mapping tools; ensure the accuracy and adaptability of the texture mapping so that the texture captured in the video naturally covers the surface of the 3D model.

[0108] Secondly, after mapping the texture, perform visual analysis and optimization to ensure that the appearance and quality of the texture on the 3D model meet expectations; the texture mapping effect can be optimized by adjusting UV mapping, texture resolution, and color correction.

[0109] After completing your texture mapping, perform a final visual check and verification to ensure that the texture appears natural under different viewing angles and lighting conditions.

[0110] Finally, the 3D model with texture mapping is exported to the required format for subsequent application and display.

[0111] S130: updating the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model.

[0112] In this embodiment, the reconstructed three-dimensional model is compared with the online three-dimensional exhibition hall model through AI visual analysis. The purpose of this step is to confirm whether the current booth displays new exhibits. If it is confirmed that the current booth displays new exhibits, the old exhibit model will be marked as belonging to the theme tag of the previous exhibition, and it will be replaced with the new exhibit model currently identified and constructed, that is, the reconstructed three-dimensional exhibit model, and marked with the theme tag of this exhibition. On the contrary, if it is confirmed that the current booth displays old exhibits, the old exhibit model will also be marked as belonging to the theme tag of the previous exhibition, and will also be marked with the theme tag of the current exhibition, and it will be regarded as an exhibit shared by two or more exhibition themes. Through this modeling method of AI analysis of video data, it is possible to realize the model update of online exhibitions and offline exhibition hall exhibits under different exhibition themes, and realize the rapid synchronization update of offline and online exhibition content.

[0113] S140: Send the updated online three-dimensional exhibition hall model to the terminal for display.

[0114] The updated online 3D exhibition hall allows users to switch between different exhibition themes according to their personal preferences. Users can dynamically load the selected current exhibition theme to experience exhibits and artifacts from previous exhibitions. Through the 3D experience of the online exhibition hall, exhibits can be rendered and displayed in real time, and users can freely switch perspectives, zoom in, zoom out, roam, and browse the exhibits in the exhibition hall to gain an immersive experience. This interactive method significantly enhances the review value and interactivity of the exhibits.

[0115] The method of this embodiment uses AI video 3D model reconstruction technology, and the exhibition hall can automatically and quickly complete the online 3D exhibition hall model of different exhibition themes, making full use of rich surveillance video data and artificial video data. This method does not require excessive manual intervention, nor does it require manual modeling of venue exhibits, which greatly improves the efficiency and accuracy of updating the exhibition hall model while reducing costs. In addition, it also speeds up the synchronization of offline and online exhibition content, significantly improving the user experience and the publicity value of the current exhibition theme.

[0116] In summary, through the multi-angle high-definition surveillance cameras installed in the exhibition hall and the multi-angle and all-round video data collected manually, combined with intelligent analysis and extraction technology, it is possible to identify and obtain information such as the appearance material and shape of exhibits and cultural relics. These data are used to generate a detailed 3D model of the exhibits in the booth area, and dynamically update the online 3D exhibition hall model according to the theme or time of the exhibition, so that the model can reflect changes in the actual scene.

[0117] By using the exhibition hall's high-definition video surveillance data and offline manually shot video data without blind spots, combined with AI video visual analysis algorithms, it is possible to automatically generate three-dimensional models of exhibits. These automatically generated models can dynamically display and update the online exhibition hall's models according to the exhibition hall's different themes and time dimensions, thereby preserving the style of previous exhibits and allowing users to view all cultural relics and artworks of different themes according to their preferences.

[0118] This automatic update technology based on video data not only improves the utilization efficiency of monitoring data, but also provides users with an immersive exhibition experience, permanently and efficiently preserves the wonderful features of exhibits in previous exhibition halls, and promotes cultural dissemination and knowledge popularization.

[0119] The above-mentioned online 3D exhibition hall updating method utilizes the video data captured by the cameras in the exhibition hall and combines it with AI video reconstruction technology to build a 3D model of the exhibits, and then updates the online exhibition hall model for display, thereby automatically updating the online 3D exhibition hall model through video data, which not only improves the utilization rate of monitoring data, but also provides users with an immersive experience.

[0120] Figure 4FIG. 1 is a flow chart of an online three-dimensional exhibition hall updating method provided by another embodiment of the present invention. Figure 4 As shown, the online 3D exhibition hall updating method of this embodiment includes steps S210-S250. Steps S220-S250 are similar to steps S110-S140 in the above embodiment, and will not be repeated here. The following is a detailed description of step S210 added in this embodiment.

[0121] S210. When the three-dimensional scene of the exhibition hall is released for the first time, a basic three-dimensional model of the exhibition hall is constructed, and the basic layout and decoration of the exhibition hall are restored to obtain an online three-dimensional exhibition hall model.

[0122] In this embodiment, when the three-dimensional scene of the exhibition hall is not released for the first time, the existing online three-dimensional exhibition hall model can be directly called during the update process. However, when the online exhibition hall is released for the first time, it is necessary to build the basic three-dimensional model of the exhibition hall through modeling, restore the basic layout and basic decoration of the exhibition hall, and provide a real three-dimensional venue environment for the arrangement of exhibits and cultural relics in the exhibition hall.

[0123] Specifically, high-definition video data from every corner of the exhibition hall is collected to ensure comprehensive coverage, including exhibition areas, corridors and other important spaces; the video data is manually and automatically analyzed to capture the basic layout, decoration details and spatial characteristics of the exhibition hall.

[0124] Use professional 3D modeling software or services to convert site information and visual features extracted from video data into a basic 3D model. This includes accurately restoring the basic structure of the exhibition hall, room layout, wall decoration, floor paving and other basic elements to ensure the authenticity and accuracy of the model.

[0125] Based on the basic model, design and arrange the placement of exhibits and artifacts. This may involve the size and space requirements of display cabinets, booths, and artworks, ensuring that the position of each exhibit in the model is consistent with the layout in the actual exhibition, so as to provide visitors with a realistic viewing experience.

[0126] The reconstructed model needs to take into account the lighting, material texture, color and other environmental factors of the exhibition hall in order to reproduce the visual effects of the actual exhibition as realistically as possible. Such restoration can enhance the user's sense of immersion and participation, making them feel as if they are really inside the exhibition hall.

[0127] Figure 5 is a schematic block diagram of an online three-dimensional exhibition hall updating device 300 provided by an embodiment of the present invention. Figure 5 As shown, corresponding to the above online 3D exhibition hall updating method, the present invention also provides an online 3D exhibition hall updating device 300. The online 3D exhibition hall updating device includes a unit for executing the above online 3D exhibition hall updating method, and the device can be configured in a server. Figure 5 The online three-dimensional exhibition hall updating device 300 includes a video data acquiring unit 302 , an exhibit model reconstruction unit 303 , an updating unit 304 and a sending unit 305 .

[0128] The video data acquisition unit 302 is used to acquire the video data captured by the camera inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends; the exhibit model reconstruction unit 303 is used to construct the exhibit model by using the AI ​​video reconstruction model in combination with the video data to obtain a reconstructed three-dimensional exhibit model; the updating unit 304 is used to update the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; the sending unit 305 is used to send the updated online three-dimensional exhibition hall model to the terminal for display.

[0129] In one embodiment, if Figure 6 As shown, the exhibit model reconstruction unit 303 includes a segmentation subunit 3031 , a matching subunit 3032 , a model reconstruction subunit 3033 , an optimization processing subunit 3034 and a mapping subunit 3035 .

[0130] The segmentation subunit 3031 is used to extract and identify exhibits from the video data, perform positioning and segmentation detection to obtain segmented areas; the matching subunit 3032 is used to match and extract multi-view feature points of the exhibits according to the segmented areas to generate an initial sparse three-dimensional point cloud model; the model reconstruction subunit 3033 is used to reconstruct a dense three-dimensional point cloud or mesh model from the initial sparse three-dimensional point cloud model and the video data; the optimization processing subunit 3034 is used to optimize the dense three-dimensional point cloud or mesh model to obtain a processed three-dimensional exhibit model; the mapping subunit 3035 is used to extract texture data from the video data and map the texture data to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

[0131] In one embodiment, the model reconstruction subunit 3033 is used to reconstruct a dense 3D point cloud or mesh model from the initial sparse 3D point cloud model and the video data using a multi-view stereo algorithm.

[0132] In one embodiment, the optimization processing subunit 3034 is used to perform smoothing, denoising, and hole repairing processing on the dense three-dimensional point cloud or mesh model to obtain a processed three-dimensional exhibit model.

[0133] In one embodiment, the mapping subunit 3035 is used to extract texture data from the video data, and map the texture data to the surface of the processed three-dimensional exhibit model by using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

[0134] Figure 7 FIG. 3 is a schematic block diagram of an online three-dimensional exhibition hall updating device 300 provided in another embodiment of the present invention. Figure 7 As shown, the online three-dimensional exhibition hall updating device 300 of this embodiment is based on the above embodiment and adds a building unit 301.

[0135] The building unit 301 is used to build a basic three-dimensional model of the exhibition hall when the three-dimensional scene of the exhibition hall is released for the first time, and to restore the basic layout and decoration of the exhibition hall to obtain an online three-dimensional exhibition hall model.

[0136] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the above-mentioned online three-dimensional exhibition hall updating device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0137] The above-mentioned online three-dimensional exhibition hall updating device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 8 Runs on the computer device shown.

[0138] See also Figure 8 , Figure 8 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0139] See also Figure 8 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0140] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute an online three-dimensional exhibition hall updating method.

[0141] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0142] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an online three-dimensional exhibition hall updating method.

[0143] The network interface 505 is used to communicate with other devices over the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0144] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0145] During the idle period after each exhibition period or exhibition theme, video data captured by cameras inside the exhibition hall are obtained; an AI video reconstruction model is used in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model; the online three-dimensional exhibition hall model is updated according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; the updated online three-dimensional exhibition hall model is sent to the terminal for display.

[0146] The video data includes videos automatically shot by a camera and multi-angle videos shot manually.

[0147] In one embodiment, before implementing the step of acquiring the video data captured by the camera inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends, the processor 502 further implements the following steps:

[0148] When the three-dimensional scene of the exhibition hall is released for the first time, build a basic three-dimensional model of the exhibition hall, and restore the basic layout and decoration of the exhibition hall to obtain an online three-dimensional exhibition hall model.

[0149] In one embodiment, when the processor 502 implements the step of using the AI ​​video reconstruction model in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model, the processor 502 specifically implements the following steps:

[0150] The exhibits are extracted and identified from the video data, and positioning and segmentation detection are performed to obtain segmented areas; multi-view feature point matching and extraction of the exhibits are performed according to the segmented areas to generate an initial sparse three-dimensional point cloud model; a dense three-dimensional point cloud or mesh model is reconstructed from the initial sparse three-dimensional point cloud model and the video data; the dense three-dimensional point cloud or mesh model is optimized to obtain a processed three-dimensional exhibit model; texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

[0151] In one embodiment, when the processor 502 implements the step of reconstructing a dense 3D point cloud or mesh model from the initial sparse 3D point cloud model and the video data, the processor 502 specifically implements the following steps:

[0152] A dense 3D point cloud or mesh model is reconstructed from the initial sparse 3D point cloud model and the video data using a multi-view stereo algorithm.

[0153] In one embodiment, when the processor 502 implements the step of optimizing the dense three-dimensional point cloud or mesh model to obtain the processed three-dimensional exhibit model, the processor 502 specifically implements the following steps:

[0154] The dense 3D point cloud or mesh model is smoothed, denoised, and hole repaired to obtain a processed 3D exhibit model.

[0155] In one embodiment, when the processor 502 implements the step of extracting texture data from the video data and mapping the texture data to the surface of the processed three-dimensional exhibit model to obtain the reconstructed three-dimensional exhibit model, the processor 502 specifically implements the following steps:

[0156] Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model by using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

[0157] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0158] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0159] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0160] During the idle period after each exhibition period or exhibition theme, video data captured by cameras inside the exhibition hall are obtained; an AI video reconstruction model is used in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model; the online three-dimensional exhibition hall model is updated according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; the updated online three-dimensional exhibition hall model is sent to the terminal for display.

[0161] The video data includes videos automatically shot by a camera and multi-angle videos shot manually.

[0162] In one embodiment, before the processor executes the computer program to implement the step of acquiring video data captured by a camera inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends, the processor further implements the following steps:

[0163] When the three-dimensional scene of the exhibition hall is released for the first time, build a basic three-dimensional model of the exhibition hall, and restore the basic layout and decoration of the exhibition hall to obtain an online three-dimensional exhibition hall model.

[0164] In one embodiment, when the processor executes the computer program to implement the step of using the AI ​​video reconstruction model in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model, the processor specifically implements the following steps:

[0165] The exhibits are extracted and identified from the video data, and positioning and segmentation detection are performed to obtain segmented areas; multi-view feature point matching and extraction of the exhibits are performed according to the segmented areas to generate an initial sparse three-dimensional point cloud model; a dense three-dimensional point cloud or mesh model is reconstructed from the initial sparse three-dimensional point cloud model and the video data; the dense three-dimensional point cloud or mesh model is optimized to obtain a processed three-dimensional exhibit model; texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

[0166] In one embodiment, when the processor executes the computer program to implement the step of reconstructing a dense three-dimensional point cloud or mesh model from the initial sparse three-dimensional point cloud model and the video data, the processor specifically implements the following steps:

[0167] A dense 3D point cloud or mesh model is reconstructed from the initial sparse 3D point cloud model and the video data using a multi-view stereo algorithm.

[0168] In one embodiment, when the processor executes the computer program to implement the step of optimizing the dense three-dimensional point cloud or mesh model to obtain the processed three-dimensional exhibit model, the processor specifically implements the following steps:

[0169] The dense 3D point cloud or mesh model is smoothed, denoised, and hole repaired to obtain a processed 3D exhibit model.

[0170] In one embodiment, when the processor executes the computer program to implement the step of extracting texture data from the video data and mapping the texture data to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model, the processor specifically implements the following steps:

[0171] Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model by using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

[0172] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0174] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0175] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0177] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. The online three-dimensional exhibition hall updating method is characterized by: include: During the idle period after each exhibition period or exhibition theme, video data captured by cameras inside the exhibition hall are obtained; Using an AI video reconstruction model in combination with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model; Updating the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; Send the updated online 3D exhibition hall model to the terminal for display.

2. The online three-dimensional exhibition hall updating method according to claim 1, characterized in that: Before obtaining the video data captured by the camera inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends, the method further includes: When the three-dimensional scene of the exhibition hall is released for the first time, build a basic three-dimensional model of the exhibition hall, and restore the basic layout and decoration of the exhibition hall to obtain an online three-dimensional exhibition hall model.

3. The online three-dimensional exhibition hall updating method according to claim 1 or 2, characterized in that: The video data includes videos automatically shot by a camera and multi-angle videos shot manually.

4. The online three-dimensional exhibition hall updating method according to claim 3 is characterized in that: The AI ​​video reconstruction model is combined with the video data to construct an exhibit model to obtain a reconstructed three-dimensional exhibit model, including: Extracting and identifying exhibits from the video data, performing positioning and segmentation detection to obtain segmented areas; Match and extract the multi-view feature points of the exhibits according to the segmented areas to generate an initial sparse 3D point cloud model; Reconstructing a dense 3D point cloud or mesh model from the initial sparse 3D point cloud model and the video data; Optimize the dense 3D point cloud or mesh model to obtain the processed 3D exhibit model; Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model.

5. The online three-dimensional exhibition hall updating method according to claim 4, characterized in that: The reconstructing a dense three-dimensional point cloud or mesh model from the initial sparse three-dimensional point cloud model and the video data comprises: A dense 3D point cloud or mesh model is reconstructed from the initial sparse 3D point cloud model and the video data using a multi-view stereo algorithm.

6. The online three-dimensional exhibition hall updating method according to claim 4, characterized in that: The step of optimizing the dense three-dimensional point cloud or mesh model to obtain a processed three-dimensional exhibit model includes: The dense 3D point cloud or mesh model is smoothed, denoised, and hole repaired to obtain a processed 3D exhibit model.

7. The online three-dimensional exhibition hall updating method according to claim 4, characterized in that: The step of extracting texture data from the video data and mapping the texture data to the surface of the processed three-dimensional exhibit model to obtain a reconstructed three-dimensional exhibit model includes: Texture data is extracted from the video data, and the texture data is mapped to the surface of the processed three-dimensional exhibit model by using visual analysis and texture mapping technology to obtain a reconstructed three-dimensional exhibit model.

8. The online three-dimensional exhibition hall updating device is characterized by: include: A video data acquisition unit, used to acquire video data captured by cameras inside the exhibition hall during the idle period after each exhibition period or exhibition theme ends; An exhibit model reconstruction unit, used to construct an exhibit model by using an AI video reconstruction model in combination with the video data to obtain a reconstructed three-dimensional exhibit model; An updating unit, used for updating the online three-dimensional exhibition hall model according to the reconstructed three-dimensional exhibit model to obtain an updated online three-dimensional exhibition hall model; The sending unit is used to send the updated online three-dimensional exhibition hall model to the terminal for display.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.