Immersive exhibition hall dynamic scene generation method and system based on real-time rendering

Abstract

The invention relates to the technical field of electronic digital data processing, and particularly discloses an immersive exhibition hall dynamic scene generation method and system based on real-time rendering, and the method comprises the steps: firstly collecting point cloud, depth, texture and other multi-source perception data in an exhibition hall space, constructing a unified reference coordinate system, dividing the exhibition hall into initial rendering space units, and carrying out the initial rendering of the initial rendering space units; the data source coverage degree is evaluated to ensure that the spatial information is complete; then multi-source data fusion is carried out, fusion process parameters are monitored through cross comparison, and the result accuracy is judged, so that errors caused by single data source deviation are reduced, and the modeling precision is improved. When the result is inaccurate, the fusion process is adjusted by the method, and reliable output is ensured; when the result is accurate, classification processing is carried out, a three-dimensional modeling data set is generated, real-time rendering is driven, and a dynamic scene of the immersive exhibition hall is achieved. According to the method, the fineness and immersion of virtual reconstruction are improved, data fluctuation can be dynamically adapted, and the robustness and real-time performance of the system are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic digital data processing, in particular to an immersive exhibition dynamic scene generation method and system based on real-time rendering. BACKGROUND

[0002] The current immersive exhibition dynamic scene generation technology based on real-time rendering mainly relies on the collection and processing of multi-source data, including the three-dimensional model of exhibits, environmental material information, audience behavior data, and real-time input of sensors, etc. Through the real-time calculation of the scene by the graphics rendering engine, a visual picture is generated. The system usually adopts hierarchical scene management, divides the scene into different spatial units, and dynamically updates the geometric data, texture data and lighting information of each unit to support real-time interaction and immersive experience. At the same time, some technologies will combine user positioning, motion capture or sensor data to dynamically adjust the lighting, perspective, special effects and physical behavior in the virtual environment, realizing environmental perception and interactive response. The commonly used optimization methods in the scene rendering process include view frustum culling, level of detail management, and multi-thread or GPU accelerated calculation to ensure the real-time and smoothness of rendering.

[0003] For example, the Chinese invention patent application with publication number CN120045067A discloses a content dynamic generation and immersive tour interaction system, which includes: a user behavior capture module for capturing the user's position, line of sight direction and interactive action in the virtual environment in real time and generating user behavior data; a distributed optimal control module connected to the user behavior capture module for generating a global control strategy for content generation based on the user behavior data; and a content generation module connected to the distributed optimal control module. The present application adopts the combination technology of distributed optimal control module and user behavior modeling, achieving the technical effect of high matching between dynamic content generation and user behavior.

[0004] For example, the Chinese invention patent application with publication number CN115079820A discloses a virtual scene dynamic switching method, system, storage medium and device, which includes: establishing an initial virtual three-dimensional scene model and a to-be-switched virtual three-dimensional scene model; binding a destruction function to the initial virtual three-dimensional scene model and a generation function to the to-be-switched virtual three-dimensional scene model; binding a trigger event on a preset target; when scene switching is needed, triggering the trigger event to trigger the destruction function and the generation function respectively to switch the initial virtual three-dimensional scene model and the to-be-switched virtual three-dimensional scene model.

[0005] The above technology at least has the following technical problems: in the existing exhibition space three-dimensional modeling process, there is a problem of insufficient data collection coverage. Specifically, the laser radar cannot collect the back area point cloud under the exhibition stand column shielding condition; the depth camera causes the wall relief detail to be missing due to light reflection interference; the panoramic camera cannot obtain the corner texture information due to the visual angle blind area; the above conditions will cause a lack of collaborative basis for multi-source data fusion, thereby causing a fusion fault, for example, the lack of point cloud anchoring in the back area causes the depth data matching to deviate, and the lack of depth support in the relief area causes the texture and structure to be misaligned; such fusion defects directly lead to insufficient three-dimensional modeling precision, thereby affecting the reality and immersion of the virtual scene. SUMMARY

[0006] In order to solve the technical problem of insufficient three-dimensional modeling precision in the prior art, the embodiment of the present application provides an immersive exhibition dynamic scene generation method and system based on real-time rendering. The technical scheme is as follows: On the one hand, an immersive exhibition dynamic scene generation method based on real-time rendering is provided, which comprises: Step one, collecting multi-source perception data in the exhibition space and arranging it into a multi-source data perception set, processing the multi-source data perception set to build a unified reference coordinate system, dividing the exhibition space into initial rendering space units, and evaluating the data source coverage of each initial rendering space unit; the multi-source data perception set includes point cloud data, depth data and texture data; Step two, after completing the data source coverage evaluation of each initial rendering space unit, multi-source data fusion is performed on each initial rendering space unit to generate the multi-source data fusion result corresponding to each initial rendering space unit, and cross comparison is performed on the multi-source data fusion process to monitor and analyze the cross comparison process parameters, thereby determining whether the multi-source data fusion result of each initial rendering space unit is accurate; Step three, if the multi-source data fusion result of each initial rendering space unit is not accurate, the multi-source data fusion process of each initial rendering space unit is adjusted, if the multi-source data fusion result of each initial rendering space unit is accurate, the multi-source data fusion result of each initial rendering space unit is classified and processed, and a three-dimensional modeling data set is generated, thereby driving the real-time rendering process and realizing the dynamic scene generation of the immersive exhibition.

[0007] On the other hand, an immersive exhibition dynamic scene generation system based on real-time rendering is provided, which comprises: The perception collection module is used for collecting multi-source perception data in the exhibition space and arranging the multi-source perception data into a multi-source data perception set, processing the multi-source data perception set to construct a unified reference coordinate system, dividing the exhibition space into initial rendering space units, and evaluating the data source coverage of each initial rendering space unit, wherein the multi-source data perception set includes point cloud data, depth data and texture data. The data fusion module is used for performing multi-source data fusion on each initial rendering space unit after the data source coverage evaluation of each initial rendering space unit is completed, thereby generating a multi-source data fusion result corresponding to each initial rendering space unit, cross-comparing the multi-source data fusion process, monitoring and analyzing cross-comparison process parameters, and determining whether the multi-source data fusion result of each initial rendering space unit is accurate. The real-time rendering module is used for adjusting the multi-source data fusion process of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is not accurate, classifying the multi-source data fusion result of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is accurate, generating a three-dimensional modeling data set, and driving a real-time rendering process to realize dynamic scene generation of an immersive exhibition hall.

[0008] The technical scheme provided by the embodiment of the application has at least the following beneficial effects: (1) The embodiment of the application provides a dynamic scene generation method and system for an immersive exhibition hall based on real-time rendering, realizes dynamic scene generation of an immersive exhibition hall through full-link design of collection-fusion-adjustment-modeling-rendering, effectively fuses multi-source perception data, improves modeling accuracy and rendering realism, introduces coverage evaluation and cross-comparison mechanisms to enhance the robustness and real-time performance of data processing, and automatically optimizes the fusion process according to data quality through a dynamic adjustment mechanism to avoid distortion of the reconstruction result due to local abnormalities. Ultimately, the system can output a dynamic scene with high fidelity and strong interaction, providing technical support for virtual exhibitions, digital twins and immersive experiences.

[0009] (2) By collecting multi-source perception data such as point cloud, depth and texture in the exhibition space, constructing a unified reference coordinate system, dividing the space into initial rendering space units, and evaluating the data source coverage of each initial rendering space unit, the structural management of the exhibition environment can be realized. Based on data source coverage evaluation, insufficiently covered areas can be identified in time to avoid local modeling loss caused by uneven data collection. This process ensures the spatial integrity and balance of three-dimensional modeling, laying a foundation for subsequent data fusion and rendering. At the same time, based on coverage threshold judgment, interpolation or compensation processing can be triggered to ensure that low-coverage areas can still obtain reasonable reconstruction results.

[0010] (3) Through multi-source data fusion of each spatial unit, the geometric accuracy of point cloud, the contour details of depth data and the surface information of texture data can be complementarily integrated, so as to improve the fineness of scene modeling. In the fusion process, a cross comparison mechanism is introduced, which can compare the fusion consistency of different data sources, monitor and analyze the cross comparison parameters in real time, and judge the accuracy of the fusion result. This mechanism effectively reduces the error transmission of a single data source and improves the reliability and robustness of data fusion.

[0011] (4) When the fusion result is detected to be inaccurate, the system can adjust the fusion process, thereby reducing errors and improving fusion accuracy. The adjusted fusion result will still enter the modeling process, ensuring that even if there are data fluctuations or partial defects, a usable three-dimensional modeling data set can still be generated, avoiding the occurrence of faults or vacancies in the rendering process. When the fusion result is accurate, a high-precision three-dimensional modeling data set is generated through classification processing, providing stable input for subsequent real-time rendering. This mechanism ensures that regardless of the quality of the data, a renderable data set can be obtained, thereby enhancing the robustness of the system and the continuity of scene generation. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0013] Figure 1 is a flow chart of the immersive exhibition dynamic scene generation method based on real-time rendering provided by the embodiments of the present application; Figure 2 is a structural schematic diagram of the immersive exhibition dynamic scene generation based on real-time rendering provided by the embodiments of the present application; Figure 3 is a multi-source data coverage and fusion initial processing flowchart of the initial rendering spatial unit provided by the embodiments of the present application; Figure 4 is a data fusion accuracy index judgment and optimization flowchart provided by the embodiments of the present application; Figure 5 is a multi-source data fusion result classification processing flowchart provided by the embodiments of the present application. DETAILED DESCRIPTION

[0014] The technical solutions in the present application will be described below in conjunction with the drawings.

[0015] In the embodiments of the present application, the words such as "example", "for example" and the like are used to represent an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0016] In the embodiments of the present application, sometimes the subscript such as W1 may be written in the form of non-subscript such as W1, and the meanings expressed thereby are consistent when the difference is not emphasized.

[0017] In order to make the technical problems, technical schemes and advantages to be solved by the present application more clear, the following will be described in detail in combination with the drawings and specific embodiments.

[0018] The embodiments of the present application provide an immersive exhibition dynamic scene generation method based on real-time rendering, as shown in Figure 1 The flow chart of the immersive exhibition dynamic scene generation method based on real-time rendering, the processing flow of the method can include the following steps: Step one, collect multi-source perception data in the exhibition space, and organize into a multi-source data perception set, process the multi-source data perception set to construct a unified reference coordinate system, at the same time, divide the exhibition space into each initial rendering space unit, and evaluate the data source coverage degree of each initial rendering space unit, the multi-source data perception set includes point cloud data, depth data and texture data; Step two, after completing the data source coverage degree evaluation of each initial rendering space unit, multi-source data fusion is performed on each initial rendering space unit, thereby generating the multi-source data fusion result corresponding to each initial rendering space unit, at the same time, cross comparison is performed on the multi-source data fusion process, and the cross comparison process parameters are monitored and analyzed, thereby judging whether the multi-source data fusion result of each initial rendering space unit has accuracy; Step three, if the multi-source data fusion result of each initial rendering space unit does not have accuracy, the multi-source data fusion process of each initial rendering space unit is adjusted, if the multi-source data fusion result of each initial rendering space unit has accuracy, the multi-source data fusion result of each initial rendering space unit is classified and processed, and a three-dimensional modeling data set is generated, and then a real-time rendering process is driven, thereby realizing the dynamic scene generation of the immersive exhibition.

[0019] In the scheme, to realize real-time rendering of the immersive exhibition dynamic scene generation, a mapping rule system is configured based on historical scheduling data, real-time operation monitoring results and multi-scene stress test results. The mapping rule system integrates the corresponding relationship between multi-dimensional scheduling parameters (such as structural complexity) and multi-level adjustment parameters, and through data modeling and normalization processing, a segmented and scalable parameter mapping table is established. The mapping rule system not only supports static configuration, but also can be dynamically updated according to the accurate index and state deviation evaluation results of the computing power allocation, so as to ensure the real-time adaptability and accuracy of the allocation strategy.

[0020] Specifically, the data source coverage of each initial rendering space unit is evaluated. The specific evaluation process is as follows: obtaining the structural complexity of each initial rendering space unit, matching the data source coverage threshold of each initial rendering space unit and the initial grid density of each initial rendering space unit based on the structural complexity of each initial rendering space unit. The data source coverage threshold represents the minimum value of the data source coverage of each initial rendering space unit within a specified range. The structural complexity can reflect the comprehensive complexity of the geometric, curvature and texture characteristics of the initial rendering space unit, and is specifically obtained by weighted average of point cloud density, curvature average and texture entropy. The specific weight is set by the professional technical personnel in the art according to the characteristics of the exhibition scene. The point cloud density represents the number of point clouds in a unit volume, reflecting the richness of the geometric information of the space unit. It is obtained by counting the number of point cloud points in the initial rendering space unit and dividing the volume of the unit. The curvature average represents the average level of the surface curvature in the initial rendering space unit, reflecting the structural fluctuation and complexity. It is obtained by calculating the local curvature of each point in the unit and then taking the average of the local curvatures of all points. The texture entropy represents the distribution complexity of the texture information in the unit, reflecting the diversity and detail richness of the surface texture. It is obtained by counting the color distribution probability in the unit and calculating the entropy value.

[0021] The data source coverage threshold of each initial rendering space unit and the initial grid density of each initial rendering space unit are matched based on the structure complexity of each initial rendering space unit. The specific matching process is as follows: a management database is preset with a structure complexity-data source coverage threshold mapping table and a structure complexity-initial grid density mapping table. The structure complexity of each initial rendering space unit is input into the management database. The management database can match the data source coverage threshold and the initial grid density of each initial rendering space unit based on the preset mapping table. The higher the structure complexity of the initial rendering space unit, the greater the data source coverage threshold, that is, more point cloud, depth, and texture information is needed to ensure modeling accuracy. The lower the structure complexity, the lower the data source coverage threshold, thereby avoiding resource waste. The higher the structure complexity of each initial rendering space unit, the smaller the grid used, so as to capture rich geometric and texture details. The lower the structure complexity of each initial rendering space unit, the larger the grid used, thereby improving the calculation efficiency.

[0022] The data source coverage of each initial rendering space unit is obtained and compared with the matched data source coverage threshold of each initial rendering space unit. The specific method for obtaining the data source coverage of each initial rendering space unit is as follows: for each space unit, the effective information amount of point cloud, depth, and texture data is counted, respectively. The point cloud data coverage is calculated by the ratio of the number of point clouds in the unit to the reference point number. The depth data coverage is calculated by the proportion of effective depth pixels to total pixels. The texture data coverage is calculated by the proportion of effective texture pixels to total pixels. Then, the above-mentioned coverage indicators are normalized and weighted averaged according to the set weight to obtain the comprehensive data source coverage of the space unit. The set weight is determined by a person skilled in the art.

[0023] If the data source coverage of a certain initial rendering space unit is not lower than the corresponding data source coverage threshold, it is judged that the data source coverage of the initial rendering space unit is sufficient, and the initial rendering space unit is marked as a high-coverage space unit for multi-source data fusion. If the data source coverage of a certain initial rendering space unit is lower than the corresponding data source coverage threshold, it is judged that the data source coverage of the initial rendering space unit is insufficient, and the initial rendering space unit is marked as a low-coverage space unit. The low-coverage space unit is subjected to interpolation processing. The interpolation processing means that the data of adjacent high-coverage space units is used for interpolation processing.

[0024] The data source coverage of the low-coverage space unit after interpolation is obtained. If it is still lower than the corresponding data source coverage threshold, secondary interpolation processing is performed. If it is not lower than the corresponding data source coverage threshold, multi-source data fusion is performed. The secondary interpolation processing is also performed by using the data of adjacent high-coverage space units for interpolation processing. After the secondary interpolation processing is completed, multi-source data fusion is also performed.

[0025] AsFigure 3 The multi-source data coverage degree of the initial rendering space unit and the fusion initial processing flow provided by the embodiment of the present application are shown in the schematic diagram, the structural complexity of the initial rendering space unit is obtained first, and then the data source coverage degree threshold and the initial grid density of each unit are matched. Then, it is judged whether the data source coverage degree of each initial rendering space unit is not lower than the corresponding data source coverage degree threshold. If yes, it is marked as a high-coverage space unit and multi-source data fusion is performed; if no, it is marked as a low-coverage space unit and interpolation processing is performed. Then, it is judged whether the data source coverage degree after interpolation is lower than the corresponding data source coverage degree threshold. If yes, secondary interpolation processing is performed before multi-source data fusion; if no, multi-source data fusion is directly performed, and finally the data fusion accuracy index is obtained.

[0026] Specifically, the cross comparison process parameters are monitored and analyzed, and the specific analysis process is as follows: The cross comparison process parameters include a point cloud overlap error proportion coefficient, an RGB deviation proportion coefficient, and a depth deviation proportion coefficient. By presetting the effect coefficient of each proportion coefficient in the management database, the weight contribution value of each proportion coefficient to the data fusion accuracy index is quantified, and finally a weighted average fusion algorithm is used to synthesize the data fusion accuracy index. The data fusion accuracy index represents the accuracy of the multi-source data fusion result in each initial rendering space unit.

[0027] The point cloud overlap error proportion coefficient represents the ratio of the point cloud overlap error to the defined point cloud overlap error, the RGB deviation proportion coefficient represents the ratio of the RGB deviation to the defined RGB deviation, and the depth deviation proportion coefficient represents the ratio of the depth deviation to the defined depth deviation.

[0028] The above-mentioned point cloud overlap error represents the deviation degree of position alignment between point clouds of different sources in the same space unit, reflects the geometric consistency of point cloud fusion, and is obtained by calculating the average value of the Euclidean distance of corresponding points in the overlapping area. The above-mentioned RGB deviation represents the color value difference of texture sources in the same space unit, reflects the consistency of texture fusion, and is obtained by comparing the data of different sources in the same area, first calculating the difference value of each point, and then averaging the difference values. The above-mentioned depth deviation represents the difference in the depth direction between different depth data sources in the same space unit, reflects the accuracy of depth information fusion, and is obtained by calculating the depth difference of overlapping points of different depth data sources in the same space unit and taking the average value.

[0029] The above-mentioned defined point cloud overlap error represents the maximum value of the point cloud overlap error within a specified range; the above-mentioned defined RGB deviation represents the maximum value of the RGB deviation within a specified range; and the above-mentioned defined depth deviation represents the maximum value of the depth deviation within a specified range.

[0030] When the point cloud overlap error is small, it indicates that the spatial registration accuracy of different data sources is high, so that the corresponding relationship of RGB and depth data in space is more reliable, and the RGB deviation and depth deviation are also reduced; on the contrary, when the point cloud overlap error is large, the matching of color and depth information will be affected, leading to the increase of RGB deviation and depth deviation. In addition, since RGB images and depth maps are usually collected at the same time, factors such as camera calibration or collection offset may affect both, so that the RGB deviation and the depth deviation present a certain indirect correlation. Therefore, the three types of deviations restrict each other and jointly affect the final data fusion accuracy index when evaluating the fusion accuracy of multi-source data.

[0031] The point cloud overlap error proportion coefficient reflects the relative size of the point cloud position deviation, and the larger the value is, the worse the point cloud alignment is, thereby leading to the decrease of the data fusion accuracy index; the RGB deviation proportion coefficient represents the relative degree of color difference of the texture source, and the larger the value is, the more inconsistent the color is, thereby leading to the decrease of the data fusion accuracy index, and the depth deviation proportion coefficient represents the relative degree of depth data difference, and the larger the value is, the more inconsistent the depth information is, thereby leading to the decrease of the data fusion accuracy index.

[0032] The specific evaluation method of the data fusion accuracy index is: ; Among them, DFAI is the data fusion accuracy index, POCE is the point cloud overlap error proportion coefficient, RGBD is the RGB deviation proportion coefficient, DDR is the depth deviation proportion coefficient, gp is the effect coefficient corresponding to the point cloud overlap error proportion coefficient preset in the management database, gr is the effect coefficient corresponding to the RGB deviation proportion coefficient preset in the management database, and gd is the effect coefficient corresponding to the depth deviation proportion coefficient preset in the management database.

[0033] The effect coefficient corresponding to the point cloud overlap error proportion coefficient indicates that when the point cloud overlap error proportion coefficient changes by a unit amplitude, the data fusion accuracy index will change by a corresponding amplitude; the effect coefficient corresponding to the RGB deviation proportion coefficient indicates that when the RGB deviation proportion coefficient changes by a unit amplitude, the data fusion accuracy index will change by a corresponding amplitude; and the effect coefficient corresponding to the depth deviation proportion coefficient indicates that when the depth deviation proportion coefficient changes by a unit amplitude, the data fusion accuracy index will change by a corresponding amplitude.

[0034] The management database stores the mapping relationship between the point cloud overlap error proportion coefficient and its corresponding effect coefficient, the mapping relationship between the RGB deviation proportion coefficient and its corresponding effect coefficient, and the mapping relationship between the depth deviation proportion coefficient and its corresponding effect coefficient. In this embodiment, the mapping relationship is a mapping table. For example, the point cloud overlap error proportion coefficient, the RGB deviation proportion coefficient, and the depth deviation proportion coefficient are input into the management database. The management data can match the effect coefficient corresponding to the point cloud overlap error proportion coefficient, the effect coefficient corresponding to the RGB deviation proportion coefficient, and the effect coefficient corresponding to the depth deviation proportion coefficient based on the preset mapping relationship. The numerical range of each type of effect coefficient is strictly controlled between 0 and 1.

[0035] Further, it is judged whether the multi-source data fusion result of each initial rendering space unit is accurate. The specific judgment process is: comparing the data fusion accuracy index with a data fusion accuracy threshold value. The data fusion accuracy threshold value represents the minimum value of the data fusion accuracy index in the specified range in the management database. When the data fusion accuracy index is not lower than the data fusion accuracy threshold value, it is judged that the multi-source data fusion result of each initial rendering space unit is accurate. When the data fusion accuracy index is lower than the data fusion accuracy threshold value, it is judged that the multi-source data fusion result of each initial rendering space unit is inaccurate. The determination of the above data fusion accuracy threshold value comprehensively considers historical fusion effect, spatial structure complexity, multi-source data quality, and experimental tuning factors. The data fusion accuracy threshold value is dynamically adjusted by statistically analyzing the distribution of the data fusion accuracy index of previous similar space units, combining the structure complexity of the unit and the collection accuracy, resolution, and noise level of each data source, and fine-tuning the data fusion accuracy threshold value in combination with the actual rendering and fusion test results to ensure that space units of different complexities and data qualities can reasonably determine the accuracy of the fusion result, thereby obtaining a dynamic data fusion accuracy threshold value that is both scientific and reliable.

[0036] Specifically, the multi-source data fusion process of each initial rendering space unit is adjusted. The specific adjustment process is: based on the data fusion accuracy index and the data fusion accuracy threshold value, a data fusion accuracy deviation value is obtained, and a threshold multiplier reduction coefficient is matched based on the data fusion accuracy deviation value, so as to reduce the threshold multiplier of the outlier filtering algorithm. The above data fusion accuracy deviation value refers to the data fusion accuracy threshold value minus the data fusion accuracy index. The above threshold multiplier reduction coefficient is matched based on the data fusion accuracy deviation value. The specific matching process is: , wherein k is a threshold multiplier reduction coefficient, a is a proportional coefficient for controlling the sensitivity of the overall reduction range, which is set according to the system requirement for the strictness of filtering, D is a data fusion accuracy deviation value, β is an index, and the value range is 0.1≤β≤2, which can be adjusted by a person skilled in the art according to the system requirement for the strictness of filtering and response sensitivity. When β is greater than 1, the adjustment range of the large deviation unit can be enhanced, and the filtering is more strict. The function is suitable for spatial units with uneven deviation distribution, can flexibly handle partial abnormal or low-precision areas, and ensures the overall improvement of data fusion accuracy. The data fusion accuracy deviation value obtained is substituted into the function to obtain the threshold multiplier reduction coefficient. The threshold multiplier reduction coefficient obtained is multiplied by the original threshold multiplier, and the result obtained is the threshold multiplier of the outlier filtering algorithm that needs to be adjusted. By adjusting the outlier filtering threshold multiplier based on the data fusion accuracy deviation, the filtering process can be dynamically optimized according to the actual fusion accuracy of each initial rendering spatial unit, the threshold multiplier of the outlier filtering algorithm is reduced, the filtering algorithm is more strictly removed from abnormal data, and thus the interference of outliers on the multi-source data fusion result is effectively reduced.

[0037] The adjusted data fusion accuracy index is obtained and marked as a data fusion accuracy comprehensive index, so as to determine whether to give a warning to the multi-source data fusion process.

[0038] Specifically, whether to give a warning to the multi-source data fusion process is determined, and the specific determination process is as follows: If the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold value, it is determined that no warning is given to the multi-source data fusion process. If the data fusion accuracy comprehensive index rises but is lower than the data fusion accuracy threshold value, it is determined that no warning is given to the multi-source data fusion process, and a corresponding point search radius increase coefficient is matched based on the data fusion accuracy deviation value, so as to increase the search range between the target point and the candidate corresponding point in the point cloud data, expand the matching space, make the target point that cannot find the correct corresponding point due to position deviation or local abnormality obtain a matching opportunity, thereby improving the point cloud registration accuracy, reducing the mismatch and missing points, enhancing the geometric consistency of multi-source data fusion, and providing more complete and reliable spatial data for subsequent three-dimensional modeling and real-time rendering. The threshold multiplier adjustment strategy and the search radius adjustment strategy are iterated continuously until the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold value, or the iteration number reaches the preset maximum iteration number. When the iteration number reaches the preset maximum iteration number and the data fusion accuracy comprehensive index is lower than the data fusion accuracy threshold value, a warning is given to the multi-source data fusion process. If the data fusion accuracy comprehensive index decreases, it is determined that a warning is given to the multi-source data fusion process.

[0039] The increase of the data fusion accuracy comprehensive index indicates that the data fusion accuracy index value of the initial rendering space unit is close to or tends to the data fusion accuracy threshold after adjusting the threshold multiplier of the outlier filtering algorithm; the decrease of the data fusion accuracy comprehensive index indicates that the data fusion accuracy index value of the initial rendering space unit is far away from the data fusion accuracy threshold after adjusting the threshold multiplier of the outlier filtering algorithm; the corresponding point search radius increasing coefficient is matched based on the data fusion accuracy deviation value, and the specific matching process is as follows: the corresponding point search radius increasing coefficient corresponding to each data fusion accuracy deviation value interval in the management database is preset, the obtained data fusion accuracy deviation value is input into the management database, the management database can match the corresponding data fusion accuracy deviation value interval, and then the corresponding point search radius increasing coefficient corresponding to the interval is the required increasing coefficient; the obtained corresponding point search radius increasing coefficient is multiplied by the original corresponding point search radius, so that the adjusted corresponding point search radius is obtained; the corresponding point search radius increasing coefficient is greater than 1, indicating that the corresponding point search radius needs to be increased by a multiple value. As Figure 4 The data fusion accuracy index judgment and optimization process provided by the embodiment of the application is shown in the schematic diagram, first, the data fusion accuracy index is obtained, whether the data fusion accuracy is not lower than the data fusion accuracy threshold is judged, if yes, it is determined that the multi-source data fusion result has accuracy; if no, the threshold multiplier of the outlier filtering algorithm is reduced based on the data fusion accuracy deviation value, and the data fusion accuracy comprehensive index is obtained. If the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold, the multi-source data fusion process is not warned; if the data fusion accuracy comprehensive index increases but is lower than the data fusion accuracy threshold, the corresponding point search radius is increased based on the deviation value, and the threshold multiplier and the search radius adjustment strategy are iterated until the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold or the maximum iteration number is reached, when the iteration number reaches the preset maximum iteration number and the data fusion accuracy comprehensive index is lower than the data fusion accuracy threshold, the multi-source data fusion process is warned; if the data fusion accuracy comprehensive index decreases, the fusion process is warned, and finally the multi-source data fusion result is classified and processed; Further, the multi-source data fusion results of each initial rendering space unit are classified and processed, and the specific analysis process is as follows: if the multi-source data fusion result corresponding to a certain initial rendering space unit is an abnormal fusion effect, the initial rendering space unit is marked as an abnormal effect space unit, and fusion abnormality correction optimization is performed on the abnormal effect space unit; if the multi-source data fusion result corresponding to a certain initial rendering space unit is a normal fusion effect, the initial rendering space unit is marked as a normal effect space unit, and a three-dimensional modeling data set is generated. The above multi-source data fusion result is automatically determined based on a trained machine learning classifier. The machine learning classifier is an automatic determination model constructed based on training data, which is used to classify and determine the multi-source data fusion result of the initial rendering space unit. In the system, the machine learning classifier can automatically output the category of the fusion effect of the space unit according to the feature vector (such as point cloud density, texture continuity, depth consistency, color deviation, etc.) of the fusion result by learning the mapping relationship between the historical fusion data and the fusion effect label. For example, the fusion effect is normal or abnormal.

[0040] Specifically, the fusion abnormality correction optimization is performed on the abnormal effect space unit, and the specific optimization process is as follows: based on the data fusion accuracy deviation value, an initial rendering space unit division parameter reduction coefficient is matched to reduce the size of the initial rendering space unit, so as to improve the fusion accuracy of the point cloud, depth and texture data in a local range. Smaller unit means that the data difference in each unit is easier to be captured and corrected, thereby reducing local fusion error, improving multi-source data consistency, and providing more detailed and reliable spatial data for three-dimensional modeling and real-time rendering. At the same time, based on the data fusion accuracy deviation value, a seam fusion width increase coefficient is matched to increase the seam fusion width during texture mapping, so that the seam area of adjacent space units during texture mapping is expanded, which can smooth the texture transition between units and reduce the visual discontinuity and discontinuity at the seam. When the deviation is large, the small misalignment or color difference between units can be effectively buffered, the overall rendering effect is more natural, the visual continuity and viewing comfort of the immersive exhibition dynamic scene are enhanced, and unnecessary processing burden on high-precision units is avoided.

[0041] The above initial rendering space unit division parameter reduction coefficient is matched based on the data fusion accuracy deviation value, and the specific process is as follows: s is the initial rendering space unit partitioning parameter reduction coefficient, σ is the scaling factor, which represents the degree of influence of the deviation on the unit partitioning reduction range. It is usually set to 0.01 to 0.1 to avoid excessive unit refinement. DT is the data fusion accuracy deviation value. To ensure system stability and prevent the unit partitioning parameter from becoming negative, a lower limit sm is usually set for s. When the calculation result is lower than the lower limit, s is taken as the lower limit value, thereby ensuring that the unit partitioning parameter is always non-negative, ensuring that the space unit size is reasonable and can be used for subsequent multi-source data fusion and real-time rendering. The value of sm is determined by professional technicians. Substituting the obtained data fusion accuracy deviation value into this formula will give the initial rendering space unit partitioning parameter reduction coefficient. Multiplying the obtained initial rendering space unit partitioning parameter reduction coefficient by the original initial rendering space unit partitioning parameter will give the result of the initial rendering space unit partitioning parameter that needs to be adjusted.

[0042] The above-mentioned seam fusion width amplification factor is matched based on the accurate deviation value of data fusion, specifically as follows: d is the seam fusion width amplification coefficient, δ is the smoothing adjustment factor (0 < δ < 1), which controls the overall amplification amplitude, and θ is the sensitivity adjustment factor. The larger the data fusion accuracy deviation value, the more sensitive it is to differences. The specific value is determined by professionals. Substituting the data fusion accuracy deviation value into the formula, the seam fusion width amplification coefficient can be obtained. Multiplying the obtained seam fusion width amplification coefficient by the original seam fusion width gives the seam fusion width that needs to be adjusted. This function can achieve a sensitive response to small differences and a gradual saturation adjustment effect to large differences, thereby avoiding the abrupt problems caused by linear growth. It can ensure smooth and natural fusion under minor deviations and prevent excessive seam expansion under large deviations. At the same time, the function parameters can be flexibly adjusted to adapt to different scenarios.

[0043] Based on the optimized initial rendering space unit partitioning parameters and seam blending width, a local gain compensation coefficient is further matched using the data fusion accuracy index to adjust texture color weights and mesh vertex positions. The specific matching process for the local gain compensation coefficient using the data fusion accuracy index is as follows: a pre-defined data fusion accuracy index-local gain compensation coefficient mapping table is used in the management database. The obtained data fusion accuracy index is input into the management database, and the management database matches the corresponding local gain compensation coefficient based on the pre-defined mapping rules. Adjusting the texture color weights refers to multiplying the original texture color weight of the initial rendering space unit by the local gain compensation coefficient; the result is the adjusted texture color weight. Regarding the mesh vertex positions... Calculate the fine-tuning displacement vector based on the local gain compensation coefficient. G is the local gain compensation coefficient. For ideal alignment, the adjusted mesh vertex positions are as follows: , so as to correct local geometric deviation, make the unit boundary smoothly connected with the adjacent unit, and avoid grid deformation caused by large movement.

[0044] Further, the three-dimensional modeling data set is generated, and the specific analysis process is as follows: the three-dimensional modeling data set includes but is not limited to the corrected grid density, the texture mapping information and the detail enhancement parameter. The corrected grid density refers to the result of correcting the initial grid density based on the data fusion accuracy comprehensive index, and is used to ensure spatial continuity and geometric accuracy. The initial grid density is corrected based on the data fusion accuracy comprehensive index, and the specific process is as follows: the data fusion accuracy comprehensive index is input into the management database, the management database can directly match the initial grid density correction coefficient based on the preset data fusion accuracy comprehensive index-initial grid density correction coefficient mapping rule, the initial grid density correction coefficient obtained is multiplied by the original initial grid density, so that the corrected grid density is obtained. The texture mapping information includes a joint fusion width parameter, a texture mapping weight and the like. The detail enhancement parameter includes a grid vertex fine adjustment displacement vector, a boundary smoothing coefficient and the like.

[0045] The three-dimensional modeling data set is taken as input data to drive the real-time rendering module. The real-time rendering module manages each rendering space unit based on a unified coordinate system, dynamically adjusts the grid density and the texture resolution according to the observer's view angle and distance, and realizes multi-view dynamic scene display of the immersive exhibition hall.

[0046] As Figure 5 , the multi-source data fusion result classification processing flowchart provided by the embodiment of the application is shown. When the multi-source data fusion result has accuracy, the result is classified and processed. If the fusion effect is abnormal, it is marked as an abnormal effect space unit, fusion anomaly correction optimization is performed, the initial rendering space unit size is reduced based on the data fusion accuracy deviation value, the texture mapping joint fusion width is increased, the local gain compensation coefficient is matched based on the data fusion accuracy index, the texture color weight and the grid vertex position are adjusted, and the three-dimensional modeling data set is generated. If the fusion effect is normal, it is marked as a normal effect space unit, and the three-dimensional modeling data set is directly generated.

[0047] The second aspect of the embodiment of the application provides an immersive exhibition hall dynamic scene generation system based on real-time rendering, as shown in Figure 2 The structure diagram of the immersive exhibition hall dynamic scene generation based on real-time rendering, the system includes a perception acquisition module, a data fusion module, a real-time rendering module and a management database.

[0048] The perception acquisition module is connected with the data fusion module, the data fusion module is connected with the real-time rendering module, the perception acquisition module, the data fusion module and the real-time rendering module are connected with the management database, and the management database is used for storing various parameters involved in the immersive exhibition hall dynamic scene generation system based on real-time rendering.

[0049] The perception acquisition module is used for collecting multi-source perception data in the exhibition hall space, and arranging the multi-source perception data into a multi-source data perception set, processing the multi-source data perception set to construct a unified reference coordinate system, dividing the exhibition hall space into initial rendering space units, and performing data source coverage evaluation on the initial rendering space units, wherein the multi-source data perception set includes point cloud data, depth data and texture data. The data fusion module is used for performing multi-source data fusion on the initial rendering space units after the data source coverage evaluation on the initial rendering space units is completed, thereby generating a multi-source data fusion result corresponding to each initial rendering space unit, cross-comparing the multi-source data fusion process, monitoring and analyzing cross-comparison process parameters, and determining whether the multi-source data fusion result of each initial rendering space unit is accurate. The real-time rendering module is used for adjusting the multi-source data fusion process of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is not accurate, classifying and processing the multi-source data fusion result of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is accurate, generating a three-dimensional modeling data set, and driving a real-time rendering process to realize dynamic scene generation of an immersive exhibition hall.

[0050] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, wherein A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects, but can also represent an "and / or" relationship, which can be understood according to the context before and after.

[0051] In the present application, "at least one" means one or more, and "a plurality of" means two or more. "At least one of the following" or the like means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c can represent a, b, c, a-b, a-c, b-c or a-b-c, wherein a, b and c can be single or multiple.

[0052] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating immersive exhibition dynamic scene based on real-time rendering, characterized in that, The method comprises: Step one, collecting multi-source perception data in the exhibition space, and organizing into a multi-source data perception set, processing the multi-source data perception set to construct a unified reference coordinate system, dividing the exhibition space into initial rendering space units, and evaluating the data source coverage of each initial rendering space unit, wherein the multi-source data perception set comprises point cloud data, depth data and texture data; Step two, after completing the data source coverage evaluation of each initial rendering space unit, multi-source data fusion is performed on each initial rendering space unit to generate the multi-source data fusion result corresponding to each initial rendering space unit, and cross comparison is performed on the multi-source data fusion process to monitor and analyze the cross comparison process parameters, so as to determine whether the multi-source data fusion result of each initial rendering space unit is accurate; Step three, if the multi-source data fusion result of each initial rendering space unit is not accurate, the multi-source data fusion process of each initial rendering space unit is adjusted, if the multi-source data fusion result of each initial rendering space unit is accurate, the multi-source data fusion result of each initial rendering space unit is classified and processed, and a three-dimensional modeling data set is generated to drive a real-time rendering process, thereby realizing dynamic scene generation of an immersive exhibition hall.

2. The method of claim 1, wherein, The data source coverage of each initial rendering space unit is evaluated, and the specific evaluation process is as follows: Obtain the structure complexity of each initial rendering space unit, match the data source coverage threshold of each initial rendering space unit and the initial grid density of each initial rendering space unit based on the structure complexity of each initial rendering space unit, wherein the data source coverage threshold represents the minimum value of the data source coverage of each initial rendering space unit within a specified range; Obtain the data source coverage of each initial rendering space unit and compare it with the matched data source coverage threshold of each initial rendering space unit; If the data source coverage of a certain initial rendering space unit is not lower than the corresponding data source coverage threshold, it is determined that the data source coverage of the initial rendering space unit is sufficient, and it is marked as a high-coverage space unit for multi-source data fusion; If the data source coverage of a certain initial rendering space unit is lower than the corresponding data source coverage threshold, it is determined that the data source coverage of the initial rendering space unit is insufficient, and it is marked as a low-coverage space unit, and the low-coverage space unit is subjected to interpolation processing, wherein the interpolation processing refers to interpolation processing using data from adjacent high-coverage space units; Obtain the data source coverage of the low-coverage space unit after interpolation, if it is still lower than the corresponding data source coverage threshold, perform secondary interpolation processing, if it is not lower than the corresponding data source coverage threshold, perform multi-source data fusion.

3. The method of claim 1, wherein, The cross comparison process parameters are monitored and analyzed, and the specific analysis process is as follows: The cross comparison process parameters include a point cloud overlap error proportion coefficient, an RGB deviation proportion coefficient, and a depth deviation proportion coefficient. By presetting effect coefficients of each proportion coefficient in a management database, a weight contribution value of the proportion coefficient to a data fusion accuracy index is quantified. Finally, a weighted average fusion algorithm is used to synthesize the data fusion accuracy index, which represents the accuracy of the multi-source data fusion result in each initial rendering space unit. The point cloud overlap error proportion coefficient represents a ratio of the point cloud overlap error to a defined point cloud overlap error. The RGB deviation proportion coefficient represents a ratio of the RGB deviation to a defined RGB deviation. The depth deviation proportion coefficient represents a ratio of the depth deviation to a defined depth deviation.

4. The method of claim 1, wherein, The accuracy of the multi-source data fusion result of each initial rendering space unit is determined by comparing the data fusion accuracy index with a data fusion accuracy threshold value. The data fusion accuracy threshold value represents the minimum value of the data fusion accuracy index within a specified range. When the data fusion accuracy index is not lower than the data fusion accuracy threshold value, the multi-source data fusion result of each initial rendering space unit is determined to be accurate. When the data fusion accuracy index is lower than the data fusion accuracy threshold value, the multi-source data fusion result of each initial rendering space unit is determined to be inaccurate. The multi-source data fusion process of each initial rendering space unit is adjusted based on the data fusion accuracy index and the data fusion accuracy threshold value. A data fusion accuracy deviation value is obtained based on the data fusion accuracy index and the data fusion accuracy threshold value. A threshold multiplier reduction coefficient is matched based on the data fusion accuracy deviation value, so as to reduce the threshold multiplier of the outlier filtering algorithm.

5. The method of claim 1, wherein, An adjusted data fusion accuracy index, which is marked as a data fusion accuracy comprehensive index, is obtained, so as to determine whether to issue a warning for the multi-source data fusion process. If the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold value, it is determined not to issue a warning for the multi-source data fusion process. If the data fusion accuracy comprehensive index rises but is lower than the data fusion accuracy threshold value, it is determined not to issue a warning for the multi-source data fusion process. Meanwhile, a corresponding point search radius increase coefficient is matched based on the data fusion accuracy deviation value, so as to increase the search range between the target point and the candidate corresponding point in the point cloud data. The threshold multiplier adjustment strategy and the search radius adjustment strategy are iterated until the data fusion accuracy comprehensive index is not lower than the data fusion accuracy threshold value, or the iteration number reaches a preset maximum iteration number.

6. The method of claim 5, wherein, If the data fusion accuracy comprehensive index decreases, it is determined to issue a warning for the multi-source data fusion process. The multi-source data fusion result of each initial rendering space unit is classified and processed. If the multi-source data fusion result of a certain initial rendering space unit is abnormal, the initial rendering space unit is marked as an abnormal effect space unit, and the abnormal effect space unit is subjected to fusion abnormality correction optimization. If the multi-source data fusion result of a certain initial rendering space unit is normal, the initial rendering space unit is marked as a normal effect space unit, and a three-dimensional modeling data set is generated. ​ ​ 7. The method of claim 1, wherein, ​ ​ ​ 8. The method of claim 7, wherein, The abnormal effect space unit is optimized by performing fusion anomaly correction, and the specific optimization process is as follows: Based on the data fusion accurate deviation value, the initial rendering space unit division parameter reduction coefficient is matched to reduce the size of the initial rendering space unit, and the joint fusion width increase coefficient is matched based on the data fusion accurate deviation value to increase the joint fusion width during texture mapping; Based on the optimized initial rendering space unit division parameter and the joint fusion width, a local gain compensation coefficient is matched through the data fusion accurate index to adjust the texture color weight and the grid vertex position.

9. The method of claim 1, wherein, The three-dimensional modeling data set is generated, and the specific analysis process is as follows: The three-dimensional modeling data set includes the corrected grid density, texture mapping information and detail enhancement parameter, the corrected grid density is the result of correcting the initial grid density based on the data fusion accurate comprehensive index, and is used to ensure the spatial continuity and geometric accuracy; The three-dimensional modeling data set is used as input data to drive the real-time rendering module, the real-time rendering module manages each rendering space unit based on a unified coordinate system, dynamically adjusts the grid density and texture resolution according to the observer's view angle and distance, and realizes the multi-view dynamic scene display of the immersive exhibition hall.

10. An immersive exhibition dynamic scene generation system based on real-time rendering, applying the immersive exhibition dynamic scene generation method based on real-time rendering according to any one of claims 1 to 9, characterized in that: It includes: The perception acquisition module is used to collect multi-source perception data in the exhibition space and arrange it into a multi-source data perception set, process the multi-source data perception set to build a unified reference coordinate system, divide the exhibition space into initial rendering space units, and evaluate the data source coverage of each initial rendering space unit. The multi-source data perception set includes point cloud data, depth data and texture data; The data fusion module is used to complete the data source coverage evaluation of each initial rendering space unit, and then perform multi-source data fusion on each initial rendering space unit to generate the multi-source data fusion result corresponding to each initial rendering space unit, and cross compare the multi-source data fusion process, monitor and analyze the cross comparison process parameters to determine whether the multi-source data fusion result of each initial rendering space unit is accurate; The real-time rendering module is used to adjust the multi-source data fusion process of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is not accurate, and to classify and process the multi-source data fusion result of each initial rendering space unit if the multi-source data fusion result of each initial rendering space unit is accurate, and then drive the real-time rendering process to realize the dynamic scene generation of the immersive exhibition hall.

Images