Cultural and tourism lighting display effect evaluation method and system based on computer vision

Abstract

The present invention provides a cultural and tourism lighting display effect evaluation method and system based on computer vision, which relates to the technical field of computer vision, including: collecting panoramic image sets and local close-up image sets, performing semantic segmentation, texture recognition and material recognition, constructing a three-dimensional space model and mapping the panoramic image sets and the local close-up image sets into the three-dimensional space model, generating point cloud data and extracting lighting effect characteristic parameters; interacting to obtain a lighting effect evaluation score sequence, updating and extracting multi-channel lighting characteristic parameters, performing regression prediction to obtain an overall effect evaluation score, and performing evaluation to obtain a lighting effect level; if the level threshold is not reached, solving the optimal lighting equipment parameters and generating a training lighting effect image, collecting the illuminance distribution to generate an illuminance deviation map, performing layout optimization and parameter adjustment again, and performing evaluation, and outputting the evaluation level and the current layout if the level threshold is reached.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for evaluating cultural and tourism lighting display effects based on computer vision. Background Art

[0002] In the process of cultural and tourism exhibition, lighting effect is one of the key factors to create atmosphere and attract audiences. Traditional lighting design mainly relies on the experience and subjective judgment of designers, lacks quantitative evaluation standards and optimization methods, and is difficult to adapt to the increasingly diversified and personalized exhibition needs;

[0003] There have been some studies and applications in the field of lighting effect evaluation and optimization, but most methods focus on the analysis of the physical characteristics of lighting, lack consideration of the audience's visual perception, and are difficult to fully evaluate the aesthetics and comfort of lighting. At the same time, the degree of automation is not high, the degree of manual participation is high, and the evaluation and optimization process is time-consuming, which is not conducive to rapid iteration and dynamic adjustment.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for evaluating cultural and tourism lighting display effects based on computer vision, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for evaluating cultural and tourism lighting display effects based on computer vision, comprising:

[0007] Collect a panoramic image set of the cultural and tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules;

[0008] The lighting effect characteristic parameters are added to a preset lighting effect evaluation model, and a lighting effect evaluation score sequence is obtained by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with a policy gradient algorithm, and a key score subsequence is selected in the lighting effect evaluation score sequence for fusion through a Monte Carlo tree search algorithm to obtain a dynamic lighting effect evaluation score. A multi-channel lighting effect image is simulated and multi-channel lighting characteristic parameters are extracted through a ray tracing algorithm, a multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain a multi-view lighting effect evaluation score, the dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted fused to obtain an overall effect evaluation score, and a fuzzy comprehensive evaluation is performed in combination with a preset multi-level scoring standard to obtain a lighting effect grade;

[0009] If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the Gaussian process optimization algorithm is combined to solve the optimal lighting equipment parameters and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, which is compared with the rendering to generate an illuminance deviation map, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0010] In an optional embodiment,

[0011] Collect panoramic image sets of the cultural and tourism lighting display environment and local close-up image sets corresponding to key cultural relics, perform semantic segmentation on the panoramic image sets through a multi-scale convolutional neural network, obtain key cultural relic areas, visit path areas and background areas in each panoramic image, perform texture recognition and material recognition on the local close-up image sets through a deep convolutional neural network based on transfer learning, obtain texture features and material features of key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules, including:

[0012] A panoramic image set of the cultural tourism lighting display environment is collected through a pre-set camera, and a close-up image set of the surface of key cultural relics is obtained by using a macro lens to obtain a local close-up image set corresponding to the current key cultural relics;

[0013] For the panoramic image set, a cultural tourism lighting display environment training set is constructed and the images in the training set are manually annotated, a multi-scale convolutional neural network based on an attention mechanism is trained using the manually annotated training set images, the parameters of the multi-scale convolutional neural network are adjusted, and semantic segmentation is performed on each image in the panoramic image set. In each panoramic image, the semantic label corresponding to each pixel is determined and the key cultural relics area, the tour path area, and the background area are identified;

[0014] Collect and organize image data of cultural relics texture and material and perform fine annotation, generate annotated close-up image set and expand it through data enhancement technology, select a transfer learning model, modify the top-level structure and adjust the loss function of the selected transfer learning model based on the characteristics and categories of cultural relics, use the expanded annotated close-up image set to train the adjusted transfer learning model, obtain pre-training parameters and migrate them to the convolution layer of the deep convolutional neural network, the convolution layer uses the image in the local close-up image set as input, combines the compact bilinear pooling layer to perform high-order bilinear encoding, and obtains the texture features and material features corresponding to the key cultural relics;

[0015] The panoramic image set obtained by the motion reconstruction structure algorithm is sparsely reconstructed to obtain the sparse point cloud and camera pose of the display environment. The multi-view stereo reconstruction algorithm is combined with the semantic segmentation results of the panoramic image set to perform dense matching and three-dimensional reconstruction to obtain a dense point cloud model of the display environment. Multi-perspective depth images are generated and registered in combination with deep learning. A triangular mesh surface model of the display environment is generated by the Poisson surface reconstruction algorithm. Texture mapping is performed based on the panoramic image and the local close-up image to obtain a three-dimensional space model corresponding to the cultural and tourism lighting display environment. Based on the texture features and the material features, a normal map and a displacement map are added to the three-dimensional space model. The environment is rendered by a physically based rendering technology, and the panoramic image set and the local close-up image set are mapped to the three-dimensional space model. The three-dimensional space model is uniformly sampled by the Poisson disk sampling algorithm to generate uniformly distributed point cloud data. Each sampling point is semantically annotated in combination with the semantic segmentation information. Lighting effect feature parameters corresponding to multiple dimensions are extracted according to pre-set lighting effect evaluation rules, and the lighting effect feature parameters are stored in a point cloud data structure.

[0016] In an optional embodiment,

[0017] The objective function of sparse 3D reconstruction of the acquired panoramic image set by the motion reconstruction structure algorithm is as follows:

[0018] ;

[0019] in, G represents the objective function value, R i Indicates i The rotation matrix of the camera, t i Indicates i The camera's translation vector, X j Indicates j The coordinates of the three-dimensional points, x ij Indicates j The three-dimensional point in i The projected coordinate points in the camera, v ij Is a visibility marker, used to indicate the j Is the point in i Within the camera field of view, d () is the Euclidean distance function, which is used to represent the reprojection error of the point. σ Represents the scale parameter.

[0020] In an optional embodiment,

[0021] The lighting effect characteristic parameters are added to a preset lighting effect evaluation model, and a lighting effect evaluation score sequence is obtained by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with a policy gradient algorithm, and key score subsequences are selected and fused in the lighting effect evaluation score sequence through a Monte Carlo tree search algorithm, so as to obtain a dynamic lighting effect evaluation score including:

[0022] Adding the lighting effect characteristic parameters to a preset lighting effect evaluation model, constructing a virtual cultural and tourism lighting display environment based on the cultural and tourism lighting display environment and the animation engine and rendering it through the three-dimensional space model, interacting the lighting effect evaluation model with the virtual cultural and tourism lighting display environment, generating state data, action data and reward data and adding them to the policy gradient algorithm, generating a lighting effect evaluation score sequence, and updating the parameters of the lighting effect evaluation model based on the lighting effect evaluation score sequence and the policy gradient algorithm;

[0023] A Monte Carlo tree is constructed according to the lighting effect evaluation score sequence, wherein each node of the Monte Carlo tree represents a lighting parameter configuration, and each path from a root node to a leaf node represents a parameter adjustment sequence. For each node in the Monte Carlo tree, an upper confidence bound algorithm is used to evaluate and generate a node evaluation score, a node with the highest node evaluation score is selected, child nodes are randomly generated and the node evaluation score corresponding to each child node is calculated again, and the generation and selection are repeated until a preset maximum number of iterations is reached to obtain a search tree, and the search paths in the search tree are traversed, and the node evaluation scores corresponding to the nodes on the search paths are summed as search path scores and sorted in descending order, and the search paths in the top 10% are selected as key score subsequences, and for the selected key score subsequences, the corresponding lighting effect evaluation scores are calculated, and weighted fusion is performed in combination with time span and time attenuation to obtain the dynamic lighting effect evaluation score.

[0024] In an optional embodiment,

[0025] The upper confidence bound algorithm is used to evaluate and generate the node evaluation score as shown in the following formula:

[0026] ;

[0027] in, w g Representation Node g The cumulative reward value, n g Representation Node g The number of visits, α Represents the power of the number of node visits, which is used to adjust the degree of exploration. N Indicates the total number of visits. V g Representation Node g The variance of the value of is used to represent the uncertainty of node rewards.

[0028] In an optional embodiment,

[0029] The multi-channel lighting effect image is simulated by the ray tracing algorithm and the multi-channel lighting feature parameters are extracted. The multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain the multi-view lighting effect evaluation score. The dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted fused to obtain the overall effect evaluation score. The lighting effect level is obtained by fuzzy comprehensive evaluation combined with the preset multi-level scoring criteria, including:

[0030] Select representative viewpoints in a virtual environment, render at each viewpoint through a ray tracing algorithm, generate a lighting effect image, extract multiple lighting feature channels for the lighting effect image, calculate features on each lighting feature channel through an image pyramid algorithm, obtain lighting information of different granularities, combine to obtain multi-channel lighting feature parameters, extract the multi-channel lighting feature parameters from each viewpoint and pre-acquired spatiotemporal statistical indicators, construct them into a multi-view lighting effect evaluation vector according to a pre-set splicing method, select a Gaussian process regression model, train with the lighting effect score scored manually as a training label, perform regression prediction through the trained Gaussian process regression model, and obtain the multi-view lighting effect evaluation score;

[0031] The user interaction behavior data is obtained and the user's preference for different lighting effects is determined through a collaborative filtering algorithm to generate user preferences. The weight corresponding to the dynamic lighting effect evaluation score and the weight corresponding to the multi-view lighting effect evaluation score are determined through a weight generation model in combination with the cultural background and historical information corresponding to the cultural tourism lighting display environment. The overall effect evaluation score is obtained by weighted fusion. According to a pre-set multi-level scoring standard, the overall effect evaluation score is added to the membership function corresponding to each level, the membership vector is calculated and synthesized into a comprehensive membership through a fuzzy comprehensive evaluation method, and the lighting effect level corresponding to the current lighting effect characteristic parameter is output based on the maximum membership principle and the comprehensive membership.

[0032] In an optional embodiment,

[0033] If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the optimal lighting equipment parameters are solved in combination with the Gaussian process optimization algorithm, and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, and the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached, including:

[0034] Acquire the lighting effect level obtained by judgment. If the lighting effect level does not reach a preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting device control module, and the lighting optimization problem is modeled as a black box optimization problem in combination with the Gaussian process optimization algorithm. The observation data set is acquired and the lighting device parameters and the corresponding lighting effect evaluation function are defined. It is assumed that the function value of the lighting effect evaluation function obeys a multivariate Gaussian distribution. The posterior distribution is calculated for the observation points in the observation data set and the acquisition function is introduced through Bayesian optimization, and the optimal parameters are searched to obtain the optimal lighting device parameters.

[0035] Based on the optimal lighting equipment parameters, randomly generate training lighting effect images through a preset deep generative adversarial network, add the training lighting effect images to the lighting effect evaluation model, extract topological structure and parameter information through a graph neural network, model each lamp as a node of the graph, and model the relative position and orientation relationship between the lamps as the edge of the graph, obtain a global representation vector corresponding to the current layout through multi-layer graph convolution and pooling, extract texture and semantic features for the rendered image through a convolutional neural network, generate image features, cascade the global representation vector and the image features and add them to a multi-layer fully connected network to obtain a layout evaluation score corresponding to the current layout, repeatedly calculate the layout evaluation scores corresponding to different training lighting effect images, and select the global representation vector with the highest layout evaluation score as the optimal lighting layout;

[0036] Based on the optimal lighting layout, lighting arrangement is performed for the cultural and tourism lighting display environment. After the arrangement is completed, the illuminance distribution is collected in real time and compared with the rendering generated by the three-dimensional space model to generate the illuminance deviation map. Based on the illuminance deviation map, the optimal lighting layout is again optimized and the lighting parameters are adjusted. The optimized lighting effect is again evaluated through a multi-level scoring standard. If the lighting effect level is greater than the level threshold, the current lighting effect level, layout parameters and lighting parameters obtained by evaluation are output. Otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0037] A second aspect of an embodiment of the present invention provides a cultural and tourism lighting display effect evaluation system based on computer vision, including:

[0038] The first unit is used to collect a panoramic image set of the cultural tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules;

[0039] The second unit is used to add the lighting effect characteristic parameters to a preset lighting effect evaluation model, obtain a lighting effect evaluation score sequence by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model, update the lighting effect evaluation model in combination with a policy gradient algorithm, and select key score subsequences in the lighting effect evaluation score sequence for fusion through a Monte Carlo tree search algorithm to obtain a dynamic lighting effect evaluation score, simulate a multi-channel lighting effect image and extract multi-channel lighting characteristic parameters through a ray tracing algorithm, construct a multi-view lighting effect evaluation vector and perform regression prediction to obtain a multi-view lighting effect evaluation score, perform adaptive weighted fusion on the dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score to obtain an overall effect evaluation score, and perform fuzzy comprehensive evaluation in combination with a preset multi-level scoring standard to obtain a lighting effect grade;

[0040] The third unit is used for, if the lighting effect level does not reach a preset level threshold, then, according to a pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the Gaussian process optimization algorithm is combined to solve the optimal lighting equipment parameters and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0041] According to a third aspect of the embodiments of the present invention,

[0042] An electronic device is provided, comprising:

[0043] processor;

[0044] a memory for storing processor-executable instructions;

[0045] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0046] A fourth aspect of the embodiments of the present invention is:

[0047] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0048] In the present invention, multi-scale convolutional neural networks and deep convolutional neural networks are used to accurately identify key information in panoramic images and local close-up images, such as key cultural relics areas, visiting paths, texture and material features, etc., which provides rich semantic information and visual features for subsequent three-dimensional space modeling and lighting effect evaluation. The three-dimensional space modeling technology is used to map the two-dimensional image information to the three-dimensional space, and generate point cloud data, which provides a real and three-dimensional spatial representation for the analysis and optimization of lighting effects. The lighting feature parameters are extracted in combination with preset evaluation rules, which lays the foundation for the quantitative evaluation of lighting effects. The lighting effect evaluation score sequence is obtained through interaction with the virtual environment, and Key subsequences are selected for fusion to generate dynamic evaluation scores, which can fully consider the impact of the time dimension on the lighting effect and improve the comprehensiveness of the evaluation. By real-time acquisition of illuminance distribution and comparison with the rendering, an illuminance deviation map is generated to guide the secondary optimization and parameter adjustment of the lighting layout. The closed-loop control mechanism of multiple iterative evaluations and optimizations ensures that the lighting effect can be continuously improved and ultimately reaches the expected high quality level. In summary, the present invention realizes the intelligent evaluation and dynamic optimization of the cultural and tourism lighting display environment, greatly improves the quality and adaptability of the lighting effect, brings a more comfortable and immersive visiting experience to the audience, and greatly improves the efficiency of lighting design and debugging. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for evaluating cultural and tourism lighting display effects based on computer vision according to an embodiment of the present invention;

[0050] Figure 2 It is a structural schematic diagram of a cultural and tourism lighting display effect evaluation system based on computer vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0053] Figure 1 Schematic diagram of a flow chart of a method for evaluating cultural and tourism lighting display effects based on computer vision according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0054] S1. Collect a panoramic image set of the cultural and tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules;

[0055] The cultural tourism lighting display environment refers to the use of specific lighting design in museums, exhibition halls or other display spaces to enhance the visual effects of exhibits and the experience of the audience. The panoramic image set is a collection of images with a wide field of view, which is usually used to present the overall layout of the environment for more comprehensive analysis and understanding. The local close-up image set refers to a collection of images containing specific areas or details, which aims to capture the detailed features of objects and is usually used for cultural relics protection or art analysis. The multi-scale convolutional neural network is a convolutional neural network structure that can process information of different scales simultaneously to enhance the perception of different features in the image and is suitable for complex scene analysis. The semantic segmentation is an image processing technology that aims to divide an image into different regions and assign corresponding labels to each region to achieve understanding and classification of different objects in the scene. analysis, the key cultural relic areas refer to the parts of cultural relics that need special attention and protection in the exhibition, usually involving their cultural, historical or artistic value, the deep convolutional neural network based on transfer learning refers to the use of convolutional neural network models trained on large-scale data sets and applying them to tasks in specific fields to improve performance and reduce training time, the point cloud data is a three-dimensional spatial data set obtained by laser scanning or other technologies, usually used to represent the shape and characteristics of objects or environments, the lighting effect evaluation rules are a set of standards or indicators used to evaluate the impact of lighting design on display effects, which may include brightness, color temperature, uniformity and other aspects, and the lighting effect characteristic parameters are specific parameters used to quantify lighting effects, which may include light intensity, illumination angle, color performance, etc., to help optimize lighting design and improve the display effect of exhibits.

[0056] In an optional embodiment,

[0057] Collect panoramic image sets of the cultural and tourism lighting display environment and local close-up image sets corresponding to key cultural relics, perform semantic segmentation on the panoramic image sets through a multi-scale convolutional neural network, obtain key cultural relic areas, visit path areas and background areas in each panoramic image, perform texture recognition and material recognition on the local close-up image sets through a deep convolutional neural network based on transfer learning, obtain texture features and material features of key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules, including:

[0058] A panoramic image set of the cultural tourism lighting display environment is collected through a pre-set camera, and a close-up image set of the surface of key cultural relics is obtained by using a macro lens to obtain a local close-up image set corresponding to the current key cultural relics;

[0059] For the panoramic image set, a cultural tourism lighting display environment training set is constructed and the images in the training set are manually annotated, a multi-scale convolutional neural network based on an attention mechanism is trained using the manually annotated training set images, the parameters of the multi-scale convolutional neural network are adjusted, and semantic segmentation is performed on each image in the panoramic image set. In each panoramic image, the semantic label corresponding to each pixel is determined and the key cultural relics area, the tour path area, and the background area are identified;

[0060] Collect and organize image data of cultural relics texture and material and perform fine annotation, generate annotated close-up image set and expand it through data enhancement technology, select a transfer learning model, modify the top-level structure and adjust the loss function of the selected transfer learning model based on the characteristics and categories of cultural relics, use the expanded annotated close-up image set to train the adjusted transfer learning model, obtain pre-training parameters and migrate them to the convolution layer of the deep convolutional neural network, the convolution layer uses the image in the local close-up image set as input, combines the compact bilinear pooling layer to perform high-order bilinear encoding, and obtains the texture features and material features corresponding to the key cultural relics;

[0061] The panoramic image set obtained by the motion reconstruction structure algorithm is sparsely reconstructed to obtain the sparse point cloud and camera pose of the display environment. The multi-view stereo reconstruction algorithm is combined with the semantic segmentation results of the panoramic image set to perform dense matching and three-dimensional reconstruction to obtain a dense point cloud model of the display environment. Multi-perspective depth images are generated and registered in combination with deep learning. A triangular mesh surface model of the display environment is generated by the Poisson surface reconstruction algorithm. Texture mapping is performed based on the panoramic image and the local close-up image to obtain a three-dimensional space model corresponding to the cultural and tourism lighting display environment. Based on the texture features and the material features, a normal map and a displacement map are added to the three-dimensional space model. The environment is rendered by a physically based rendering technology, and the panoramic image set and the local close-up image set are mapped to the three-dimensional space model. The three-dimensional space model is uniformly sampled by the Poisson disk sampling algorithm to generate uniformly distributed point cloud data. Each sampling point is semantically annotated in combination with the semantic segmentation information. Lighting effect feature parameters corresponding to multiple dimensions are extracted according to pre-set lighting effect evaluation rules, and the lighting effect feature parameters are stored in a point cloud data structure.

[0062] The manual labeling refers to manually adding labels or annotations to images or objects in the dataset for training and verifying machine learning models. The semantic labels are used to describe the labels of objects or regions in the image, usually pointing to specific categories or features for semantic understanding and analysis. The annotated close-up image set refers to a collection of close-up images that have been manually annotated, focusing on specific objects or regions for analysis and model training. The data augmentation technology refers to increasing the diversity of the dataset by transforming the training data (such as rotation, scaling, flipping, etc.) to improve the generalization ability of the model. The top-level structure modification refers to modifying the top layer of the model (such as the classification layer) to adapt to new tasks or dataset requirements, usually involving adjusting the number of output categories. The compact bilinear pooling layer is an efficient pooling layer that aims to capture the bilinear relationship between features in a compact way to improve the performance of the model. The high-order bilinear encoding refers to encoding the input data by considering higher-order feature interactions to capture more complex relationships and patterns. The sparse 3D reconstruction refers to the technology of reconstructing 3D models using sparse data (such as sparse point clouds), which is usually used to reduce computational complexity and memory consumption. A cloud refers to a data set consisting of a small number of three-dimensional coordinate points, which is usually used to represent the basic shape and structure of an object or scene. The camera pose refers to the position and orientation of a camera in three-dimensional space, which is usually used to reconstruct a scene or perform image registration. The multi-view stereo reconstruction algorithm is an algorithm that uses multiple images with different perspectives to perform three-dimensional reconstruction, and restores the three-dimensional structure by matching image feature points. The Poisson surface reconstruction algorithm is a three-dimensional reconstruction technology based on the Poisson equation, which uses point cloud data to generate a smooth surface model and is suitable for high-quality geometric reconstruction. The triangular mesh surface model is a three-dimensional model composed of triangular patches, which is used to represent the geometric shape of the surface of an object and is widely used in computer graphics and game development. The normal map is a technology for enhancing the surface details of a three-dimensional model, which simulates more complex lighting effects by adding normal information to the model surface. The displacement map is a technology that simulates more complex surface shapes and details by changing the position of surface vertices during rendering, providing higher visual realism. The physically based rendering technology is a rendering method that simulates the physical behavior of light to produce more realistic image effects, which usually involves factors such as lighting models, material properties and environmental effects.

[0063] Use a pre-set camera to take panoramic photos of the target cultural tourism lighting display environment to obtain a panoramic image set, use a macro lens to take close-up photos of the key cultural relics in the display environment to obtain a local close-up image set, select some images from the panoramic image set to construct a training set, and manually annotate the training set images. Use the manually annotated training set images to train a multi-scale convolutional neural network based on the attention mechanism, adjust the network parameters, and use the trained multi-scale convolutional neural network to perform semantic segmentation on each image in the panoramic image set. In each panoramic image, determine the semantic label corresponding to each pixel, identify the key cultural relics area, reference For example, a panoramic camera is used to take panoramic photos of the museum exhibition hall to obtain a panoramic image set of the exhibition hall, a macro lens is used to take close-up photos of key cultural relics (such as bronze ware) in the exhibition hall to obtain a local close-up image set of the surface of the bronze ware, 100 images are selected from the panoramic image set of the museum exhibition hall to construct a training set, and the cultural relics, visiting paths and background areas in the training set images are manually annotated, a multi-scale convolutional neural network is trained using the annotated training set images, and the trained network is used to perform semantic segmentation on the panoramic image set to identify the cultural relics area, visiting path area and background area in the exhibition hall;

[0064] Collect and organize image data of the texture and material of cultural relics, and perform fine annotation to generate an annotated close-up image set. Use data enhancement technology to expand the annotated close-up image set, select a suitable transfer learning model, and modify the top-level structure and adjust the loss function of the model according to the characteristics and categories of cultural relics. Use the expanded annotated close-up image set to train the adjusted transfer learning model to obtain pre-training parameters, and migrate the pre-training parameters to the convolution layer of the deep convolutional neural network. Use the image in the local close-up image set as input, connect the compact bilinear pooling layer after the convolution layer, perform high-order bilinear encoding, and obtain the texture features and material features corresponding to the key cultural relics. For example, collect image data of the texture and material of the surface of bronze ware, and annotate it to generate an annotated close-up image set. Use data enhancement technology to expand the annotated close-up image set, and select ResNet-50 as the transfer learning model. According to the characteristics of bronze ware, modify the top-level structure of ResNet-50, adjust the loss function, and use the expanded annotated close-up image set to train the model. Migrate the parameters obtained by training to another deep convolutional neural network, use the local close-up image of the bronze ware as input, and extract the texture features and material features of the surface of the bronze ware;

[0065] Use the motion reconstruction structure algorithm to perform sparse three-dimensional reconstruction on the collected panoramic image set to obtain a sparse point cloud and camera pose of the display environment. Use the multi-view stereo reconstruction algorithm to perform dense matching and three-dimensional reconstruction in combination with the semantic segmentation results of the panoramic image set to obtain a dense point cloud model of the display environment. Generate multi-perspective depth images based on deep learning and perform registration. Use the Poisson surface reconstruction algorithm to generate a triangular mesh surface model of the display environment. Perform texture mapping based on the panoramic image and local close-up images to obtain a three-dimensional space model corresponding to the cultural tourism lighting display environment. Exemplarily, use the motion reconstruction structure algorithm to perform sparse three-dimensional reconstruction on the panoramic image set of the museum exhibition hall to obtain a sparse point cloud and camera pose of the exhibition hall. Use the multi-view stereo reconstruction algorithm to perform dense matching and three-dimensional reconstruction in combination with the semantic segmentation results of the panoramic image set to obtain a dense point cloud model of the exhibition hall. Generate multi-perspective depth images based on deep learning and perform registration. Use the Poisson surface reconstruction algorithm to generate a triangular mesh surface model of the exhibition hall. Perform texture mapping based on the panoramic image and local close-up images of cultural relics to obtain a three-dimensional space model of the museum exhibition hall.

[0066] In the generated three-dimensional space model, based on the extracted texture features and material features, normal maps and displacement maps are added, and the three-dimensional space model is rendered using physically based rendering technology. The panoramic image set and the local close-up image set are mapped to the three-dimensional space model, and the three-dimensional space model is uniformly sampled using a Poisson disk sampling algorithm to generate uniformly distributed point cloud data. Each sampling point is semantically labeled in combination with semantic segmentation information, and lighting effect feature parameters corresponding to multiple dimensions are extracted according to pre-set lighting effect evaluation rules. The lighting effect feature parameters are stored in a point cloud data structure. For example, in the three-dimensional space model of a museum exhibition hall, According to the extracted surface texture features and material features of the bronze ware, normal maps and displacement maps are added, and the exhibition hall model is rendered using physical rendering technology. The panoramic image of the exhibition hall and the local close-up image of the bronze ware are mapped to the three-dimensional space model. The Poisson disk sampling algorithm is used to uniformly sample the three-dimensional space model of the museum exhibition hall to generate uniformly distributed point cloud data. Each sampling point is semantically annotated in combination with semantic segmentation information, and the cultural relics, visiting paths and background areas are annotated. According to the preset lighting effect evaluation rules, the characteristic parameters of the lighting in the exhibition hall, such as brightness, color temperature, and color rendering index, are extracted, and these parameters are stored in the point cloud data structure.

[0067] In this embodiment, by taking panoramic photos of the display environment and taking close-up photos of key cultural relics, the visual information of the display environment can be fully collected, including the overall spatial layout and the detailed texture, material and other features of the key cultural relics, providing a rich data basis for subsequent semantic segmentation, three-dimensional reconstruction and environment rendering. By performing semantic segmentation on the panoramic image set, the key cultural relic area, the visiting path area and the background area in the display environment can be identified, and the semantic understanding of the display environment can be achieved. It can be used to guide the subsequent three-dimensional reconstruction and point cloud data annotation, and improve the accuracy of reconstruction and annotation. By extracting texture and material features from the local close-up image set, the fine texture and material features of the surface of the key cultural relics can be obtained, which can be used for subsequent three-dimensional model texture mapping and material rendering, and improve the realism and detail expression of the three-dimensional model. By uniformly sampling the three-dimensional space model, uniformly distributed point cloud data is generated, and semantic annotation and lighting effect feature parameter extraction are performed on each sampling point, so as to achieve quantitative analysis of the lighting effect of the display environment, and provide a basis for subsequent lighting optimization. In summary, this embodiment realizes the comprehensive digitalization and intelligent analysis of the cultural and tourism lighting display environment, and provides a new technical means and application idea for the field of cultural and tourism lighting.

[0068] In an optional embodiment,

[0069] The objective function of sparse 3D reconstruction of the acquired panoramic image set by the motion reconstruction structure algorithm is as follows:

[0070] ;

[0071] in, G represents the objective function value, R i Indicates i The rotation matrix of the camera, t i Indicates i The camera's translation vector, X j Indicates j The coordinates of the three-dimensional points, x ij Indicates j The three-dimensional point in i The projected coordinate points in the camera, v ij Is a visibility marker, used to indicate the j Is the point in i Within the camera field of view, d () is the Euclidean distance function, which is used to represent the reprojection error of the point. σ Represents the scale parameter.

[0072] In this embodiment, the rotation matrix and translation vector of each camera and the coordinates of the three-dimensional points in the scene can be estimated by minimizing the objective function, and the joint optimization of the camera pose and the three-dimensional point coordinates can be achieved. The joint optimization can improve the accuracy of three-dimensional reconstruction, so that the reconstructed sparse point cloud can more accurately reflect the real structure of the scene. The sparse point cloud and camera pose obtained by the motion reconstruction structure algorithm can provide a good initialization for subsequent dense matching and three-dimensional reconstruction steps. The sparse point cloud can be used to guide the search range of dense matching, reduce the amount of matching calculations, and accelerate the convergence of the algorithm. In summary, this embodiment uses the motion reconstruction structure algorithm to perform sparse three-dimensional reconstruction on a panoramic image set, so as to restore the three-dimensional structure information of the scene, achieve joint optimization of the camera pose and three-dimensional point coordinates, deal with occlusion problems in the scene, improve the robustness and stability of sparse reconstruction, and provide a good initialization for subsequent dense matching and three-dimensional reconstruction.

[0073] S2. Add the lighting effect characteristic parameters to the preset lighting effect evaluation model, and obtain a lighting effect evaluation score sequence by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with the strategy gradient algorithm, and key score subsequences are selected in the lighting effect evaluation score sequence by the Monte Carlo tree search algorithm for fusion to obtain a dynamic lighting effect evaluation score. The multi-channel lighting effect image is simulated by the ray tracing algorithm and the multi-channel lighting characteristic parameters are extracted. A multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain a multi-view lighting effect evaluation score. The dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted and fused to obtain an overall effect evaluation score. The lighting effect grade is obtained by fuzzy comprehensive evaluation in combination with the preset multi-level scoring criteria.

[0074] The lighting effect evaluation score sequence is a series of scores for evaluating lighting effects, which is usually formed by evaluating multiple lighting configurations to form a time series. The Monte Carlo tree search algorithm is a search algorithm for decision-making processes, which evaluates possible decision paths through random simulation and tree structure, and is widely used in game AI. The key score subsequence is a score subset extracted from the lighting effect evaluation score sequence that has a greater impact on the overall performance or performance. The dynamic lighting effect evaluation score is a lighting effect evaluation score calculated in real time, which can adapt to environmental changes and user feedback. The ray tracing algorithm is a computer graphics technology that generates high-quality images by simulating the propagation of light and is often used to render realistic scenes. The multi-channel lighting feature parameters are multiple parameters used to describe lighting effects, which may involve the intensity, color and direction of different light sources. The regression prediction is a technology for predicting future values ​​using a regression model, which usually involves modeling the relationship between input features and target variables. The multi-level scoring criteria is a set of hierarchical evaluation criteria for detailed scoring of lighting effects, which may include details and criteria at different levels. The fuzzy comprehensive evaluation is an evaluation method that uses fuzzy logic to handle uncertainty and comprehensively considers multiple factors to conduct a comprehensive evaluation.

[0075] In an optional embodiment,

[0076] The lighting effect characteristic parameters are added to a preset lighting effect evaluation model, and a lighting effect evaluation score sequence is obtained by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with a policy gradient algorithm, and key score subsequences are selected and fused in the lighting effect evaluation score sequence through a Monte Carlo tree search algorithm, so as to obtain a dynamic lighting effect evaluation score including:

[0077] Adding the lighting effect characteristic parameters to a preset lighting effect evaluation model, constructing a virtual cultural and tourism lighting display environment based on the cultural and tourism lighting display environment and the animation engine and rendering it through the three-dimensional space model, interacting the lighting effect evaluation model with the virtual cultural and tourism lighting display environment, generating state data, action data and reward data and adding them to the policy gradient algorithm, generating a lighting effect evaluation score sequence, and updating the parameters of the lighting effect evaluation model based on the lighting effect evaluation score sequence and the policy gradient algorithm;

[0078] A Monte Carlo tree is constructed according to the lighting effect evaluation score sequence, wherein each node of the Monte Carlo tree represents a lighting parameter configuration, and each path from a root node to a leaf node represents a parameter adjustment sequence. For each node in the Monte Carlo tree, an upper confidence bound algorithm is used to evaluate and generate a node evaluation score, a node with the highest node evaluation score is selected, child nodes are randomly generated and the node evaluation score corresponding to each child node is calculated again, and the generation and selection are repeated until a preset maximum number of iterations is reached to obtain a search tree, and the search paths in the search tree are traversed, and the node evaluation scores corresponding to the nodes on the search paths are summed as search path scores and sorted in descending order, and the search paths in the top 10% are selected as key score subsequences, and for the selected key score subsequences, the corresponding lighting effect evaluation scores are calculated, and weighted fusion is performed in combination with time span and time attenuation to obtain the dynamic lighting effect evaluation score.

[0079] The animation engine is a technical framework for generating and playing animations, which usually includes physical simulation, rendering and interactive functions. The policy gradient algorithm is a reinforcement learning method that improves the decision-making ability of the intelligent agent by directly optimizing the strategy, which usually involves gradient estimation of the strategy. The leaf node is a node without child nodes in the tree structure, which usually represents the final decision or result. The root node is the starting node of the tree structure, and all other nodes extend from this node. The search tree is a tree structure used to represent possible decision paths. The node represents the state or decision, and the edge represents the state transfer. The time span is used to measure the duration of an event or decision, which usually affects the accuracy of evaluation or prediction. The time decay refers to a strategy that gives lower weight to past information in the evaluation model to emphasize the impact of recent data.

[0080] The extracted lighting effect characteristic parameters are added to the pre-set lighting effect evaluation model as the input data of the model, a virtual cultural and tourism lighting display environment is constructed based on the cultural and tourism lighting display environment and the animation engine, the virtual cultural and tourism lighting display environment is rendered using the generated three-dimensional space model, the lighting effect evaluation model is interacted with the virtual cultural and tourism lighting display environment, state data, action data and reward data are generated, and added to the policy gradient algorithm. For example, the characteristic parameters such as lighting brightness, color temperature, and color rendering index extracted from the museum exhibition hall are added to the lighting effect evaluation model, the virtual display environment of the museum exhibition hall is constructed using the Unity engine, and the virtual environment is rendered using the generated three-dimensional space model of the exhibition hall, the lighting effect evaluation model is interacted with the virtual museum exhibition hall environment, and state data under different lighting configurations in the exhibition hall, action data for adjusting lighting parameters, and corresponding lighting effect evaluation reward data are generated, and added to the policy gradient algorithm;

[0081] A lighting effect evaluation score sequence is generated based on a policy gradient algorithm, and the parameters of the lighting effect evaluation model are updated using the lighting effect evaluation score sequence and the policy gradient algorithm. A Monte Carlo tree is constructed based on the lighting effect evaluation score sequence, where each node represents a lighting parameter configuration, and each path from the root node to the leaf node represents a parameter adjustment sequence. For each node in the Monte Carlo tree, an upper confidence bound algorithm is used to evaluate and generate a node evaluation score, and the node with the highest node evaluation score is selected. Child nodes are randomly generated and the node evaluation score corresponding to each child node is calculated again. The generation and selection are repeated until a preset maximum number of iterations is reached to obtain a search tree, and the search paths in the search tree are traversed. The node evaluation scores corresponding to the nodes on the search path and the sum of the node evaluation scores are used as the search path scores and are sorted in descending order. The search paths in the top 10% are selected as the key score subsequences. For example, a museum exhibition is generated based on a policy gradient algorithm. The lighting effect evaluation score sequence under different lighting configurations in the hall is used. The evaluation score sequence and the policy gradient algorithm are used to update the parameters of the lighting effect evaluation model so that the model can more accurately evaluate the lighting effect. A Monte Carlo tree is constructed based on the evaluation score sequence of different lighting configurations in the museum exhibition hall. Each node represents a lighting parameter configuration (such as brightness, color temperature, etc.), and each path represents a series of lighting parameter adjustments. The upper confidence bound algorithm is used to evaluate each node and generate a node evaluation score. In the Monte Carlo tree of the lighting parameters of the museum exhibition hall, the node with the highest evaluation score is selected, and child nodes are randomly generated (such as increasing brightness, reducing color temperature, etc.) and the evaluation scores of the child nodes are calculated. This process is repeated until the preset number of iterations is reached to obtain a search tree. The paths in the search tree are traversed, and the score of each path (the sum of the evaluation scores of the nodes on the path) is calculated. The paths are sorted in descending order by score, and the paths with scores in the top 10% are selected as the key score subsequences.

[0082] For the selected key score subsequence, the corresponding lighting effect evaluation score is calculated, and the evaluation score is weightedly fused in combination with the time span and time attenuation to obtain a dynamic lighting effect evaluation score. By way of example, for the selected museum exhibition hall lighting parameter adjustment path (key score subsequence), the lighting effect evaluation score corresponding to each path is calculated, and the evaluation scores at different time points are considered and weightedly fused in combination with the time span and time attenuation factor to obtain a comprehensive dynamic lighting effect evaluation score, which reflects the dynamic changes in the exhibition hall lighting effect over a period of time.

[0083] In this embodiment, by interacting the lighting effect evaluation model with a virtual cultural and tourism lighting display environment rendered based on a three-dimensional space model, state data, action data, and reward data under different lighting parameter configurations can be generated in the virtual environment, allowing the evaluation model to learn and optimize in the virtual environment, thereby improving the adaptability and robustness of the evaluation model. By adding the generated state data, action data, and reward data to the policy gradient algorithm, and using the lighting effect evaluation score sequence to update the parameters of the evaluation model, the adaptive optimization of the evaluation model is achieved, thereby improving the accuracy and generalization ability of the evaluation model. By constructing a Monte Carlo tree and evaluating the nodes, the lighting parameters can be efficiently explored. configuration space, find the optimal or near-optimal parameter configuration. By selectively expanding and evaluating promising nodes, a good parameter configuration can be found within a limited number of searches, avoiding the high computational overhead of exhaustive search. The focusing mechanism can filter out most of the less important adjustment paths, thereby improving the efficiency of evaluation and optimization. In summary, this embodiment realizes the dynamic lighting effect evaluation of the cultural and tourism lighting display environment, can adaptively optimize the evaluation model, efficiently explore the lighting parameter configuration space, focus on the key adjustment paths, and comprehensively consider the time factor to give a dynamic evaluation score, providing an explainable and optimizable evaluation result, and providing strong technical support for the design and optimization of the cultural and tourism lighting display environment.

[0084] In an optional embodiment,

[0085] The upper confidence bound algorithm is used to evaluate and generate the node evaluation score as shown in the following formula:

[0086] ;

[0087] in, w g Representation Node g The cumulative reward value, n g Representation Node g The number of visits, α Represents the power of the number of node visits, which is used to adjust the degree of exploration. N Indicates the total number of visits. V g Representation Node g The variance of the value of is used to represent the uncertainty of node rewards.

[0088] In this embodiment, by adding an exploration reward item to the average reward value of the node to balance the utilization of known good nodes and the exploration of unknown nodes, the optimal or near-optimal node can be found efficiently within a limited number of searches. By considering the uncertainty of the node reward, the uncertainty of the reward estimation can be more robustly dealt with, avoiding over-reliance on unreliable reward estimates. The time-optimization feature can continuously optimize the node evaluation during the search process and gradually converge to the optimal or near-optimal node. In summary, this embodiment provides effective technical support for the evaluation and optimization of lighting effects in cultural and tourism lighting display environments.

[0089] In an optional embodiment,

[0090] The multi-channel lighting effect image is simulated by the ray tracing algorithm and the multi-channel lighting feature parameters are extracted. The multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain the multi-view lighting effect evaluation score. The dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted fused to obtain the overall effect evaluation score. The lighting effect level is obtained by fuzzy comprehensive evaluation combined with the preset multi-level scoring criteria, including:

[0091] Select representative viewpoints in a virtual environment, render at each viewpoint through a ray tracing algorithm, generate a lighting effect image, extract multiple lighting feature channels for the lighting effect image, calculate features on each lighting feature channel through an image pyramid algorithm, obtain lighting information of different granularities, combine to obtain multi-channel lighting feature parameters, extract the multi-channel lighting feature parameters from each viewpoint and pre-acquired spatiotemporal statistical indicators, construct them into a multi-view lighting effect evaluation vector according to a pre-set splicing method, select a Gaussian process regression model, train with the lighting effect score scored manually as a training label, perform regression prediction through the trained Gaussian process regression model, and obtain the multi-view lighting effect evaluation score;

[0092] The user interaction behavior data is obtained and the user's preference for different lighting effects is determined through a collaborative filtering algorithm to generate user preferences. The weight corresponding to the dynamic lighting effect evaluation score and the weight corresponding to the multi-view lighting effect evaluation score are determined through a weight generation model in combination with the cultural background and historical information corresponding to the cultural tourism lighting display environment. The overall effect evaluation score is obtained by weighted fusion. According to a pre-set multi-level scoring standard, the overall effect evaluation score is added to the membership function corresponding to each level, the membership vector is calculated and synthesized into a comprehensive membership through a fuzzy comprehensive evaluation method, and the lighting effect level corresponding to the current lighting effect characteristic parameter is output based on the maximum membership principle and the comprehensive membership.

[0093] The image pyramid algorithm is an image processing technology that improves the efficiency of image analysis and processing by constructing image hierarchies of different resolutions. The spatiotemporal statistical indicators are statistics used to analyze and describe the relationship between time and space data, and are usually used to understand dynamic changes and patterns. The Gaussian process regression model is a non-parametric regression method that models data through a Gaussian process and is suitable for complex and uncertain relationship predictions. The cultural background refers to the values, beliefs and customs in a specific cultural environment, which usually affect people's behavior and decision-making. The historical information refers to historical data and background related to an event or object, and is often used to analyze and understand the current state. The membership function refers to a function used to describe the relationship between an element and a fuzzy set in fuzzy logic, and is usually output as a value between 0 and 1. The maximum membership principle refers to selecting the option with the largest membership value as the final decision basis in fuzzy decision-making to ensure that the most suitable solution is selected.

[0094] In the virtual cultural tourism lighting display environment, multiple representative viewpoints are selected, and rendering is performed at each selected viewpoint through a ray tracing algorithm to generate a lighting effect image under the viewpoint. For the lighting effect image generated at each viewpoint, multiple lighting feature channels are extracted, such as brightness channel, contrast channel, color temperature channel, etc., and features are calculated on each lighting feature channel through an image pyramid algorithm to obtain lighting information of different granularities (such as original size, 1 / 2 size, 1 / 4 size, etc.), and lighting information of different granularities is combined to obtain multi-channel lighting feature parameters. For each viewpoint, the extracted multi-channel lighting feature parameters and the pre-acquired spatiotemporal statistical indicators (such as lighting duration, change frequency, etc.) are constructed into a multi-view lighting effect evaluation vector according to a pre-set splicing method. By way of example, in the virtual museum exhibition hall environment, 10 representative viewpoints are selected, such as the entrance of the exhibition hall, the center of the main exhibition area, and key exhibits. Nearby, etc., use the ray tracing algorithm to render each viewpoint, generate 10 lighting effect images at different viewing angles in the exhibition hall, extract 6 lighting feature channels such as brightness, contrast, and color temperature for the 10 exhibition hall lighting effect images, respectively, use the image pyramid algorithm to calculate the features on each channel, and obtain lighting information of three granularities: original size, 1 / 2 size, and 1 / 4 size. Combine these lighting information of different granularities to obtain multi-channel lighting feature parameters of each image, a total of 60 (10 images × 6 channels). For 10 viewpoints, 60 multi-channel lighting feature parameters are extracted from each viewpoint, and spatiotemporal statistical indicators such as lighting duration and change frequency of each viewpoint are obtained. According to the preset splicing method, these feature parameters and statistical indicators are constructed into 10 multi-view lighting effect evaluation vectors, each of which contains 62 elements (60 lighting feature parameters + 2 spatiotemporal statistical indicators);

[0095] A Gaussian process regression model was selected, and the manually scored lighting effect scores were used as training labels to train the model. The trained Gaussian process regression model was used to regress and predict the multi-perspective lighting effect evaluation vector to obtain the multi-perspective lighting effect evaluation score. The interactive behavior data of users in the virtual environment was obtained, and the user's preference for different lighting effects was analyzed through a collaborative filtering algorithm to generate a user preference vector. The weights of the dynamic lighting effect evaluation score and the multi-perspective lighting effect evaluation score were determined through a weight generation model in combination with the cultural background and historical information corresponding to the cultural tourism lighting display environment. The dynamic lighting effect evaluation score and the multi-perspective lighting effect evaluation score were calculated according to the weight generation model. The determined weights are weighted fused to obtain the overall effect evaluation score. According to the pre-set multi-level scoring standard, the overall effect evaluation score is added to the membership function corresponding to each level, the membership vector is calculated, and the membership vector is synthesized by the fuzzy comprehensive evaluation method to obtain the comprehensive membership. Based on the maximum membership principle and the comprehensive membership, the lighting effect level corresponding to the current lighting effect characteristic parameters is output. For example, 100 manual ratings of different lighting effects are collected as training labels, and the Gaussian process regression model is used to train these 100 samples. The multi-view lighting effect evaluation vectors of 10 viewpoints are input into the trained model for regression prediction. The multi-view lighting effect evaluation scores of 10 viewpoints were obtained. The interactive behavior data of 1,000 users in the virtual museum exhibition hall, such as clicks, dwell time, etc., were collected. The collaborative filtering algorithm was used to analyze these data, and the degree of preference of each user for different lighting effects was generated. 1,000 user preference vectors were obtained. Taking into account the cultural background and historical information of the museum exhibition hall, the weight of the dynamic lighting effect evaluation score was determined to be 0.6, and the weight of the multi-view lighting effect evaluation score was determined to be 0.4 through the weight generation model. The dynamic lighting effect evaluation score of the museum exhibition hall (such as 0.85) and the multi-view lighting effect evaluation score (such as 0.92) were weighted. Weighted fusion is performed, and the overall effect evaluation score is calculated to be 0.876 (0.85×0.6+0.92×0.4). A 5-level scoring standard is preset (very poor, poor, average, good, and very good). The overall effect evaluation score of 0.876 is added to the membership function of each level, and the membership vector is calculated to be [0.0, 0.0, 0.1, 0.7, 0.2]. The fuzzy comprehensive evaluation method is used to synthesize the membership vector, and the comprehensive membership is 0.82. According to the maximum membership principle, the level corresponding to the comprehensive membership of 0.82 is "good", so the lighting effect level corresponding to the current lighting effect characteristic parameters is output as "good".

[0096] In this embodiment, by rendering the lighting effect image at different viewpoints and using the image pyramid algorithm to extract different granularity information of multiple lighting feature channels, the characteristics of the lighting effect can be fully characterized from multiple angles and multiple scales, providing a rich and comprehensive feature representation for the subsequent lighting effect evaluation. By splicing the extracted multi-channel lighting feature parameters with the pre-acquired spatiotemporal statistical indicators (such as lighting duration, change frequency, etc.), a multi-view lighting effect evaluation vector is constructed, which can simultaneously consider the influence of the spatial dimension and the time dimension on the lighting effect, and can more comprehensively characterize the performance of the lighting effect at different perspectives, and consider the dynamic change characteristics of the lighting effect. The prediction method based on machine learning can automatically and intelligently evaluate the lighting effect, reduce the workload of manual scoring, and integrate the weight generation method of user preferences and cultural background to make the evaluation results closer to user needs and the characteristics of the display environment. In summary, this embodiment can generate personalized, comprehensive and balanced evaluation results, and present them in the form of intuitive lighting effect grades, which provides an important basis and reference for optimizing the lighting design of the cultural and tourism lighting display environment.

[0097] S3. If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the Gaussian process optimization algorithm is combined to solve the optimal lighting equipment parameters and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, and the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0098] The level threshold is a boundary used to divide different levels or categories, which is usually set in the scoring, evaluation or decision-making process to determine whether a certain indicator meets the standard. The online lighting optimization model is a model that updates and adjusts lighting parameters in real time, and can dynamically optimize lighting effects according to environmental changes or user feedback. The Gaussian process optimization algorithm is an optimization method based on Gaussian processes. It evaluates the objective function by building a probability model and is suitable for complex and uncertain optimization problems. The optimal lighting equipment parameters are equipment settings (such as intensity, color, angle, etc.) obtained through optimization calculation in lighting design to achieve the best lighting effect. The deep generative adversarial network refers to a deep learning model composed of a generator and a discriminator. It generates high-quality images or data through adversarial training and is widely used in image synthesis and enhancement. The training lighting effect image is an image collection used to train machine learning models, focusing on the effect performance under different lighting conditions, helping the model learn the impact of lighting changes. The illuminance distribution describes the distribution of light intensity in a space or on a surface. It is usually obtained by measurement and calculation and is used to analyze lighting effects. The illuminance deviation map is an image that reflects the difference between actual illuminance and expected illuminance, and is usually used to evaluate the effectiveness and consistency of lighting design.

[0099] In an optional embodiment,

[0100] If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the optimal lighting equipment parameters are solved in combination with the Gaussian process optimization algorithm, and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, and the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached, including:

[0101] Acquire the lighting effect level obtained by judgment. If the lighting effect level does not reach a preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting device control module, and the lighting optimization problem is modeled as a black box optimization problem in combination with the Gaussian process optimization algorithm. The observation data set is acquired and the lighting device parameters and the corresponding lighting effect evaluation function are defined. It is assumed that the function value of the lighting effect evaluation function obeys a multivariate Gaussian distribution. The posterior distribution is calculated for the observation points in the observation data set and the acquisition function is introduced through Bayesian optimization, and the optimal parameters are searched to obtain the optimal lighting device parameters.

[0102] Based on the optimal lighting equipment parameters, randomly generate training lighting effect images through a preset deep generative adversarial network, add the training lighting effect images to the lighting effect evaluation model, extract topological structure and parameter information through a graph neural network, model each lamp as a node of the graph, and model the relative position and orientation relationship between the lamps as the edge of the graph, obtain a global representation vector corresponding to the current layout through multi-layer graph convolution and pooling, extract texture and semantic features for the rendered image through a convolutional neural network, generate image features, cascade the global representation vector and the image features and add them to a multi-layer fully connected network to obtain a layout evaluation score corresponding to the current layout, repeatedly calculate the layout evaluation scores corresponding to different training lighting effect images, and select the global representation vector with the highest layout evaluation score as the optimal lighting layout;

[0103] Based on the optimal lighting layout, lighting arrangement is performed for the cultural and tourism lighting display environment. After the arrangement is completed, the illuminance distribution is collected in real time and compared with the rendering generated by the three-dimensional space model to generate the illuminance deviation map. Based on the illuminance deviation map, the optimal lighting layout is again optimized and the lighting parameters are adjusted. The optimized lighting effect is again evaluated through a multi-level scoring standard. If the lighting effect level is greater than the level threshold, the current lighting effect level, layout parameters and lighting parameters obtained by evaluation are output. Otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0104] The black-box optimization problem refers to the challenge of optimizing a certain objective function without a clear mathematical model or analytical expression. It is usually impossible to directly obtain the internal structure or gradient information of the function, and optimization can only be performed by observing the input and output results. The observation data set is an actual data set used for analysis and modeling, which contains observations of specific phenomena or processes. The multivariate Gaussian distribution is a probability distribution that describes the joint distribution of multiple related variables. It is characterized by a mean vector and a covariance matrix. It is widely used in statistics and machine learning, especially when modeling continuous data. In machine learning or deep learning, it refers to a vector representing the entire data instance or set obtained by some means (such as feature extraction or encoding), which can capture the global characteristics of the data and be used for further analysis or prediction.

[0105] Obtain the lighting effect level, compare the lighting effect level with a preset level threshold, and if the lighting effect level does not reach the level threshold, enter the online lighting optimization process; otherwise, output the current lighting effect level, layout parameters, and lighting parameters, and according to the pre-built online lighting optimization model, communicate in real time through the three-dimensional space model and the lighting equipment control module, and combine the Gaussian process optimization algorithm to model the lighting optimization problem as a black box optimization problem, obtain the observation data set, define the lighting equipment parameters and the corresponding lighting effect evaluation function, assume that the function value of the lighting effect evaluation function obeys the multivariate Gaussian distribution, calculate the posterior distribution of the observation points in the observation data set, introduce the acquisition function through Bayesian optimization, search for the optimal parameters, and obtain the optimal lighting equipment parameters. For example, assuming the lighting effect level The default level threshold is "better". Since "bad" does not reach the threshold of "better", the online lighting optimization process is entered. Through the online lighting optimization model, the three-dimensional space model and the lighting equipment control module communicate, and 100 observation data points are obtained. Each data point contains lighting equipment parameters (such as lamp position, direction, brightness, etc.) and the corresponding lighting effect evaluation function value. Assuming that the evaluation function value obeys the multivariate Gaussian distribution, the posterior distribution of these 100 observation points is calculated, and the acquisition function is introduced through Bayesian optimization to search for the optimal parameters. A set of optimal lighting equipment parameters is obtained, such as lamp position (2.5m, 3.0m, 4.0m), direction (30°, 45°, 60°), brightness (800lux, 1000lux, 1200lux), etc.

[0106] Based on the optimal lighting equipment parameters, a training lighting effect image is randomly generated through a pre-set deep generative adversarial network, and the training lighting effect image is added to the lighting effect evaluation model. The topological structure and parameter information are extracted through the graph neural network, and each lamp is modeled as a node of the graph. The relative position and orientation relationship between the lamps are modeled as the edge of the graph. The global representation vector corresponding to the current layout is obtained through multi-layer graph convolution and pooling. For the rendered image, the texture and semantic features are extracted through the convolutional neural network to generate image features. The global representation vector and image features are cascaded and added to the multi-layer fully connected network to obtain the layout evaluation score corresponding to the current layout. The layout evaluation scores corresponding to different training lighting effect images are repeatedly calculated, and the global representation vector with the highest layout evaluation score is selected as the optimal lighting layout. Based on the optimal lighting layout, the cultural and tourism lighting display environment is evaluated. Lighting arrangement. After the arrangement is completed, the illuminance distribution is collected in real time and compared with the rendering generated by the three-dimensional space model to generate an illuminance deviation map. Based on the illuminance deviation map, the optimal lighting layout is optimized and the lighting parameters are adjusted again. The optimized lighting effect is evaluated again through a multi-level scoring standard. If the optimized lighting effect level is greater than the level threshold, the current lighting effect level, layout parameters and lighting parameters obtained by the evaluation are output; otherwise, the optimization is repeated until the preset maximum number of iterations is reached. For example, based on the obtained optimal lighting equipment parameters, 1,000 training lighting effect images are generated through a deep generative adversarial network. These images are added to the lighting effect evaluation model, and the topological structure and parameter information are extracted through a graph neural network. The relationship between lamp nodes and edges is established, and the global representation vector of each layout is obtained through multi-layer graph convolution and pooling. For each rendered image, the texture and semantic features are extracted through a convolutional neural network, and the global representation vector and image features are cascaded and input into a multi-layer fully connected network to obtain the evaluation score of each layout. The evaluation scores of 1,000 layouts are recalculated, and the layout with the highest evaluation score (such as 0.95) is selected as the optimal lighting layout. According to the optimal lighting layout, the lighting layout is arranged in the virtual museum exhibition hall environment. After the arrangement is completed, the illuminance distribution in the exhibition hall is collected in real time and compared with the three-dimensional space model rendering to generate an illuminance deviation map. According to the deviation map, the optimal lighting layout is optimized, the lamp position and lighting parameters are adjusted, and the optimized lighting effect is evaluated using a multi-level scoring standard. The lighting effect level is obtained as "good". Since "good" is not less than the preset level threshold, the current lighting effect level, layout parameters (such as lamp position (2.3m, 3.2m, 4.1m), orientation (28°, 42°, 58°)) and lighting parameters (such as brightness (850lux, 1050lux, 1250lux)) are output.

[0107] In this embodiment, by optimizing the lighting equipment parameters, layout and lighting parameters, the lighting effect is continuously improved until the preset level threshold is reached. This real-time evaluation and optimization closed-loop control method can dynamically adjust the lighting effect to ensure that it always meets the preset quality requirements. By modeling the lighting optimization problem as a black box optimization problem and combining the Gaussian process optimization algorithm, the posterior distribution can be calculated on the observed data set, and the acquisition function is introduced for Bayesian optimization, which can efficiently search for the optimal lighting equipment parameters, such as lamp position, orientation, brightness, etc., which provides a good parameter basis for subsequent lighting layout optimization. The method of integrating the deep generative model and the graph neural network can intelligently evaluate the effects of different lighting layouts, select the optimal layout for actual lighting arrangement, and perform secondary optimization and parameter adjustment on the optimal layout based on the deviation map. The difference between the actual lighting effect and the expected effect can be dynamically corrected, and the rationality and effectiveness of the lighting layout and parameters can be continuously improved. In summary, this embodiment can dynamically adjust the lighting equipment parameters, layout and lighting parameters, continuously improve the lighting effect, and ensure that it meets the preset high quality requirements. Through the intelligent layout evaluation and parameter search method, the optimization efficiency and accuracy are improved, and a dynamic adaptable and high-quality immersive lighting experience is provided for cultural and tourism scenes.

[0108] Figure 2 Schematic diagram of the structure of a computer vision-based cultural and tourism lighting display effect evaluation system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0109] The first unit is used to collect a panoramic image set of the cultural tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules;

[0110] The second unit is used to add the lighting effect characteristic parameters to a preset lighting effect evaluation model, obtain a lighting effect evaluation score sequence by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model, update the lighting effect evaluation model in combination with a policy gradient algorithm, and select key score subsequences in the lighting effect evaluation score sequence for fusion through a Monte Carlo tree search algorithm to obtain a dynamic lighting effect evaluation score, simulate a multi-channel lighting effect image and extract multi-channel lighting characteristic parameters through a ray tracing algorithm, construct a multi-view lighting effect evaluation vector and perform regression prediction to obtain a multi-view lighting effect evaluation score, perform adaptive weighted fusion on the dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score to obtain an overall effect evaluation score, and perform fuzzy comprehensive evaluation in combination with a preset multi-level scoring standard to obtain a lighting effect grade;

[0111] The third unit is used for, if the lighting effect level does not reach a preset level threshold, then, according to a pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the Gaussian process optimization algorithm is combined to solve the optimal lighting equipment parameters and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

[0112] According to a third aspect of the embodiments of the present invention,

[0113] An electronic device is provided, comprising:

[0114] processor;

[0115] a memory for storing processor-executable instructions;

[0116] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0117] A fourth aspect of the embodiments of the present invention is:

[0118] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0119] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computer vision-based cultural and tourism lighting display effect evaluation method, characterized in that: include: Collect a panoramic image set of the cultural and tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules; The lighting effect characteristic parameters are added to a preset lighting effect evaluation model, and a lighting effect evaluation score sequence is obtained by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with a policy gradient algorithm, and a key score subsequence is selected in the lighting effect evaluation score sequence for fusion through a Monte Carlo tree search algorithm to obtain a dynamic lighting effect evaluation score. A multi-channel lighting effect image is simulated and multi-channel lighting characteristic parameters are extracted through a ray tracing algorithm, a multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain a multi-view lighting effect evaluation score, the dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted fused to obtain an overall effect evaluation score, and a fuzzy comprehensive evaluation is performed in combination with a preset multi-level scoring standard to obtain a lighting effect grade; If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the optimal lighting equipment parameters are solved in combination with the Gaussian process optimization algorithm, and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, and the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached; The lighting effect characteristic parameters are added to a preset lighting effect evaluation model, and a lighting effect evaluation score sequence is obtained by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model. The lighting effect evaluation model is updated in combination with a policy gradient algorithm, and key score subsequences are selected and fused in the lighting effect evaluation score sequence through a Monte Carlo tree search algorithm, so as to obtain a dynamic lighting effect evaluation score including: Adding the lighting effect characteristic parameters to a preset lighting effect evaluation model, constructing a virtual cultural and tourism lighting display environment based on the cultural and tourism lighting display environment and the animation engine and rendering it through the three-dimensional space model, interacting the lighting effect evaluation model with the virtual cultural and tourism lighting display environment, generating state data, action data and reward data and adding them to the policy gradient algorithm, generating a lighting effect evaluation score sequence, and updating the parameters of the lighting effect evaluation model based on the lighting effect evaluation score sequence and the policy gradient algorithm; Constructing a Monte Carlo tree according to the lighting effect evaluation score sequence, wherein each node of the Monte Carlo tree represents a lighting parameter configuration, and each path from the root node to the leaf node represents a parameter adjustment sequence. For each node in the Monte Carlo tree, an upper confidence bound algorithm is used to evaluate and generate a node evaluation score, a node with the highest node evaluation score is selected, a child node is randomly generated and a node evaluation score corresponding to each child node is calculated again, and the generation and selection are repeated until a preset maximum number of iterations is reached to obtain a search tree, traverse the search path in the search tree, and the sum of the node evaluation scores corresponding to the nodes on the search path is used as the search path score and sorted in descending order, and the search paths in the top 10% are selected as key score subsequences, and for the selected key score subsequences, the corresponding lighting effect evaluation score is calculated, and weighted fusion is performed in combination with time span and time attenuation to obtain the dynamic lighting effect evaluation score; The upper confidence bound algorithm is used to evaluate and generate the node evaluation score as shown in the following formula: ； in, w g Representation Node g The cumulative reward value, n g Representation Node g The number of visits, α Represents the power of the number of node visits, which is used to adjust the degree of exploration. N Indicates the total number of visits. V g Representation Node g The variance of the value of is used to represent the uncertainty of node rewards; The multi-channel lighting effect image is simulated by the ray tracing algorithm and the multi-channel lighting feature parameters are extracted. The multi-view lighting effect evaluation vector is constructed and regression prediction is performed to obtain the multi-view lighting effect evaluation score. The dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score are adaptively weighted fused to obtain the overall effect evaluation score. The lighting effect level is obtained by fuzzy comprehensive evaluation combined with the preset multi-level scoring criteria, including: Select representative viewpoints in a virtual environment, render at each viewpoint through a ray tracing algorithm, generate a lighting effect image, extract multiple lighting feature channels for the lighting effect image, calculate features on each lighting feature channel through an image pyramid algorithm, obtain lighting information of different granularities, combine to obtain multi-channel lighting feature parameters, extract the multi-channel lighting feature parameters from each viewpoint and pre-acquired spatiotemporal statistical indicators, construct them into a multi-view lighting effect evaluation vector according to a pre-set splicing method, select a Gaussian process regression model, train with the lighting effect score scored manually as a training label, perform regression prediction through the trained Gaussian process regression model, and obtain the multi-view lighting effect evaluation score; The user interaction behavior data is obtained and the user's preference for different lighting effects is determined through a collaborative filtering algorithm to generate user preferences. The weight corresponding to the dynamic lighting effect evaluation score and the weight corresponding to the multi-view lighting effect evaluation score are determined through a weight generation model in combination with the cultural background and historical information corresponding to the cultural tourism lighting display environment. The overall effect evaluation score is obtained by weighted fusion. According to a pre-set multi-level scoring standard, the overall effect evaluation score is added to the membership function corresponding to each level, the membership vector is calculated and synthesized into a comprehensive membership through a fuzzy comprehensive evaluation method, and the lighting effect level corresponding to the current lighting effect characteristic parameter is output based on the maximum membership principle and the comprehensive membership.

2. The method according to claim 1, characterized in that Collect panoramic image sets of the cultural and tourism lighting display environment and local close-up image sets corresponding to key cultural relics, perform semantic segmentation on the panoramic image sets through a multi-scale convolutional neural network, obtain key cultural relic areas, visit path areas and background areas in each panoramic image, perform texture recognition and material recognition on the local close-up image sets through a deep convolutional neural network based on transfer learning, obtain texture features and material features of key cultural relics, construct a three-dimensional space model corresponding to the cultural and tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules, including: A panoramic image set of the cultural tourism lighting display environment is collected through a pre-set camera, and a close-up image set of the surface of key cultural relics is obtained by using a macro lens to obtain a local close-up image set corresponding to the current key cultural relics; For the panoramic image set, a cultural tourism lighting display environment training set is constructed and the images in the training set are manually annotated, a multi-scale convolutional neural network based on an attention mechanism is trained using the manually annotated training set images, the parameters of the multi-scale convolutional neural network are adjusted, and semantic segmentation is performed on each image in the panoramic image set. In each panoramic image, the semantic label corresponding to each pixel is determined and the key cultural relics area, the tour path area, and the background area are identified; Collect and organize image data of cultural relics texture and material and perform fine annotation, generate annotated close-up image set and expand it through data enhancement technology, select a transfer learning model, modify the top-level structure and adjust the loss function of the selected transfer learning model based on the characteristics and categories of cultural relics, use the expanded annotated close-up image set to train the adjusted transfer learning model, obtain pre-training parameters and migrate them to the convolution layer of the deep convolutional neural network, the convolution layer uses the image in the local close-up image set as input, combines the compact bilinear pooling layer to perform high-order bilinear encoding, and obtains the texture features and material features corresponding to the key cultural relics; The panoramic image set obtained by the motion reconstruction structure algorithm is sparsely reconstructed to obtain the sparse point cloud and camera pose of the display environment. The multi-view stereo reconstruction algorithm is combined with the semantic segmentation results of the panoramic image set to perform dense matching and three-dimensional reconstruction to obtain a dense point cloud model of the display environment. Multi-perspective depth images are generated and registered in combination with deep learning. A triangular mesh surface model of the display environment is generated by the Poisson surface reconstruction algorithm. Texture mapping is performed based on the panoramic image and the local close-up image to obtain a three-dimensional space model corresponding to the cultural and tourism lighting display environment. Based on the texture features and the material features, a normal map and a displacement map are added to the three-dimensional space model. The environment is rendered by a physically based rendering technology, and the panoramic image set and the local close-up image set are mapped to the three-dimensional space model. The three-dimensional space model is uniformly sampled by the Poisson disk sampling algorithm to generate uniformly distributed point cloud data. Each sampling point is semantically annotated in combination with the semantic segmentation information. Lighting effect feature parameters corresponding to multiple dimensions are extracted according to pre-set lighting effect evaluation rules, and the lighting effect feature parameters are stored in a point cloud data structure.

3. The method according to claim 2, characterized in that The objective function of sparse 3D reconstruction of the acquired panoramic image set by the motion reconstruction structure algorithm is as follows: ； in, G represents the objective function value, R i Indicates i The rotation matrix of the camera, t i Indicates i The camera's translation vector, X j Indicates j The coordinates of the three-dimensional points, x ij Indicates j The three-dimensional point in i The projected coordinate points in the camera, v ij Is a visibility marker, used to indicate the j Is the 3D point in i Within the camera field of view, d () is the Euclidean distance function, which is used to represent the reprojection error of the point. σ Represents the scale parameter.

4. The method according to claim 1, characterized in that If the lighting effect level does not reach the preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the optimal lighting equipment parameters are solved in combination with the Gaussian process optimization algorithm, and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural tourism lighting display environment is arranged based on the optimal lighting layout, and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, and the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached, including: Acquire the lighting effect level obtained by judgment. If the lighting effect level does not reach a preset level threshold, then according to the pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting device control module, and the lighting optimization problem is modeled as a black box optimization problem in combination with the Gaussian process optimization algorithm. The observation data set is acquired and the lighting device parameters and the corresponding lighting effect evaluation function are defined. It is assumed that the function value of the lighting effect evaluation function obeys a multivariate Gaussian distribution. The posterior distribution is calculated for the observation points in the observation data set and the acquisition function is introduced through Bayesian optimization, and the optimal parameters are searched to obtain the optimal lighting device parameters. Based on the optimal lighting equipment parameters, randomly generate training lighting effect images through a preset deep generative adversarial network, add the training lighting effect images to the lighting effect evaluation model, extract topological structure and parameter information through a graph neural network, model each lamp as a node of the graph, and model the relative position and orientation relationship between the lamps as the edge of the graph, obtain a global representation vector corresponding to the current layout through multi-layer graph convolution and pooling, extract texture and semantic features for the rendered image through a convolutional neural network, generate image features, cascade the global representation vector and the image features and add them to a multi-layer fully connected network to obtain a layout evaluation score corresponding to the current layout, repeatedly calculate the layout evaluation scores corresponding to different training lighting effect images, and select the global representation vector with the highest layout evaluation score as the optimal lighting layout; Based on the optimal lighting layout, lighting arrangement is performed for the cultural and tourism lighting display environment. After the arrangement is completed, the illuminance distribution is collected in real time and compared with the rendering generated by the three-dimensional space model to generate the illuminance deviation map. Based on the illuminance deviation map, the optimal lighting layout is again optimized and the lighting parameters are adjusted. The optimized lighting effect is again evaluated through a multi-level scoring standard. If the lighting effect level is greater than the level threshold, the current lighting effect level, layout parameters and lighting parameters obtained by evaluation are output. Otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

5. A computer vision-based cultural and tourism lighting display effect evaluation system, used to implement the method described in any one of claims 1 to 4, characterized in that: include: The first unit is used to collect a panoramic image set of the cultural tourism lighting display environment and a local close-up image set corresponding to key cultural relics, perform semantic segmentation on the panoramic image set through a multi-scale convolutional neural network, obtain the key cultural relic area, the tour path area and the background area in each panoramic image, perform texture recognition and material recognition on the local close-up image set through a deep convolutional neural network based on transfer learning, obtain the texture features and material features of the key cultural relics, construct a three-dimensional space model corresponding to the cultural tourism lighting display environment and map the panoramic image set and the local close-up image set to the three-dimensional space model, generate point cloud data and extract lighting effect feature parameters in combination with pre-set lighting effect evaluation rules; The second unit is used to add the lighting effect characteristic parameters to a preset lighting effect evaluation model, obtain a lighting effect evaluation score sequence by interacting with the virtual cultural tourism lighting display environment rendered by the three-dimensional space model, update the lighting effect evaluation model in combination with a policy gradient algorithm, and select key score subsequences in the lighting effect evaluation score sequence for fusion through a Monte Carlo tree search algorithm to obtain a dynamic lighting effect evaluation score, simulate a multi-channel lighting effect image and extract multi-channel lighting characteristic parameters through a ray tracing algorithm, construct a multi-view lighting effect evaluation vector and perform regression prediction to obtain a multi-view lighting effect evaluation score, perform adaptive weighted fusion on the dynamic lighting effect evaluation score and the multi-view lighting effect evaluation score to obtain an overall effect evaluation score, and perform fuzzy comprehensive evaluation in combination with a preset multi-level scoring standard to obtain a lighting effect grade; The third unit is used for, if the lighting effect level does not reach a preset level threshold, then, according to a pre-constructed online lighting optimization model, real-time communication is performed through the three-dimensional space model and the lighting equipment control module, the Gaussian process optimization algorithm is combined to solve the optimal lighting equipment parameters and a training lighting effect image is generated through a deep generative adversarial network, the training lighting effect image is added to the lighting effect evaluation model to generate a layout evaluation score corresponding to each lighting layout, the optimal lighting layout is screened according to the layout evaluation score, the cultural and tourism lighting display environment is arranged based on the optimal lighting layout and the illuminance distribution is collected in real time, and the illuminance deviation map is generated by comparison with the rendering, and the layout optimization and parameter adjustment are performed again in combination with the illuminance deviation map, the adjusted lighting effect is evaluated again, and if the level threshold is reached, the evaluation level and the current layout are output, otherwise, the optimization is repeated until the preset maximum number of iterations is reached.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images