Virtual exhibition hall generation method and device based on arc-shaped screen

Abstract

The invention relates to the technical field of virtual exhibition halls, and discloses a virtual exhibition hall generation method and device based on an arc-shaped screen, and the method comprises the steps: projecting subtitle contents inputted in a virtual exhibition hall in advance to the surface of a display screen through an arc-shaped LED display screen, and obtaining a subtitle display region mapping matrix and an initial subtitle image; performing feature extraction and adaptive fusion operation on the display area to obtain a target mapping matrix and a hierarchical feature image set; executing LED module interaction analysis on the target mapping matrix and the hierarchical feature image set to obtain a display parameter matrix; inputting the display parameter matrix into a deep residual teacher network and a lightweight student network for multi-angle display consistency loss training to obtain a subtitle display model; the subtitle position and the display brightness are jointly predicted to obtain the target display parameter set, and the target subtitle display data are output, so that the subtitle display refinement degree of the virtual exhibition hall is effectively improved, and the display effect consistency under different visual angles is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of virtual exhibition halls, and in particular to a method and device for generating a virtual exhibition hall based on a curved screen. Background Art

[0002] The curved LED display screen, due to its unique spatial curved surface characteristics, can bring an immersive visual experience to the audience and play an important role in the virtual exhibition hall. However, when displaying subtitles on a curved LED display screen, the curvature of the display screen surface changes, causing the subtitle display effect to be distorted as the viewing angle changes, affecting the exhibition experience.

[0003] Existing subtitle display technologies are mainly designed for flat display screens and are difficult to directly apply to curved display scenarios. Traditional methods often use simple geometric transformations for subtitle projection, which cannot effectively deal with the visual distortion caused by curved surfaces. At the same time, due to the modular design of curved LED screens, the display effects between adjacent LED modules are often discontinuous, resulting in visual defects such as breaks and jagged edges in subtitle display. Summary of the invention

[0004] The present application provides a method and device for generating a virtual exhibition hall based on a curved screen, thereby effectively improving the refinement of the subtitle display in the virtual exhibition hall and ensuring the consistency of the display effects under different viewing angles.

[0005] The first aspect of the present application provides a method for generating a virtual exhibition hall based on a curved screen, and the method for generating a virtual exhibition hall based on a curved screen includes: The subtitle content pre-entered in the virtual exhibition hall is projected onto the display surface through the curved LED display screen to obtain the subtitle display area mapping matrix and the initial subtitle image; Based on the subtitle display area mapping matrix and the initial subtitle image, feature extraction and adaptive fusion operation are performed on the display area to obtain a target mapping matrix and a hierarchical feature atlas; Performing LED module interaction analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; Inputting the display parameter matrix into the deep residual teacher network and the lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; The subtitle display model is used to jointly predict the subtitle position and display brightness to obtain a target display parameter set, and according to the target display parameter set, the subtitle edge is subjected to gradient domain fusion and temporal smoothing to output target subtitle display data.

[0006] The second aspect of the present application provides a virtual exhibition hall generation device based on a curved screen, and the virtual exhibition hall generation device based on a curved screen includes: The projection module is used to project the subtitle content pre-input in the virtual exhibition hall onto the surface of the display screen through the curved LED display screen, and obtain the subtitle display area mapping matrix and the initial subtitle image; A feature extraction module, used to perform feature extraction and adaptive fusion operations on the display area based on the subtitle display area mapping matrix and the initial subtitle image, to obtain a target mapping matrix and a hierarchical feature atlas; An interactive analysis module, used for performing LED module interactive analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; A training module, used for inputting the display parameter matrix into a deep residual teacher network and a lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; The output module is used to jointly predict the subtitle position and display brightness using the subtitle display model to obtain a target display parameter set, and to perform gradient domain fusion and temporal smoothing on the subtitle edge according to the target display parameter set to output target subtitle display data.

[0007] Compared with the prior art, the present application has the following beneficial effects: by establishing an accurate three-dimensional spatial coordinate system and a multi-angle scanning mechanism, accurate modeling of the surface characteristics of the curved LED display screen is achieved, a multi-scale feature hierarchical network is used for display area feature extraction and fusion, and the self-attention mechanism and feature pyramid integration are combined to effectively improve the refinement of subtitle display. A graph convolutional network is introduced for LED module interaction analysis, and the problem of discontinuous display between adjacent LED modules is solved by constructing a connection relationship matrix between modules. A collaborative training framework of a deep residual teacher network and a lightweight student network is designed, and the consistency of display effects under different viewing angles is ensured through knowledge distillation and multi-angle consistency loss constraints. The joint optimization of subtitle position and display brightness is achieved, and the rationality of subtitle layout and uniformity of display brightness are ensured through the synergy of dual-path prediction branches. A gradient domain fusion scheme based on the Poisson equation is proposed, which is combined with ambient light compensation and timing smoothing to effectively eliminate the problems of subtitle edge display mutation and timing jitter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0009] The structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0010] Figure 1 is a flow chart of a method for generating a virtual exhibition hall based on a curved screen provided by an embodiment of the present invention; Figure 2 It is a schematic block diagram of the structure of a virtual exhibition hall generating device based on a curved screen provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0013] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0014] It should be further understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. Figure 1 In the embodiment of the present application, an embodiment of a method for generating a virtual exhibition hall based on a curved screen includes: Step 100: projecting the subtitle content pre-input in the virtual exhibition hall onto the surface of the display screen through the curved LED display screen to obtain a subtitle display area mapping matrix and an initial subtitle image; It is understandable that the execution subject of the present application can be a virtual exhibition hall generation device based on a curved screen, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0015] Specifically, the surface of the curved LED display screen is divided into a plurality of grid points by partitioning the display screen surface. Each grid point is assigned a unique identifier to form a grid point identification matrix. The grid points are scanned at multiple angles using a laser rangefinder. During the scanning process, the laser rangefinder emits a laser beam according to a preset angle range, and measures the distance between the grid point and the rangefinder according to the time delay of the reflected signal to obtain accurate distance data. These distance data are organized into a grid point distance matrix, which corresponds to the grid point identification matrix one by one. Through multi-angle scanning, grid point data under different viewing angles are obtained. Based on the grid point identification matrix and the distance matrix, the specific coordinate value of each grid point in the three-dimensional space is calculated to establish a surface point cloud data set of the curved LED display screen. A surface fitting operation is performed on the surface point cloud data set, and a display surface equation that can describe the surface shape of the display screen is generated using a mathematical interpolation method or a least squares fitting method. The subtitle content pre-entered in the virtual exhibition hall is converted into a font outline vector. The font outline vector is a vectorized description of the subtitle shape, which can adapt to different resolutions and deformation requirements. According to the display screen surface equation, the font outline is projected on the surface of the display screen to determine the projection position of the subtitle on the curved display screen and generate the subtitle outline projection data. The subtitle outline projection data is rasterized to map the continuous projection outline to the discrete LED module position, and a subtitle display area mapping matrix is ​​generated to describe the correspondence between each LED module and the projection outline, so that the subtitle content can be accurately displayed on the screen. Font anti-aliasing processing is performed based on the subtitle outline projection data, and techniques such as bilinear interpolation or sub-pixel rendering are used to eliminate the jagged effect of the font edge and improve the smoothness of the display. At the same time, rasterization processing is performed to convert the vectorized font outline into a high-resolution font bitmap. Resolution adaptive transformation is performed on the high-resolution font bitmap to scale the font bitmap to the actual resolution of the display to ensure that the final image fully matches the hardware resolution of the display and generate the initial subtitle image.

[0016] Step 200: Based on the subtitle display area mapping matrix and the initial subtitle image, feature extraction and adaptive fusion operations are performed on the display area to obtain a target mapping matrix and a hierarchical feature atlas; Specifically, the subtitle display area mapping matrix and the initial subtitle image are input into a multi-scale feature hierarchical network for multi-scale decomposition. The multi-scale hierarchical network splits the input data into n feature layers of different scales through hierarchical processing. Each feature layer corresponds to a specific resolution and spatial perception characteristics. These feature layers together constitute a multi-dimensional description of the subtitle display area. Convolution operations and local feature extraction are performed on the n feature layers of different scales to generate n groups of original feature maps. The convolution operation can effectively capture the complex edges, textures and other significant regional characteristics of the subtitle display area by extracting local spatial information and semantic features. At the same time, local feature extraction can refine the feature representation so that each original feature map has rich spatial hierarchical information and semantic hierarchical information. A feature association graph is constructed based on the n groups of original feature maps to characterize the relationship between each feature layer. In the feature association graph, each feature node represents an original feature map, and the edges between nodes represent the correlation of these feature maps in a specific semantic or spatial dimension. By calculating the similarity weights between feature nodes, a feature association matrix is ​​generated, which reflects the global similarity and correlation between different feature maps. The feature correlation matrix is ​​used to perform weighted integration of n groups of original feature maps, and the feature areas with strong correlation are fused to generate a fused feature map. The self-attention operation is performed on the fused feature map to obtain the attention weight matrix. The self-attention operation identifies the salient areas in the subtitle display area that are critical to the display effect by calculating the attention weight matrix in the global range. The fused feature map is spatially reweighted based on the attention weight matrix to further enhance the high-weight area and suppress the influence of the low-weight area to generate a target mapping matrix. The fused feature map and the attention weight matrix are input into the deconvolution network for feature upsampling and reconstruction. The deconvolution network gradually restores the fused feature map from low resolution to high resolution representation through inverse convolution operation, and dynamically adjusts the reconstruction process using the attention weight matrix to generate feature representations of different resolutions. The feature representations of different resolutions are integrated through feature pyramid to generate a hierarchical feature atlas. The feature pyramid integration process can strike a balance between high-resolution detail expression and low-resolution global semantic expression, forming a multi-level and multi-scale feature representation system.

[0017] Step 300, performing LED module interaction analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; It should be noted that the target mapping matrix is ​​segmented into regions. By analyzing the geometry and display characteristics of the target mapping matrix, the entire display area is divided into multiple LED display modules with independent display capabilities, and a module partition matrix is ​​generated to clarify the display area range of each LED module and its positional relationship in the global mapping. A display module adjacency graph is constructed based on the module partition matrix, in which each LED display module is regarded as a node in the graph. In order to characterize the spatial relationship between modules, the spatial distance between each pair of module nodes is calculated by the Euclidean distance of the geometric position of the nodes in the mapping matrix to generate a spatial adjacency matrix. The adjacency matrix quantifies the spatial distribution and adjacency relationship between modules in numerical form. Regional aggregation operations are performed on each feature map in the hierarchical feature map set, and the global features are combined with the module characteristics. The feature regions corresponding to each LED display module are extracted and aggregated to form a module feature matrix, which contains multidimensional feature information of each module, including features such as brightness, contrast, and relationship with surrounding modules. Combined with the spatial adjacency matrix, the module features are messaged and updated. Through graph neural networks or similar graph processing algorithms, the spatial relationship between modules captured in the spatial adjacency matrix is ​​introduced into the module feature update process to generate a node state matrix to describe the semantic and spatial meaning of each module's features in the global context. Edge attention operations are performed on the node state matrix. By dynamically adjusting the weights of the connection edges between modules, the module connection relationships that have the greatest impact on the display effect are identified, and the module connection relationship matrix is ​​generated. The module connection relationship matrix is ​​used to reflect the display coordination requirements between different modules. For example, adjacent modules on a curved screen need to adjust the brightness or contrast synchronously to avoid visual discontinuity. After obtaining the module connection relationship matrix, the brightness and contrast of the LED display module are collaboratively optimized in combination with the node state matrix and the module feature matrix to make the overall visual effect of the display area more uniform and natural. After the optimization is completed, the entire display area is densely sampled using the module control parameters, and a display unit parameter distribution map is generated by performing more refined parameter sampling and distribution fitting on the display units inside each module. The distribution map refines the display parameters such as brightness and contrast of each display unit in the module, so that the overall display effect achieves higher accuracy. The display unit parameter distribution map is processed with boundary constraints to ensure that the connection area between modules complies with physical and visual rules in display transition. Through boundary constraints, the visual sense of fragmentation caused by the uneven transition between modules is effectively reduced, and the display parameter matrix is ​​finally output, which contains the optimized parameters of all modules and display units.

[0018] Step 400: Input the display parameter matrix into the deep residual teacher network and the lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; Specifically, the display parameter matrix is ​​input into the deep residual teacher network. The deep residual teacher network performs multi-level feature transformation on the input display parameters. It extracts the spatial characteristics, brightness characteristics and contrast characteristics in the display parameters layer by layer through the residual block, and generates a high-dimensional teacher network feature map. At the same time, in order to enhance the generalization ability of the training, data enhancement operations are performed on the display parameter matrix. The data enhancement methods include perspective change, brightness perturbation, contrast adjustment, etc., so as to generate a multi-view training data set for simulating the performance of subtitle content under different viewing angles and display conditions of the curved screen. The multi-view training data set is input into the lightweight student network for feature extraction and transformation. The lightweight student network realizes efficient processing of multi-view training data through a simplified convolution structure. The generated student network feature map is lighter than the feature map of the teacher network, but still retains the key features required for subtitle display. In this process, feature alignment operations are performed, and the knowledge distillation loss is calculated by comparing the similarity between the teacher network feature map and the student network feature map. This loss value quantifies the degree of closeness of the student network to the learning results of the teacher network, ensuring that the student network can reproduce the display effect of the teacher network at a lower computational complexity. At the same time, based on the multi-view training data set, the display effect differences under different view angles are calculated, and the multi-view display consistency loss value is constructed. This loss value quantifies the difference in display under different view angles by analyzing the brightness uniformity, contrast consistency and subtitle edge smoothness of the display effect of each view angle, and provides an optimization target for network training. In order to comprehensively consider the knowledge distillation loss and the multi-view display consistency loss, the two are weighted and combined to form an overall loss function. The lightweight student network is back-propagated according to the overall loss function. Back-propagation updates the network parameters by gradient layer by layer, so that the student network can gradually optimize its feature extraction ability and reduce the knowledge distillation loss and display consistency loss. After multiple rounds of training, an optimized lightweight student network is obtained. The optimized student network is pruned and quantized. The pruning operation reduces the model complexity by removing redundant neurons and connections in the network, while the quantization process reduces the model size and computational overhead by reducing the precision of parameter representation (such as from 32-bit floating point to 8-bit integer). Generate a subtitle display model.

[0019] Step 500: Use the subtitle display model to jointly predict the subtitle position and display brightness to obtain a target display parameter set, and perform gradient domain fusion and temporal smoothing on the subtitle edge according to the target display parameter set to output target subtitle display data.

[0020] Specifically, the subtitle display model is set to the inference mode, and the model generates the initial prediction results by inputting the current subtitle content and display area information. After decoding and analysis, these prediction results are separated into the output data of the position prediction branch and the brightness prediction branch to obtain the dual-path prediction features. The output data of the position prediction branch mainly describes the geometric layout characteristics of the subtitles, while the brightness prediction branch provides an estimate of the brightness distribution of each display unit. The position prediction branch in the dual-path prediction features is subjected to coordinate transformation, and the subtitle position predicted by the model is mapped to the physical coordinate system of the display screen. The coordinate transformation process needs to consider the geometric characteristics of the curved screen and the display area defined by the target mapping matrix, so that the generated subtitle layout coordinates can accurately adapt to the actual display environment of the curved screen. At the same time, for the output data of the brightness prediction branch, the brightness distribution of the display unit is calculated by performing a brightness normalization operation. The purpose of the brightness normalization operation is to adjust the brightness value range to match the physical brightness range of the display screen and generate a brightness control sequence. The brightness control sequence is the distributed regulation result of the brightness of each display unit. The position-brightness coupling coefficient is calculated according to the subtitle layout coordinates and the brightness control sequence. By calculating the position-brightness coupling coefficient, the mutual influence between the subtitle position and the brightness distribution is quantified. The coupling relationship can reflect the compensation effect of brightness on the subtitle layout and the constraint effect of the subtitle position on the brightness distribution. The coupling coefficient is used to generate the fusion weight, and the subtitle layout coordinates and the brightness control sequence are weighted and combined according to these weights to generate the target display parameter set. According to the target display parameter set, the ambient light compensation operation is performed, and the subtitle edge is subjected to gradient domain fusion and temporal smoothing. In the gradient domain fusion process, the gradient distribution of the subtitle edge area is calculated, and the continuity of the edge transition area is optimized by the fusion algorithm to eliminate the jagged or broken problems on the subtitle edge. Gradient domain fusion makes the subtitle edge transition more natural and smooth by adjusting the brightness gradient of adjacent pixels, enhancing the overall visual perception. At the same time, in order to cope with the movement or change of subtitles in dynamic display scenes, temporal smoothing is performed. Temporal smoothing effectively reduces the jitter and jump problems when the subtitle is updated by filtering the changes in the target display parameter set in the time dimension, and ensures the display stability of the subtitle in a dynamic environment. Combined with the influence of ambient light, the ambient light compensation operation is performed on the target display parameter set. The ambient light compensation operation collects the lighting conditions of the current display environment and adjusts the brightness distribution in the target display parameter set in real time, so that the display effect of subtitles can adapt to the viewing needs under different lighting conditions. For example, the contrast and brightness of subtitles are increased in strong light environments, while the brightness is reduced in weak light environments to avoid glare. After ambient light compensation, the target subtitle display data has higher adaptability and visual comfort.

[0021] According to the target display parameter set, the local gradient of the subtitle edge area is calculated to generate a gradient value matrix and extract edge contour data. The gradient value matrix reflects the change amplitude of the subtitle edge pixel brightness, while the edge contour data provides the geometric structure information of the subtitle edge. The Poisson boundary equation is constructed based on the gradient value matrix and the edge contour data. The Poisson equation takes the gradient field as input and combines the boundary conditions to generate the edge fusion function through numerical solution. This function is used to describe the pixel-level transition relationship between the subtitle edge and the background area. Using the edge fusion function, the subtitle edge area and the background area are subjected to pixel-level gradient processing, and the jagged and broken phenomena of the edge are eliminated by smoothing the gradient to generate the edge transition area data. On this basis, according to the edge transition area data and the distribution characteristics of the display area, the regional fusion coefficient is calculated to generate the regional weight matrix. The regional weight matrix is ​​used to quantify the contribution ratio of different display areas to the overall display effect. In order to enhance the visibility of subtitle display under different lighting conditions, the ambient light of the display area is collected at multiple points and the spatial distribution is analyzed. The ambient light sensor collects the light intensity data at multiple locations in the display area, and generates the ambient light intensity map based on these data. The ambient light intensity map describes the lighting conditions of the display screen in the form of spatial distribution, which can reflect the changes in light intensity. According to the ambient light intensity map, block brightness compensation is performed on the display area, and the influence of ambient light is balanced by adjusting the brightness of each display block to generate a light response matrix. The light response matrix ensures that the subtitle display will not be too dark in a strong light environment, and will not be too bright in a weak light condition, thereby improving the adaptability of the overall display effect. The regional weight matrix and the light response matrix are jointly optimized to construct a display compensation model. The display compensation model generates regional compensation parameters by integrating the results of regional weights and light response, quantifies the brightness adjustment requirements of each display unit under specific ambient light conditions, and ensures that the subtitle display can adapt to complex lighting environments. These compensation parameters are used to adaptively adjust the brightness of the subtitle display area and generate compensation display data, thereby achieving dynamic brightness balance and display quality improvement on a global scale. The compensated display data is reconstructed into a frame sequence according to the display timing to construct a dynamic display sequence. The frame sequence reconstruction reorganizes the compensation data in the time dimension to ensure that the content and brightness of each frame transition naturally during the dynamic display of subtitles. In order to eliminate the inter-frame jitter problem in the dynamic display sequence, a time domain filtering operation is performed on the frame sequence. The time domain filtering generates smoothed sequence data by smoothing the inter-frame data and suppressing the sudden change of pixel values. The smoothed sequence data is input into the display buffer for frame synchronization control. Frame synchronization control ensures that the subtitle display data can be loaded and output at a stable rate by coordinating the refresh timing and synchronization signal of the display screen. In this process, the frame data is reorganized according to the hardware characteristics of the display screen to adapt it to the pixel distribution and refresh characteristics of the display screen, and finally the target subtitle display data is generated.

[0022] In the embodiment of the present application, by establishing an accurate three-dimensional spatial coordinate system and a multi-angle scanning mechanism, accurate modeling of the surface characteristics of the curved LED display screen is achieved, and a multi-scale feature hierarchical network is used to extract and fuse display area features. Combined with the self-attention mechanism and feature pyramid integration, the refinement of the subtitle display is effectively improved. A graph convolutional network is introduced to analyze the interaction of LED modules, and the problem of discontinuous display between adjacent LED modules is solved by constructing a connection relationship matrix between modules. A collaborative training framework of a deep residual teacher network and a lightweight student network is designed, and the consistency of display effects under different viewing angles is guaranteed through knowledge distillation and multi-angle consistency loss constraints. The joint optimization of subtitle position and display brightness is achieved, and the rationality of subtitle layout and uniformity of display brightness are ensured through the synergy of dual-path prediction branches. A gradient domain fusion scheme based on the Poisson equation is proposed, which is combined with ambient light compensation and timing smoothing to effectively eliminate the problems of subtitle edge display mutations and timing jitter.

[0023] In a specific embodiment, the process of executing step 100 may specifically include the following steps: The surface of the curved LED display screen is divided into a plurality of grid points, and a unique identifier is assigned to each grid point to obtain a grid point identification matrix; The laser rangefinder is used to perform multi-angle scanning and distance data analysis on the grid points within a preset angle range to obtain a grid point distance matrix; Based on the grid point identification matrix and the grid point distance matrix, the three-dimensional space coordinate value of each grid point is calculated, the surface point cloud data set of the arc-shaped LED display screen is established, and the surface fitting operation is performed on the surface point cloud data set to obtain the surface equation of the display screen; The subtitle content pre-entered in the virtual exhibition hall is converted into a font outline vector, and the projection position of the font outline on the surface is calculated according to the display surface equation to obtain the subtitle outline projection data. The subtitle outline projection data is rasterized, and the continuous projection outline is mapped to the discrete LED module position to obtain the subtitle display area mapping matrix. Font anti-aliasing and rasterization processing are performed based on the subtitle outline projection data to obtain a high-resolution font bitmap, and resolution adaptive transformation is performed on the high-resolution font bitmap to scale the font bitmap to the display resolution to obtain an initial subtitle image.

[0024] Specifically, the surface of the curved LED display is divided into a plurality of grid points. Based on the geometric shape of the display, it is divided into a regular two-dimensional grid, where each grid point represents an independent physical location. Assuming the width of the display is and height , the number of rows of grid division is , the number of columns is , then the total number of grid points is Each grid point is numbered by row and column Unique identification, forming a grid point identification matrix ,in and Respectively represent Row and The number of the column, satisfying , For example, if , then the grid point identification matrix is ​​a The two-dimensional matrix is ​​numbered from (1,1) to (10,20). The laser rangefinder is used to perform multi-angle scanning and distance data analysis on the grid points within the preset angle range to obtain the distance data of the grid points. Assume that the laser rangefinder is located at the reference point , for the grid points The scanning angle is ,in is the horizontal angle, is a vertical angle, satisfying ,in is the number of scanning angles. According to the distance measurement formula, the distance measured by the laser rangefinder is satisfy: ; Through multi-angle scanning, the distance data matrix of each grid point is obtained , where each element Indicates The distance measured by the scan. Based on the grid point identification matrix and the distance matrix, the three-dimensional spatial coordinates of each grid point are calculated through trigonometric relationships. Assume that the unit vector of the laser direction is , then the three-dimensional coordinates of the grid points It is expressed as: ; ; ; The average value of all scanning angles of each grid point is taken to reduce the measurement error and obtain the point cloud coordinates. The three-dimensional coordinates of all grid points constitute the point cloud data set of the display surface. Perform surface fitting operations on the point cloud data set to establish the surface equation of the display screen. Assuming that the surface of the display screen is represented by a cubic polynomial, its surface equation is: ; in, The surface parameters are determined by the least squares fitting method. As input, solve the error After obtaining the display surface equation, the subtitle content pre-entered in the virtual exhibition hall is converted into a font outline vector. Assuming that the subtitle content is a text string, it is represented as vectorized outline data through the font generation tool. , where each point is the outline coordinate of the font. According to the surface equation , project the font outline onto the display surface to generate three-dimensional outline data: ; In order to adapt to the discrete structure of the LED display module, the projected subtitle contour data is rasterized to convert the continuous three-dimensional contour into Mapped to discrete LED module positions. Let the center coordinates of each LED module be The rasterization process is implemented by finding the nearest neighbor module to generate the subtitle display area mapping matrix. The subtitle outline projection data is subjected to font anti-aliasing and rasterization to generate a high-resolution font bitmap. The bilinear interpolation algorithm is used to reduce the aliasing effect and ensure the smoothness of the font edge. The high-resolution font bitmap is scaled to the actual resolution of the display screen through resolution adaptive transformation. , and finally get the initial subtitle image.

[0025] In a specific embodiment, the process of executing step 200 may specifically include the following steps: Input the subtitle display area mapping matrix and the initial subtitle image into a multi-scale feature hierarchical network for multi-scale decomposition to obtain n feature layers of different scales, and perform convolution operations and local feature extraction on the n feature layers of different scales respectively to obtain n groups of original feature maps; Construct a feature association graph based on n groups of original feature graphs, calculate the similarity weights between feature nodes, obtain a feature association matrix, and use the feature association matrix to perform weighted integration on the n groups of original feature graphs to obtain a fused feature graph; Perform self-attention operation on the fused feature map to obtain the attention weight matrix, and perform spatial reweighting on the fused feature map based on the attention weight matrix to obtain the target mapping matrix; The fused feature map and the attention weight matrix are input into the deconvolution network for feature upsampling and reconstruction to obtain feature representations of different resolutions. The feature representations of different resolutions are then integrated into feature pyramids to obtain a hierarchical feature atlas.

[0026] Specifically, the subtitle display area mapping matrix and the initial subtitle image Input the multi-scale feature hierarchical network for multi-scale decomposition. Assume that the input mapping matrix and image size are , that is, the height is , width is , the multi-scale feature hierarchical network reduces the size of the input data layer by layer through downsampling operations. Suppose the decomposed The size of the layer feature map is , then the number of scales of decomposition Determines the minimum resolution of the feature map, for example, , the size of the last layer feature map is These multi-scale feature layers capture spatial information and semantic features at different resolutions. After decomposition, convolution operations and local feature extraction are performed on each feature layer to generate A set of original feature maps. The layer feature map is , the features obtained after convolution operation are: ; Among them, Conv represents the convolution operation, It is The convolution operation extracts local spatial information, such as the gradient of the subtitle edge and the regional brightness distribution, while enhancing the feature expression capability. For each set of feature maps , extract salient features through pooling operations, such as the contrast characteristics between the subtitle outline and the background area. After the original feature graphs are assembled, the relationship between these features is analyzed by constructing a feature association graph. The feature association graph uses each layer of feature graph as a node and uses edges to describe their similarities. Layer and The similarity of the layer feature map is , then the similarity is calculated by cosine similarity or correlation: ; in, represents the inner product of two feature maps, Represents the norm of the feature graph. By calculating the similarity between all feature graphs, the feature association matrix is ​​constructed. , where each element Indicates Layer and The feature correlation strength of the layer. When the original feature maps of the group are weighted integrated, the integrated feature maps Calculated by the following formula: ; in, is the weight of the feature association matrix, which indicates the weighted ratio between feature maps. The weighted integration eliminates redundant information in multi-scale features and enhances the cross-scale feature expression capability. The self-attention operation is performed on the fused feature map to highlight the key area features. The self-attention mechanism calculates the global relationship matrix of the fused feature map. and , and the corresponding feature weight matrix , get the attention weight matrix : ; in, They are the query, key and value of the fused feature graph, is the normalization coefficient of the feature dimension. Attention weight matrix Fusion feature map Perform spatial reweighting to generate the target mapping matrix: ; The target mapping matrix further improves the ability to accurately express features in space, making the display effect of subtitles more prominent. The fused feature map and attention weight matrix are input into the deconvolution network for feature upsampling and reconstruction to generate feature representations of different resolutions. The deconvolution network gradually enlarges the feature map through the inverse convolution operation, and the generated first The layer feature map size is: ; in and is the height and width of the minimum resolution layer. The upsampled multi-resolution feature maps are fused through feature pyramid integration. The integration formula is: ; in, is the weight of each layer of feature map, It is The hierarchical feature atlas generated by the feature pyramid integration can retain both high-resolution details and low-resolution semantic features.

[0027] In a specific embodiment, the process of executing step 300 may specifically include the following steps: Performing regional segmentation on the target mapping matrix, dividing the display area into multiple LED display modules to obtain a module partition matrix, and constructing a display module adjacency graph based on the module partition matrix, taking each LED display module as a graph node, calculating the spatial distance between nodes, and obtaining a spatial adjacency matrix; Perform regional aggregation operations on each feature map in the hierarchical feature map set to obtain a module feature matrix, and perform message transmission and update on the node features based on the spatial adjacency matrix and the module feature matrix to obtain a node state matrix; Perform edge attention operation on the node state matrix to obtain the inter-module connection relationship matrix, and collaboratively optimize the brightness and contrast of the LED display module based on the inter-module connection relationship matrix to obtain the module control parameters; The module control parameters are used to densely sample the display area to obtain a display unit parameter distribution map, and the display unit parameter distribution map is bounded to obtain a display parameter matrix.

[0028] Specifically, the target mapping matrix Perform region segmentation to divide the entire display area into multiple independent LED display modules. Assume that the size of the target mapping matrix is , that is, the height is , width is , by setting the row and column partition parameters of the module and , dividing the display area into rectangular modules. The scope of each module is defined as: ; in Indicates the row number, Represents the column number. After the segmentation is completed, each module is assigned a unique identifier to form a module partition matrix ,in Represents pixel The module number to which it belongs. Based on the module partition matrix , construct a display module adjacency graph, regard each module as a node in the graph, and use edges to describe the relationship between adjacent modules. and modules Represent two adjacent modules respectively, and their spatial distance is calculated by the Euclidean distance of their geometric centers: ; in and Modules and modules Based on these distances, a spatial adjacency matrix is ​​generated. ,in Representation Module and The adjacency relationship is limited by the threshold method, and only the distance less than a certain value is retained. . Perform regional aggregation operations on each feature map in the hierarchical feature map set to obtain a module feature matrix. Assume that the feature map set contains Layer feature map, the feature map size of each layer is , No. The layer feature map is ,in is the number of channels. Through regional aggregation operation, the pixel values ​​in the feature map are averaged and pooled according to the module division to obtain the module feature matrix , the specific formula is: ; in, Representation Module The set of pixels contained, is the number of elements in the set. The module feature matrix is ​​combined with the spatial adjacency matrix, and the node features are messaged and updated through the graph neural network to generate the node state matrix. The core of the node state update is based on the adjacency matrix The features between modules are weighted and aggregated, and the update formula is: ; in, It is After the iteration module Features, Representation Module The set of adjacent modules, is a learnable weight matrix, is the activation function (such as ReLU). After several iterations, the generated node state matrix It reflects the global characteristics of each module. It performs edge attention operation on the node state matrix, calculates the connection weights between modules, and generates the connection relationship matrix between modules. The edge attention mechanism adjusts the adjacency weights by learning the feature relationship between each pair of adjacent modules: ; in, represents feature concatenation, is the learned weight vector. The brightness and contrast of the module are optimized collaboratively to obtain the module control parameter matrix. The module control parameter matrix is ​​used to densely sample the display area, and the display unit parameter distribution map is generated by mapping the module parameters to the pixel level. The brightness value of each pixel is interpolated and calculated according to the module control parameters. At the same time, the module boundary characteristics are combined to perform boundary constraint processing, adjust the parameter distribution of the edge transition area, and generate the final display parameter matrix.

[0029] In a specific embodiment, the process of executing step 400 may specifically include the following steps: Input the display parameter matrix into the deep residual teacher network, perform multi-level feature transformation on the display parameters to obtain the teacher network feature map, and perform data enhancement on the display parameter matrix to generate a multi-view training dataset; The multi-view training dataset is input into the lightweight student network for feature extraction and transformation to obtain the student network feature map, and the feature alignment operation is performed on the teacher network feature map and the student network feature map to obtain the knowledge distillation loss; Based on the multi-view training data set, the display effect differences under different viewpoints are calculated, and the multi-view display consistency loss value is constructed. The knowledge distillation loss and the multi-view display consistency loss value are weightedly combined to obtain the overall loss function. The lightweight student network is back-propagated and trained according to the overall loss function to update the network parameters to obtain the optimized student network. The optimized student network is then pruned and quantized to obtain a subtitle display model.

[0030] Specifically, the parameter matrix will be displayed Enter the Deep Residual Teacher Network. and Respectively represent the height and width of the display screen, is the number of channels of the parameter (such as brightness, contrast, etc.). The deep residual teacher network performs multi-level feature transformation on the display parameters through multiple residual modules. Each residual module consists of convolution, batch normalization and activation function, and its core formula is: ; in, It is The input feature map of the layer, and are the convolution weights and biases, respectively. is the activation function (such as ReLU), Represents the convolution operation. By stacking residual modules layer by layer, the teacher network can extract the global and local features of the display parameters and generate a high-dimensional teacher network feature map ,in is the final number of feature channels. In order to enhance the generalization ability of the training data, data enhancement is performed on the input display parameter matrix to generate a multi-view training dataset Data enhancement includes operations such as rotation, flipping, brightness perturbation, and contrast adjustment. For example, if the initial brightness value of the display parameter matrix is , through random gain Perform brightness perturbation: ; in, Used to limit the range of brightness values ​​to [0, 1]. These enhancement operations generate a diverse dataset for training by simulating display characteristics under different viewing angles and environmental conditions. The multi-view training dataset is input into the lightweight student network for feature extraction and transformation. The lightweight student network uses efficient structures such as deep separable convolution to reduce computational complexity while retaining key features. Assume that the first The layer feature map is Its output is expressed as: ; in, and are the weight matrices of the depthwise convolution and pointwise convolution respectively. The feature map finally generated by the student network is ,in , indicating that the dimension of the feature is compressed. In order to achieve knowledge distillation, the teacher network feature mapping and student network feature mapping Perform feature alignment. Feature alignment minimizes the difference between the two and defines the knowledge distillation loss as: ; in, and Respectively represent the teacher and student feature maps at positions At the same time, based on the multi-view training data set, the display effect difference under different view angles is calculated, and the multi-view display consistency loss value is defined. . Let the viewpoint set be , at every perspective Below, the loss of the display effect is defined as the KL divergence between the target parameter distribution and the actual distribution: ; in, and Denote the target display distribution and the actual display distribution respectively. By weighted combination of knowledge distillation loss and multi-angle display consistency loss, the overall loss function is defined as: ; in, and is a weight hyperparameter used to balance the contribution of the two losses. After defining the overall loss function, the lightweight student network is trained using the back propagation algorithm to gradually update the network parameters. The formula for gradient update is: ; in, is the learning rate, It is The layer weights are The value at the iteration. After training is completed, the optimized student network is pruned and quantized to reduce computational complexity. Pruning reduces redundant parameters by removing unimportant neuron connections. Quantization compresses weights and activation values ​​from floating-point representation to low-precision representation (such as 8-bit integers), reducing storage requirements and computational overhead. Finally, the subtitle display model is obtained.

[0031] In a specific embodiment, the process of executing step 500 may specifically include the following steps: The subtitle display model is set to inference mode, the current subtitle content and display area information are input, the initial prediction result is obtained, and the initial prediction result is decoded and analyzed, the output data of the position prediction branch and the brightness prediction branch are separated, and the dual-path prediction feature is obtained; Performing coordinate transformation on the position prediction branch in the dual-path prediction feature, mapping the predicted position to the physical coordinate system of the display screen, and obtaining the subtitle layout coordinates; at the same time, calculating the brightness distribution of the display unit based on the brightness prediction branch in the dual-path prediction feature, performing brightness normalization operation, and obtaining a brightness control sequence; The position-brightness coupling coefficient is calculated according to the subtitle layout coordinates and the brightness control sequence to obtain a fusion weight, and the subtitle layout coordinates and the brightness control sequence are weightedly combined using the fusion weight to obtain a target display parameter set; According to the target display parameter set, the ambient light compensation operation is performed, the gradient domain fusion and temporal smoothing processing are performed on the subtitle edge, and the target subtitle display data is output.

[0032] Specifically, the trained subtitle display model is set to inference mode. The model parameters are frozen to ensure stable performance during inference and avoid unnecessary gradient calculations. and display area information Input the model to generate initial prediction results. Assume that the input subtitle content It is the vectorized text contour information, represented as a point set , and display area information is a set of parameters that describe the geometry and resolution of the curved display. The model generates initial predictions based on the inputs ,in Contains location prediction branch and brightness prediction branch Output data of the initial prediction results When performing decoding analysis, predict branches from positions Extract the two-dimensional position information of the subtitles, expressed as relative coordinates ,in Describe the position of the subtitle in the normalized display area. At the same time, from the brightness prediction branch Extract the brightness distribution of the display unit, expressed as a set of brightness values ,in Indicates The normalized brightness value of the subtitle pixel. The coordinate transformation is performed on the output of the position prediction branch, and the normalized position Mapped to the physical coordinate system of the display. Assume that the physical resolution of the display is , the conversion formula is: ; in, is the position coordinate in the physical coordinate system of the display screen, and are screen width and height respectively. The converted coordinate set constitutes the subtitle layout coordinate, which is used to accurately describe the distribution of subtitle content on the physical screen. For the output of the brightness prediction branch, based on the brightness value set Calculate the brightness distribution of the display unit and perform brightness normalization to adapt to the brightness range of the display. Assume that the maximum and minimum brightness values ​​of the display are and , the brightness normalization formula is: ; in, is the standardized brightness value. The standardization operation ensures that the brightness value output by the model can be accurately mapped to the hardware range of the display and generate a brightness control sequence. After obtaining the subtitle layout coordinates and brightness control sequence, the coupling relationship between the two is calculated. The position-brightness coupling coefficient is defined by quantifying the influence of the brightness of each display unit on its position adjustment. Suppose the coupling coefficient of each display unit is , and its calculation formula is: ; in, is the total number of display units, Indicates The relative brightness weight of each unit. Using the coupling coefficient Subtitle layout coordinates and brightness control sequence Perform weighted combination to generate target display parameter set The position coordinates and brightness values ​​in the target display parameters are calculated using the following formula: ; in, is the physical coordinate after coupling adjustment, is the final brightness value. According to the target display parameter set , perform ambient light compensation calculations to adapt to different lighting conditions. During the compensation process, the ambient light intensity around the display is collected to generate an ambient light distribution map The compensation formula is: ; in, is the compensation coefficient, is the brightness value after ambient light compensation. Gradient domain fusion is performed on the edge area of ​​the subtitles, and the transition area is smoothed by calculating the edge gradient field to ensure the visual continuity between the subtitles and the background area. The fusion formula is: ; in, is the brightness value after gradient correction, and denote the Laplace operator and the gradient operator respectively, is the brightness gradient. The display parameters are smoothed in time dimension to eliminate the jumps and jitters in the dynamic change of subtitles. Assume the time step is , then the time series smoothing formula is: ; in, is the smoothing coefficient, Indicates The smooth brightness value of the frame. The parameters after ambient light compensation, gradient domain fusion and timing smoothing are input into the display buffer to ensure synchronization with the refresh of the display screen. Through the above steps, the generated target subtitle display data can present the subtitle content with the best effect under different lighting and viewing angles.

[0033] In a specific embodiment, the execution step performs ambient light compensation calculation according to the target display parameter set, performs gradient domain fusion and temporal smoothing processing on the subtitle edge, and outputs the target subtitle display data, which may specifically include the following steps: According to the target display parameter set, local gradient calculation is performed on the edge area of ​​the subtitle to obtain the gradient value matrix and edge contour data, and the Poisson boundary equation is constructed according to the gradient value matrix and the edge contour data to obtain the edge fusion function; The edge fusion function is used to perform pixel-level gradient processing on the edge area and the background area of ​​the subtitles to obtain edge transition area data, and the regional fusion coefficient is calculated based on the edge transition area data and the display area distribution characteristics to obtain the regional weight matrix; Perform multi-point acquisition and spatial distribution analysis on the ambient light of the display area to obtain an ambient light intensity map, and perform block brightness compensation on the display area according to the ambient light intensity map to obtain a light response matrix; The regional weight matrix and the illumination response matrix are jointly optimized to construct a display compensation model, obtain regional compensation parameters, and adaptively adjust the brightness of the subtitle display area according to the regional compensation parameters to obtain compensated display data; Reconstructing the compensated display data into a frame sequence according to the display timing to obtain a dynamic display sequence, and performing a time domain filtering operation on the dynamic display sequence to eliminate inter-frame jitter to obtain smooth sequence data; The smooth sequence data is input into the display buffer for frame synchronization control, and the data is reorganized according to the refresh timing and synchronization signal of the display screen to obtain the target subtitle display data.

[0034] Specifically, based on the target display parameter set , local gradient calculation is performed on the edge area of ​​the subtitle to capture the brightness change information of the edge. Assume that the brightness distribution of the subtitle area is , then the horizontal and vertical components of the local gradient are expressed as: ; Through the above calculation, a gradient value matrix is ​​generated for the subtitle edge area. . Extract edge contour data based on the gradient value matrix combined with the high contrast area in the pixel brightness distribution , which describes the geometric shape and characteristics of the edge area. After obtaining the gradient value matrix and edge contour data, the Poisson equation is used to build an edge fusion model to achieve continuous brightness transition between the edge area and the background area. The basic form of the Poisson equation is: ; in, is the Laplace operator, Represents the divergence of the gradient field. By numerically solving the above equations and combining the edge contour data As the boundary condition, we get the edge fusion function The output brightness value achieves pixel-level gradient in the edge area. The edge fusion function is used to fuse the subtitle edge area and the background area to generate edge transition area data. On this basis, combined with the distribution characteristics of the display area (such as modular structure or geometric characteristics), the regional fusion coefficient is calculated , and generate the regional weight matrix The fusion coefficient is defined as the normalized weight of the brightness difference between the edge area and the background area, for example: ; Regional Weight Matrix Indicates the relative weight of each pixel in the fusion process. In order to adapt to different lighting environments, the display area is subjected to ambient light compensation calculations. The distribution data of the ambient light around the display screen is obtained through a multi-point light intensity acquisition device to generate an ambient light intensity map. The intensity map reflects the influence of ambient light on each pixel. Perform block brightness compensation on the display area to generate a light response matrix , and its calculation formula is: ; in, is the illumination compensation coefficient, is the uncompensated target brightness. The regional weight matrix and the light response matrix Perform joint optimization operations to build a display compensation model to generate a regional compensation parameter matrix The formula for joint optimization is: ; Based on the regional compensation parameters, the subtitle display area is adaptively adjusted to obtain the compensated display data : ; The compensated display data is reconstructed into frame sequence according to the display timing to ensure smooth display of subtitles when they change dynamically. Assume that the time step is , the frame sequence reconstruction formula is: ; in, is the smoothing coefficient, which is used to adjust the weight distribution between frames. To further eliminate inter-frame jitter, a time domain filtering operation is performed on the frame sequence data, and a low-pass filter is used to suppress high-frequency noise to obtain a smooth sequence data. The filtering formula is: ; in, is the filter window size, which determines the intensity of time smoothing. Input display buffer, through the refresh timing of the display and sync signal Align, complete data reorganization, and generate target subtitle display data.

[0035] In this embodiment, the interactive subtitles of the user and the virtual exhibition hall are also dynamically followed: the user's motion video data is collected by a depth camera, the human joints in each frame are marked and extracted, a two-dimensional human skeleton node graph is constructed, and the two-dimensional human skeleton node graph is converted into a three-dimensional space coordinate system to obtain a three-dimensional feature sequence of the user's motion; the three-dimensional feature sequence of the user's motion is input into an improved Joint-UNet network, and the joint features at different scales are extracted by multi-scale node pooling to obtain hierarchical motion features, and the detail information is retained by skip connections to obtain a multi-level posture feature pyramid; a self-adjusting graph convolutional network is constructed based on the multi-level posture feature pyramid, and the spatial relationship and motion correlation between the joints are automatically adjusted through a learnable adjacency matrix, and the node features are dynamically updated and propagated to obtain an adaptive connection weight matrix; a user motion trajectory prediction model is constructed according to the adaptive connection weight matrix, and the user's motion direction and speed are modeled, and the result is a self-adjusting graph convolutional network based on the multi-level posture feature pyramid. The motion prediction is performed based on the historical trajectory data to obtain a predicted position sequence; the predicted position sequence is projected into the three-dimensional scene of the virtual exhibition hall, the relative position relationship between the user and the display content is calculated, and an interactive mapping function based on the attention mechanism is constructed to obtain the subtitle following strategy parameters; the target position and orientation of the subtitles in the three-dimensional space are calculated according to the subtitle following strategy parameters, and the following trajectory is smoothly constrained by using the L1 and L2 fusion losses to eliminate the following jitter and obtain a smooth motion trajectory; the smooth motion trajectory is input into the motion compensation network, and the position, size and direction of the subtitles are adjusted in real time to ensure that the subtitles are always in the best viewing angle for the user, and dynamically updated display parameters are obtained; display optimization processing is performed on the dynamically updated display parameters to adjust the spatial layout and display effect of the subtitles, and the step of "performing ambient light compensation operation according to the target display parameter set, performing gradient domain fusion and temporal smoothing processing on the subtitle edges, and outputting the target subtitle display data" is called to realize continuous and smooth following display of the subtitles.

[0036] The above describes the method for generating a virtual exhibition hall based on a curved screen in the embodiment of the present application. The following describes the device 10 for generating a virtual exhibition hall based on a curved screen in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of a virtual exhibition hall generation device 10 based on a curved screen includes: The projection module 11 is used to project the subtitle content pre-input in the virtual exhibition hall onto the surface of the display screen through the arc-shaped LED display screen, and obtain the subtitle display area mapping matrix and the initial subtitle image; A feature extraction module 12 is used to perform feature extraction and adaptive fusion operations on the display area based on the subtitle display area mapping matrix and the initial subtitle image to obtain a target mapping matrix and a hierarchical feature atlas; An interactive analysis module 13 is used to perform LED module interactive analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; A training module 14 is used to input the display parameter matrix into the deep residual teacher network and the lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; The output module 15 is used to jointly predict the subtitle position and display brightness using the subtitle display model to obtain a target display parameter set, and perform gradient domain fusion and temporal smoothing on the subtitle edge according to the target display parameter set to output the target subtitle display data.

[0037] Through the cooperation of the above components, by establishing an accurate three-dimensional spatial coordinate system and a multi-angle scanning mechanism, the surface characteristics of the curved LED display screen are accurately modeled. The multi-scale feature hierarchical network is used to extract and fuse the display area features. The self-attention mechanism and feature pyramid integration are combined to effectively improve the refinement of the subtitle display. The graph convolutional network is introduced to analyze the interaction of LED modules. By constructing the connection relationship matrix between modules, the problem of discontinuous display between adjacent LED modules is solved. A collaborative training framework of deep residual teacher network and lightweight student network is designed. Through knowledge distillation and multi-angle consistency loss constraints, the consistency of display effects under different viewing angles is guaranteed. The joint optimization of subtitle position and display brightness is achieved. Through the synergy of dual-path prediction branches, the rationality of subtitle layout and the uniformity of display brightness are ensured. A gradient domain fusion scheme based on Poisson's equation is proposed, which is combined with ambient light compensation and timing smoothing to effectively eliminate the problems of subtitle edge display mutation and timing jitter.

[0038] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0039] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0040] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a virtual exhibition hall based on a curved screen, characterized in that: The method comprises: The subtitle content pre-entered in the virtual exhibition hall is projected onto the display surface through the curved LED display screen to obtain the subtitle display area mapping matrix and the initial subtitle image; Based on the subtitle display area mapping matrix and the initial subtitle image, feature extraction and adaptive fusion operation are performed on the display area to obtain a target mapping matrix and a hierarchical feature atlas; Performing LED module interaction analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; Inputting the display parameter matrix into the deep residual teacher network and the lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; The subtitle display model is used to jointly predict the subtitle position and display brightness to obtain a target display parameter set, and according to the target display parameter set, the subtitle edge is subjected to gradient domain fusion and temporal smoothing to output target subtitle display data.

2. The method for generating a virtual exhibition hall based on a curved screen according to claim 1, characterized in that: The method of projecting the subtitle content pre-input in the virtual exhibition hall onto the surface of the display screen through the curved LED display screen to obtain the subtitle display area mapping matrix and the initial subtitle image includes: The surface of the curved LED display screen is divided into a plurality of grid points, and a unique identifier is assigned to each grid point to obtain a grid point identification matrix; Using a laser rangefinder to perform multi-angle scanning and distance data analysis on the grid points within a preset angle range to obtain a grid point distance matrix; Based on the grid point identification matrix and the grid point distance matrix, the three-dimensional space coordinate value of each grid point is calculated, a surface point cloud data set of the arc-shaped LED display screen is established, and a surface fitting operation is performed on the surface point cloud data set to obtain a display screen surface equation; The subtitle content pre-input in the virtual exhibition hall is converted into a font outline vector, the projection position of the font outline on the surface is calculated according to the display screen surface equation, the subtitle outline projection data is obtained, and the subtitle outline projection data is rasterized, and the continuous projection outline is mapped to the discrete LED module position to obtain the subtitle display area mapping matrix; Font anti-aliasing and rasterization processing are performed based on the subtitle outline projection data to obtain a high-resolution font bitmap, and resolution adaptive transformation is performed on the high-resolution font bitmap to scale the font bitmap to the display screen resolution to obtain an initial subtitle image.

3. The method for generating a virtual exhibition hall based on a curved screen according to claim 2, characterized in that: The method of performing feature extraction and adaptive fusion operation on the display area based on the subtitle display area mapping matrix and the initial subtitle image to obtain a target mapping matrix and a hierarchical feature atlas includes: Inputting the subtitle display area mapping matrix and the initial subtitle image into a multi-scale feature hierarchical network for multi-scale decomposition to obtain n feature layers of different scales, and performing convolution operations and local feature extraction on the n feature layers of different scales respectively to obtain n groups of original feature maps; Constructing a feature association graph according to the n groups of original feature graphs, calculating the similarity weights between feature nodes to obtain a feature association matrix, and using the feature association matrix to perform weighted integration on the n groups of original feature graphs to obtain a fused feature graph; Performing a self-attention operation on the fused feature map to obtain an attention weight matrix, and spatially reweighting the fused feature map based on the attention weight matrix to obtain a target mapping matrix; The fused feature map and the attention weight matrix are input into a deconvolution network for feature upsampling and reconstruction to obtain feature representations of different resolutions, and feature pyramid integration is performed on the feature representations of different resolutions to obtain a hierarchical feature atlas.

4. The method for generating a virtual exhibition hall based on a curved screen according to claim 3, characterized in that: The performing LED module interaction analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix includes: Performing region segmentation on the target mapping matrix, dividing the display area into a plurality of LED display modules to obtain a module partition matrix, and constructing a display module adjacency graph based on the module partition matrix, taking each LED display module as a graph node, calculating the spatial distance between nodes, and obtaining a spatial adjacency matrix; Performing a regional aggregation operation on each feature graph in the hierarchical feature graph set to obtain a module feature matrix, and performing message transmission and updating on node features based on the spatial adjacency matrix and the module feature matrix to obtain a node state matrix; Performing edge attention operation on the node state matrix to obtain an inter-module connection relationship matrix, and collaboratively optimizing the brightness and contrast of the LED display module according to the inter-module connection relationship matrix to obtain module control parameters; The module control parameters are used to perform dense sampling on the display area to obtain a display unit parameter distribution map, and boundary constraints are performed on the display unit parameter distribution map to obtain a display parameter matrix.

5. The method for generating a virtual exhibition hall based on a curved screen according to claim 4, characterized in that: The display parameter matrix is ​​input into the deep residual teacher network and the lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model, including: Inputting the display parameter matrix into a deep residual teacher network, performing multi-level feature transformation on the display parameters to obtain a teacher network feature map, and performing data enhancement on the display parameter matrix to generate a multi-view training data set; Inputting the multi-view training data set into a lightweight student network for feature extraction and transformation to obtain a student network feature map, and performing a feature alignment operation on the teacher network feature map and the student network feature map to obtain a knowledge distillation loss; Calculating the display effect difference under different viewing angles based on the multi-view training data set, constructing a multi-view display consistency loss value, and weightedly combining the knowledge distillation loss with the multi-view display consistency loss value to obtain an overall loss function; The lightweight student network is back-propagated and trained according to the overall loss function to update network parameters to obtain an optimized student network. The optimized student network is model pruned and quantized to obtain a subtitle display model.

6. The method for generating a virtual exhibition hall based on a curved screen according to claim 5, characterized in that: The method of using the subtitle display model to jointly predict the subtitle position and display brightness to obtain a target display parameter set, and performing gradient domain fusion and temporal smoothing on the subtitle edge according to the target display parameter set to output target subtitle display data includes: The subtitle display model is set to an inference mode, current subtitle content and display area information are input, an initial prediction result is obtained, and the initial prediction result is decoded and analyzed, and output data of a position prediction branch and a brightness prediction branch are separated to obtain a dual-path prediction feature; Performing coordinate transformation on the position prediction branch in the dual-path prediction feature, mapping the predicted position to the physical coordinate system of the display screen, and obtaining the subtitle layout coordinates; at the same time, calculating the brightness distribution of the display unit based on the brightness prediction branch in the dual-path prediction feature, performing brightness normalization operation, and obtaining a brightness control sequence; Calculating a position-brightness coupling coefficient according to the subtitle layout coordinates and the brightness control sequence to obtain a fusion weight, and performing a weighted combination of the subtitle layout coordinates and the brightness control sequence using the fusion weight to obtain a target display parameter set; An ambient light compensation operation is performed according to the target display parameter set, gradient domain fusion and temporal smoothing are performed on the edge of the subtitles, and target subtitle display data is output.

7. The method for generating a virtual exhibition hall based on a curved screen according to claim 6, characterized in that: The step of performing ambient light compensation operation according to the target display parameter set, performing gradient domain fusion and temporal smoothing processing on the edge of the subtitles, and outputting target subtitle display data includes: Performing local gradient calculation on the edge area of ​​the subtitles according to the target display parameter set to obtain a gradient value matrix and edge contour data, and constructing a Poisson boundary equation according to the gradient value matrix and the edge contour data to obtain an edge blending function; Using the edge fusion function to perform pixel-level gradient processing on the subtitle edge area and the background area to obtain edge transition area data, and calculating the regional fusion coefficient according to the edge transition area data and the display area distribution characteristics to obtain the regional weight matrix; Perform multi-point acquisition and spatial distribution analysis on the ambient light of the display area to obtain an ambient light intensity map, and perform block brightness compensation on the display area according to the ambient light intensity map to obtain a light response matrix; Performing a joint optimization operation on the regional weight matrix and the illumination response matrix to construct a display compensation model, obtain regional compensation parameters, and adaptively adjusting the brightness of the subtitle display area according to the regional compensation parameters to obtain compensated display data; Reconstructing the compensated display data into a frame sequence according to the display timing to obtain a dynamic display sequence, and performing a time domain filtering operation on the dynamic display sequence to eliminate inter-frame jitter to obtain smooth sequence data; The smooth sequence data is input into a display buffer for frame synchronization control, and data is reorganized according to a refresh timing and a synchronization signal of a display screen to obtain target subtitle display data.

8. A virtual exhibition hall generation device based on a curved screen, characterized in that: The method for generating a virtual exhibition hall based on a curved screen according to claim 1, wherein the device for generating a virtual exhibition hall based on a curved screen comprises: The projection module is used to project the subtitle content pre-input in the virtual exhibition hall onto the surface of the display screen through the curved LED display screen, and obtain the subtitle display area mapping matrix and the initial subtitle image; A feature extraction module, used to perform feature extraction and adaptive fusion operations on the display area based on the subtitle display area mapping matrix and the initial subtitle image, to obtain a target mapping matrix and a hierarchical feature atlas; An interactive analysis module, used for performing LED module interactive analysis on the target mapping matrix and the hierarchical feature atlas to obtain a display parameter matrix; A training module, used for inputting the display parameter matrix into a deep residual teacher network and a lightweight student network to perform multi-angle display consistency loss training to obtain a subtitle display model; The output module is used to jointly predict the subtitle position and display brightness using the subtitle display model to obtain a target display parameter set, and to perform gradient domain fusion and temporal smoothing on the subtitle edge according to the target display parameter set to output target subtitle display data.

Images