Low-delay image communication collecting, editing and displaying system and method for non-primitive tourism news

Abstract

The invention relates to the technical field of video image processing and communication, and discloses a low-delay image communication collecting and editing display system and method for non-abandoned tourism news, and the system comprises a background base map storage module which is used for pre-sending a static base map and an interaction description file; the area illumination detection module is used for calculating a sparse illumination parameter matrix of the real-time picture; the illumination smoothing processing module is used for generating a continuous pixel-level illumination adjustment graph through interpolation; the dynamic foreground extraction and difference transmission module is configured to use the adjustment graph to correct the base graph so as to construct a virtual reference base graph, extract dynamic foreground image fragments through difference, and package and send the illumination parameters and the foreground fragments; and the terminal synthesis display module is used for rendering the local base map by using the illumination parameters and superposing the foreground fragments to restore the picture, and meanwhile, responding to the touch interaction. Only the illumination parameters and the foreground fragments are transmitted, so that the transmission data volume and delay are remarkably reduced, and the problems of background synthesis and semantic interaction in a complex illumination environment are solved.

Description

Technical Field

[0001] This invention relates to the fields of video image processing and communication technology, specifically to a low-latency image communication acquisition, editing, and display system and method for intangible cultural heritage tourism news. Background Technology

[0002] With the development of the digital cultural tourism industry, the collection, editing, and dissemination of intangible cultural heritage tourism news increasingly rely on real-time video communication technology. In the existing news gathering and editing process, reporters on the ground typically use mobile camera equipment to film intangible cultural heritage performances or folk activities, and then transmit the video stream to user terminals via 4G or 5G mobile networks. To adapt to network bandwidth limitations, traditional solutions mainly adopt common video coding standards such as H.264 and H.265, using intra-frame prediction and inter-frame motion compensation techniques to compress image sequences, thereby reducing the data bitrate during transmission and enabling remote display of intangible cultural heritage images.

[0003] Intangible cultural heritage performances are typically characterized by dramatic and complex changes in lighting, such as switching stage lights and fluctuations in outdoor light intensity. While traditional video coding techniques can compress data, the amount of residual data from inter-frame prediction remains large when faced with frequent changes in overall lighting. This leads to high latency or stuttering in video transmission at tourist sites where network signals fluctuate or bandwidth is limited. Although existing technologies include foreground extraction transmission schemes based on background differentiation, attempting to transmit only dynamic targets to reduce bandwidth, these schemes struggle to effectively distinguish between sudden changes in lighting and object movement. When the ambient lighting changes, the pixel values ​​of the static background fluctuate accordingly, causing the difference algorithm to mistakenly identify the static background as a dynamic foreground. This results in background stripping failure, making it impossible to maintain image reconstruction quality while ensuring low-bandwidth transmission.

[0004] Existing video streaming modes are limited to the one-way presentation of visual information, and the transmitted video data lacks semantic connection with specific cultural entities within the frame. Viewers watching live broadcasts of intangible cultural heritage news can only passively receive image signals and cannot interact with the distinctive costumes, musical instruments, or intangible cultural heritage props appearing on screen. This separation of video presentation and information retrieval makes it difficult for users to obtain real-time cultural interpretations of objects of interest during viewing, limiting the information depth of intangible cultural heritage tourism news and the user's participatory experience. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a low-latency image communication acquisition, editing, and display system and method for intangible cultural heritage tourism news. It solves the problems of existing video transmission technologies being unable to effectively separate background lighting changes from dynamic foreground targets in complex lighting environments at intangible cultural heritage performance sites, resulting in large redundancy in video encoding data and the lack of semantic interaction information based on target entities in traditional video streams.

[0006] To achieve the above objectives, the present invention provides a low-latency image communication acquisition and display system for intangible cultural heritage tourism news, including a background map storage module, a regional illumination detection module, an illumination smoothing processing module, a dynamic foreground extraction and differential transmission module, and a terminal synthesis display module.

[0007] The background image storage module is configured to control the camera acquisition device to capture static environmental images of the intangible cultural heritage performance site. These images are then analyzed for stability and determined as the initial baseline image. The module utilizes a pre-trained object detection neural network model to extract features and predict classifications from the initial baseline image, identifying target objects with specific cultural attributes and obtaining their bounding box coordinates. Based on the identified target object identifiers, the module retrieves associated cultural tourism metadata, binds the bounding box coordinates to the cultural tourism metadata to generate an interactive specification file, and sends the initial baseline image and the interactive specification file to the receiving end for storage.

[0008] The region illumination detection module is connected to the background base map storage module and configured to logically divide the real-time captured video footage into regions. For all logical detection regions, the region illumination detection module extracts the pixel brightness set of the current real-time video frame and the pixel brightness set of the initial reference base map, and constructs a linear illumination transformation model. The region illumination detection module uses the least squares method to solve for the brightness gain coefficient and brightness bias value of the logical detection region, minimizing the linear transformation error between the pixel brightness set of the real-time video frame and the pixel brightness set of the initial reference base map within the logical detection region, and generating a sparse illumination parameter matrix composed of the brightness gain coefficient and brightness bias value of all regions.

[0009] The illumination smoothing module is connected to the region illumination detection module and configured to perform spatial interpolation on the sparse illumination parameter matrix to generate a continuous pixel-level illumination adjustment map covering the entire image. The continuous pixel-level illumination adjustment map includes a continuous gain map and a continuous bias map. For any target pixel in the entire image, the illumination smoothing module determines the center points of the adjacent logical detection regions surrounding the target pixel and uses a bilinear interpolation algorithm to calculate the smoothed gain and smoothed bias values ​​of the target pixel based on the normalized distance weights of the target pixel relative to the center points of the adjacent logical detection regions.

[0010] The dynamic foreground extraction and differential transmission module is connected to the background base map storage module and the illumination smoothing processing module, respectively. It is configured to use a continuous pixel-level illumination adjustment map to correct the initial reference base map to generate a virtual reference base map. When generating the virtual reference base map, the dynamic foreground extraction and differential transmission module obtains the original brightness value of each pixel in the initial reference base map, multiplies the original brightness value by the smoothed gain value of the corresponding coordinate in the continuous gain map, and adds the smoothed offset value of the corresponding coordinate in the continuous offset map. The calculated result is used as the virtual brightness value of that pixel in the virtual reference base map. The dynamic foreground extraction and differential transmission module calculates the absolute value of the brightness difference between the current real-time video frame and the virtual reference base map at the same coordinates, and compares the absolute value with a preset tolerance threshold. If the absolute value is greater than the tolerance threshold, the dynamic foreground extraction and differential transmission module determines that the pixel is a dynamic foreground; otherwise, it is determined to be a static background. Based on the determination result, the dynamic foreground extraction and differential transmission module generates a binarized foreground mask and crops the image region containing the dynamic foreground pixels from the current real-time video frame as a dynamic foreground image fragment. The dynamic foreground extraction and differential transmission module constructs a transmission data packet from the sparse illumination parameter matrix, dynamic foreground image fragments and their location information and sends it to the receiving end. The transmission data packet does not contain image data of the static background.

[0011] The terminal-based composite display module, located at the receiving end, is configured to parse transmitted data packets and perform color rendering on the locally stored initial baseline map based on illumination parameters, overlaying dynamic foreground image fragments to restore the displayed image. During the restoration process, the terminal-based composite display module reconstructs a continuous pixel-level illumination adjustment map locally at the receiving end. For all pixel coordinates of the target display image, the terminal-based composite display module determines whether the pixel coordinates are within the location range of the dynamic foreground image fragments. If so, the terminal-based composite display module directly extracts the pixel value of the foreground fragment at that pixel coordinate as the display value; if not, the terminal-based composite display module uses the reconstructed continuous pixel-level illumination adjustment map to perform linear modulation calculations on the locally stored initial baseline map, and uses the calculation result as the display value. Simultaneously, the terminal-based composite display module monitors screen touch events and obtains the touch point coordinates, traverses the target object bounding box parameters recorded in the interaction specification file, and determines whether the touch point coordinates fall within the range of any target object. If a match is found, the terminal-based composite display module calls and displays the cultural and tourism metadata associated with the target object.

[0012] A second aspect of this invention provides a low-latency image communication acquisition, editing, and display method for intangible cultural heritage tourism news, comprising the following steps: The system controls the camera acquisition equipment to obtain static environmental images of the performance site as the initial reference base map. The target detection algorithm is used to identify target objects in the base map and generate an interactive description file containing position coordinates and related information. The initial reference base map and the interactive description file are then sent to the receiving end.

[0013] The system acquires live video streams of the performance and divides them into logical regions. For all logical detection regions, it calculates the linear brightness transformation relationship between the current video frame and the initial reference base map. By minimizing the linear transformation error, it solves for the brightness gain coefficient and brightness bias value, generating a sparse illumination parameter matrix.

[0014] Spatial interpolation is performed on the sparse illumination parameter matrix to map the discrete regional illumination parameters into a continuous pixel-level illumination adjustment map covering the entire screen. The continuous pixel-level illumination adjustment map includes a continuous gain map that records the illumination scaling ratio of each point in the entire screen and a continuous offset map that records the illumination offset of each point in the entire screen.

[0015] A virtual reference base map is generated by linearly modulating an initial reference base map using a continuous pixel-level illumination adjustment map, which is consistent with the current real-time illumination distribution. The pixel difference between the current real-time video frame and the virtual reference base map is calculated, and dynamic foreground image fragments are extracted based on the difference threshold. The sparse illumination parameter matrix, dynamic foreground image fragments, and their location information are constructed into a transmission data packet and sent to the receiving end.

[0016] The receiving end parses the transmitted data packets, reconstructs a continuous pixel-level illumination adjustment map locally, and renders the locally stored initial baseline map as a virtual background. Decoded dynamic foreground image fragments are then overlaid onto the corresponding coordinate positions of the virtual background to synthesize the final display image. Simultaneously, screen touch events are monitored, and user actions are responded to according to the coordinate definitions in the interaction specification file, displaying corresponding intangible cultural heritage tourism information.

[0017] This invention provides a low-latency image communication acquisition, editing, and display system and method for intangible cultural heritage tourism news. It has the following beneficial effects: 1. This invention achieves extremely low-bandwidth transmission of video data by constructing a transmission mechanism that includes a sparse illumination parameter matrix and dynamic foreground image fragments. The system only needs to transmit a static baseline map once in the initial stage, and in subsequent processes, it only transmits illumination change parameters, which require a very small amount of data, and the cropped dynamic foreground region. This differential transmission strategy effectively removes a large amount of spatiotemporal redundant data in the video sequence, significantly reduces network load, and ensures low-latency, high-smoothness real-time video transmission even in network-constrained environments at intangible cultural heritage tourism sites.

[0018] 2. This invention utilizes regional illumination detection and spatial smoothing interpolation algorithms to solve the interference caused by complex lighting changes at the performance venue on background subtraction. By calculating brightness gain and bias and performing pixel-level reconstruction, the receiving end can synthesize a virtual background in real time using a local base map that adapts to the current lighting changes at the venue. This not only ensures the visual continuity of the synthesized image and avoids false background detection caused by sudden changes in illumination, but also ensures the accuracy of dynamic foreground extraction, so that the final displayed image maintains the clarity and integrity of image details while restoring the atmosphere of the performance.

[0019] 3. This invention combines object detection algorithms with interactive documentation to endow instant messaging video streams with semantic interactive capabilities. By pre-identifying cultural entities in a static background and binding metadata at the acquisition end, the receiving end can respond to user touch operations based on preset coordinate information. This changes the traditional passive viewing mode of news live broadcasts, allowing users to click on specific clothing, musical instruments, or props in the screen in real time to obtain detailed introductions to intangible cultural heritage, thereby increasing the information carrying capacity and dissemination depth of tourism news. Attached Figure Description

[0020] Figure 1 This is a system framework diagram of the present invention; Figure 2 This is a schematic diagram of the background image storage module processing flow of the present invention; Figure 3 This is a schematic diagram of the regional illumination detection and illumination smoothing process of the present invention; Figure 4 This is a schematic diagram of the dynamic foreground extraction and differential transmission processing flow of the present invention; Figure 5 This is a schematic diagram of the terminal synthesis display and interactive processing flow of the present invention; Figure 6 This is a flowchart of the method of the present invention.

[0021] Among them, 10 is the background image storage module; 20 is the area illumination detection module; 30 is the illumination smoothing module; 40 is the dynamic foreground extraction and differential transmission module; and 50 is the terminal synthesis and display module. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See attached document Figure 1 , Figure 1This is a structural block diagram of a low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to an embodiment of the present invention. The present invention provides a low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news, comprising: Background image storage module 10 is configured to pre-capture a static environment without people at the intangible cultural heritage performance site as a high-definition background image and send it to the receiving end. The background image is divided into several detection areas. At the same time, it identifies entities with tourism value such as ancient buildings, plaques or intangible cultural heritage tools in the background image, records their coordinates, and generates a description file containing cultural tour information and sends it together. The area illumination detection module 20 is configured to analyze the video footage captured by the camera in real time, calculate the brightness gain and offset values ​​of each detection area relative to the initial base map, and transform the complex stage lighting and shadow changes on site into a set of sparse parameter matrices with a very small amount of data, so as to capture the atmosphere of the scene with a low amount of data. The lighting smoothing module 30 is configured to use a bilinear interpolation algorithm to perform spatial transition processing on sparse brightness parameters, generate a continuous pixel-level lighting adjustment map covering the entire screen, simulate the natural gradient effect of light in the real physical world, and eliminate the edge discontinuity phenomenon caused by partition detection. The dynamic foreground extraction and differential transmission module 40 is configured to use the illumination adjustment map to correct the original base map to generate a virtual base map consistent with the on-site lighting and shadow, and to perform pixel-level differential comparison between the real-time video and the virtual base map. It uses the principle of background pixel value cancellation to eliminate interference from stage lighting changes, accurately locks and separates the dynamic images of intangible cultural heritage performers and props, and prevents the system from mistakenly judging a background wall that has only become brighter as new content. This enables the transmission of only dynamic foreground fragments, position coordinates and illumination parameters, eliminating the need for repeated transmission of static backgrounds. The terminal synthesis display module 50 is configured to recolor the local base map and overlay a dynamic foreground to restore the image based on the lighting parameters at the receiving end. At the same time, it uses a pre-transmitted instruction file to construct a transparent interactive layer aligned with the base map. While playing the performance, it responds to the user's touch operation on the background ancient buildings or utensils and calls up related cultural and tourism knowledge to realize a multimedia interactive display that integrates dynamic performance and static guided tour.

[0024] The specific working principles and logical relationships of the above functional modules will be explained in detail below.

[0025] See attached document Figure 2 , Figure 2 This is a schematic diagram of the background image storage module processing flow according to an embodiment of the present invention. The background image storage module 10 is the initialization stage of the entire intangible cultural heritage tourism news low-latency image communication acquisition and display system. Its core task is to establish benchmark data for subsequent differential calculations and index data for terminal interaction.

[0026] The background image storage module 10 first controls the camera acquisition device, positioned at a fixed location, to scan the intangible cultural heritage performance site. Before the performance officially begins and the performers enter the frame area, the acquisition device acquires multiple consecutive frames of the site environment. The image processor inside the background image storage module 10 uses an inter-frame difference or Gaussian mixture model to perform stability analysis on this set of consecutive images, eliminating any possible instantaneous noise or minor disturbances, and synthesizing or selecting the image with the highest clarity and no dynamic foreground interference as the initial reference background image of the system (denoted as...). Background base map storage module 10 stores the initial reference base map. The data format is converted to a color space suitable for subsequent processing, such as the YUV color space, to separate the luminance signal from the chrominance signal.

[0027] To support the subsequent calculations of the regional illumination detection module 20, the background base map storage module 10 stores the initial reference base map. Logically, it is divided into several rectangular detection regions. Specifically, assuming an initial baseline map... The horizontal resolution is Vertical resolution is The background image storage module 10 has a preset horizontal grid number of [number missing]. The number of grid cells in the vertical direction is Then the first line, number Detection area of ​​the column The pixel range is defined as: ; In the formula, , , This indicates the floor function. and These are the x and y coordinates of the pixel, respectively. The index for "row" (the first row) OK), Index for "column" (the first column) (Column). Background map storage module 10 divides the above-mentioned area into parameters ( This is stored as a system configuration parameter to ensure that all subsequent modules calculate illumination parameters based on the same spatial reference, rather than just obtaining the global average brightness.

[0028] Based on this, the background map storage module 10 performs key scene semantic analysis and interactive data construction functions. The background map storage module 10 is configured with a pre-trained object detection neural network model. The object detection neural network model includes a feature extraction network (Backbone), a feature pyramid network (Neck), and a detection head (Head). The background map storage module 10 stores the initial baseline map... The pixel matrix is ​​fed into the object detection neural network model as input data, and high-dimensional semantic features of the image are extracted through multi-layer convolution operations. The detection head performs classification prediction and bounding box regression prediction based on the extracted feature map, thereby identifying target objects with specific cultural attributes or tourism value (such as the eaves of ancient buildings, plaques, and traditional musical instruments) in complex background textures.

[0029] Output data of neural network models The background image storage module 10 parses and defines a set of vectors containing target category probabilities and location coordinates as an interactive hotspot set. : ; In the formula, The total number of objects identified; For the first A unique identifier for the category of an object; Let be the coordinate vector of the object's position on the base map, represented as , where represents the x-coordinate of the top-left corner, y-coordinate, width, and height of the rectangular bounding box, respectively.

[0030] Subsequently, the background image storage module 10 stores the background image based on the identified target object identifier. The system performs index matching within a locally pre-built intangible cultural heritage tourism database. This database pre-stores text descriptions, historical origins, and multimedia guide links for various intangible cultural heritage tools and architectural components. The background map storage module 10 stores the aforementioned location coordinate data. The retrieved metadata is then linked to generate an additional description file independent of the image data. This additional description file uses a structured data format (such as XML or JSON) and records in detail the identifier, pixel coordinate range, and corresponding display content for each interactive target.

[0031] Finally, the background base map storage module 10 stores the processed high-definition initial reference base map. The data packet is packaged with the generated additional documentation. It is then sent to the receiving terminal device via a communication interface. The receiving end saves the initial baseline map in its local memory as the canvas for image reconstruction; simultaneously, it parses the additional documentation to prepare data for subsequently establishing a transparent touch response area on the screen that precisely corresponds to the image position.

[0032] The specific sensor control logic, color space conversion matrix operations, and basic image encoding and compression standards in the above image acquisition process can be implemented by those skilled in the art using existing mature technical solutions, which are well-known technologies in the field and will not be elaborated here.

[0033] See attached document Figure 3 , Figure 3 This is a schematic diagram of a regional illumination detection and illumination smoothing process according to an embodiment of the present invention. The regional illumination detection module 20 and the illumination smoothing processing module 30 work together to transform the complex illumination changes of the on-site environment into low-dimensional mathematical features and perform high-fidelity reconstruction.

[0034] The area illumination detection module 20 is connected to the camera acquisition device to receive the video stream of the intangible cultural heritage performance in real time. The area illumination detection module 20 extracts the luminance component (Y component) of the video frame by frame and calls the preset grid division parameters in the background base map storage module 10 to logically divide the current real-time video frame into several rectangular detection areas that are completely consistent with the initial reference base map.

[0035] To accurately quantify illumination changes, the region illumination detection module 20 constructs a linear illumination transformation model for each detection region. Specifically, for the first... line, number Detection area of ​​the column The area illumination detection module 20 extracts the set of pixel brightness of the current real-time video frame within the detection area. The set of pixel brightness of the initial reference base map The area illumination detection module 20 solves for the brightness gain coefficient of the detection area using the least squares method. and brightness offset values To minimize the linear transformation error between the two, the objective function is defined as follows: ; In the formula, This indicates that the result obtained through calculation is for the first... line, number Column detection area (i.e., region) The optimal brightness gain coefficient and brightness offset values; The parameters represent the parameters that minimize the objective function (i.e., the sum of squared errors of the linear transformation). and The possible values ​​of ; Indicates the position of the first line, number Detection area of ​​the column Summation operation is performed on all pixels within the range; This represents the current local detection region (or grid block), containing the set of coordinates of all pixels within the local detection region; Indicates the area The pixel coordinates of any pixel within the range; Indicates the coordinates of the current real-time video frame. The pixel brightness value (usually a grayscale value or the Y component) at that location; This indicates that the initial reference base map (i.e., the pre-stored static background image) is on the same coordinate system. The pixel brightness value at that location; This represents the linear gain variable to be solved during the least squares fitting calculation; This represents the linear bias variable to be solved during the least squares fitting calculation. Through the above calculations, the area illumination detection module 20 outputs the gain coefficients of all detection areas. With bias value The resulting sparse lighting parameter matrix is ​​only related to the number of grid cells, achieving an extremely high compression ratio for capturing ambient light atmosphere.

[0036] The illumination smoothing module 30 uses a bilinear interpolation algorithm to generate a pixel-level illumination adjustment map covering the entire image. For any pixel in the target image... The lighting smoothing module 30 first determines the center points of the four adjacent grids surrounding any pixel (denoted as ). ), and calculate pixel points Relative to the top left control point normalized horizontal distance and normalized vertical distance The illumination smoothing module 30 calculates the value of any pixel according to the following formula. Smoothing gain value at : ; in, This represents the brightness gain coefficient for the corresponding control point. , The illumination smoothing module 30 uses the same interpolation logic to calculate any pixel point. Smooth bias value at .

[0037] After traversing and calculating every pixel in the entire image, the lighting smoothing module 30 finally generates two floating-point mapping maps with the same resolution as the original video: one is a continuous gain map that records the lighting scaling ratio at every point in the entire image. The other image is a continuous offset map that records the illumination offset at every point in the entire image. These two images, collectively known as pixel-level lighting adjustment images, can delicately present the natural attenuation and gradation effects of ambient lighting in space, providing accurate data support for subsequently correcting the static base map into a virtual base map consistent with real-time lighting.

[0038] The extraction of image luminance components (such as RGB to YUV) and the implementation of basic matrix operations involved in the above process can be implemented by those skilled in the art using existing digital image processing libraries or hardware acceleration instruction sets. These are well-known technologies in the field and will not be elaborated here.

[0039] See attached document Figure 4 , Figure 4 This is a schematic diagram of the dynamic foreground extraction and differential transmission processing flow according to an embodiment of the present invention. The dynamic foreground extraction and differential transmission module 40 is the core processing unit for realizing low-latency and low-bandwidth communication. It is configured to eliminate background redundancy by utilizing a computationally synthesized virtual environment, thereby accurately extracting the dynamic content of the intangible cultural heritage performance.

[0040] Specifically, the dynamic foreground extraction and differential transmission module 40 utilizes a continuous gain graph. and continuous bias map For the initial base map Pixel-level linear modulation is performed to construct a virtual reference base map that matches the current lighting distribution at the performance venue. For any coordinates in the image For each pixel at a given location, the dynamic foreground extraction and differential transmission module 40 calculates its virtual brightness value according to the following formula: ; in, As the initial base map in coordinates The brightness value at that location. This represents the illumination gain coefficient corresponding to this coordinate point. This represents the illumination offset value corresponding to that coordinate point. Through this calculation, the dynamic foreground extraction and differential transmission module 40 calculates the theoretical brightness distribution that the static background should present under the current illumination parameters, making the pixel values ​​of the virtual reference base map in the background area highly approximate the actual on-site video frame.

[0041] Based on this, the dynamic foreground extraction and differential transmission module 40 acquires the current real-time video frame input from the camera acquisition device. It then executes differential extraction logic based on background culling. The dynamic foreground extraction and differential transmission module 40 compares the current real-time video frame pixel by pixel. With virtual base map The brightness difference. Since the lighting in the background area has been synchronously corrected using the above formula, the background pixel values ​​will cancel each other out and approach zero when the two are subtracted, while the areas of intangible cultural heritage performers and props that are less affected by lighting or have independent textures will retain a significant difference. The dynamic foreground extraction and differential transmission module 40 sets a preset judgment threshold. A binarized foreground mask is generated using the following discrimination logic. : ; in, Representing coordinates Foreground mask determination results at the location, Indicates the coordinates of the current real-time video frame. Pixel brightness value at that location, This represents the virtual brightness value of the virtual reference base map at the same coordinates. This is a tolerance threshold set based on the camera sensor noise level. This represents absolute value calculation, where 1 indicates the pixel is classified as dynamic foreground and 0 indicates it is classified as static background. The tolerance threshold is set to shield against background noise interference and residual errors from lighting fitting during image acquisition. Specifically, the camera's photosensitive element generates inherent thermal noise when converting light signals into electrical signals, and the aforementioned lighting adjustment calculations may exhibit slight fitting deviations at edges of drastic light and shadow changes. If the brightness fluctuations of these non-target objects exceed the threshold, the system will incorrectly identify them as dynamic foreground. Therefore, the tolerance threshold is typically set to cover the sum of the sensor's maximum noise amplitude and the peak value of the fitting residual. For general eight-bit grayscale video images, the specific value of this tolerance threshold is generally set between five percent and ten percent of the total pixel brightness level. When the shooting scene is dark and the camera's light sensitivity gain is high, resulting in increased noise, a higher value in this range is used to suppress background flicker. In contrast, in scenes with sufficient light and a high signal-to-noise ratio, a lower value in this range is used to ensure that the embroidery textures and hair strands on the costumes of intangible cultural heritage performers are preserved, thereby achieving a balance between maintaining background purity and foreground detail integrity.

[0042] The dynamic foreground extraction and differential transmission module 40 uses the generated foreground mask... For the current real-time video frame Connectivity analysis is performed to extract the minimum set of bounding rectangles covering all pixel regions marked as 1. The dynamic foreground extraction and differential transmission module 40 extracts data from the current real-time video frame. Image data within these rectangular regions is cropped out as dynamic foreground image fragments, and the coordinates of the top-left vertex of each fragment in the original image are recorded. Finally, the dynamic foreground extraction and differential transmission module 40 constructs a transmission data packet. This transmission data packet contains only the following: a sparse illumination parameter matrix (containing the grid region) generated by the area illumination detection module 20. and The data packet contains compressed data streams of dynamic foreground image fragments, along with vertex coordinates for each foreground fragment. This data packet does not contain any static background image data. In this way, the system avoids repeatedly transmitting unchanged background walls, floors, and building structures, significantly reducing bandwidth requirements while maintaining high-definition image quality.

[0043] The foreground fragment encoding (such as using macroblock encoding technology in HEVC or H.264 standards) and data packet serialization encapsulation involved in the above process can be implemented by those skilled in the art using existing multimedia communication protocol stacks, which are well-known technologies in the field and will not be elaborated here.

[0044] See attached document Figure 5 , Figure 5This is a schematic diagram of a terminal synthesis display and interactive processing flow according to an embodiment of the present invention. The terminal synthesis display module 50 is configured to reconstruct a high-fidelity scene image based on compressed data packets at the receiving end and respond to touch operations to display intangible cultural heritage tourism information.

[0045] The terminal synthesis and display module 50 receives data packets from the sending end via a communication interface. These data packets contain a sparse illumination parameter matrix generated by the region illumination detection module 20, a dynamic foreground image fragment data stream generated by the dynamic foreground extraction and differential transmission module 40, the position coordinates of the foreground fragments, and a synchronization timestamp. Simultaneously, the terminal synthesis and display module 50 reads a locally pre-stored initial baseline map and an additional explanatory file containing interactive hotspot definitions.

[0046] The terminal compositing and display module 50 analyzes the sparse illumination parameter matrix to recover the brightness gain coefficient and brightness offset values ​​of all grid regions. To eliminate the discontinuities caused by gridding, the terminal compositing and display module 50 performs the same spatial smoothing interpolation operation as the transmitting end, reconstructing a continuous gain map and a continuous offset map covering the entire screen in local memory, thereby obtaining the accurate illumination gain value and illumination offset value corresponding to all pixels on the screen.

[0047] Based on this, the terminal compositing and display module 50 performs pixel-level compositing of the final display frame. The terminal compositing and display module 50 decodes dynamic foreground image fragments to obtain a foreground pixel set; simultaneously, it modulates the initial reference base map with illumination to generate background pixels. To clarify the compositing logic, the terminal compositing and display module 50 performs a layer priority-based numerical fill operation on all pixel coordinates in the target display image. Specifically, the terminal compositing and display module 50 determines whether the current pixel coordinates are within the coordinate range of any dynamic foreground image fragment. If the determination result is yes, the terminal compositing and display module 50 directly extracts the pixel value of the decoded foreground fragment at the corresponding position as the display value of the current pixel; if the determination result is no, the terminal compositing and display module 50 performs background linear modulation calculation, that is, multiplying the original brightness value of the initial reference base map at the current coordinates by the illumination gain value corresponding to the current coordinates, then adding the illumination bias value corresponding to the current coordinates, and using the calculated result as the display value of the current pixel. Through the above logic, the terminal compositing and display module 50 achieves seamless stitching of dynamic foreground content and a virtual background corrected by real-time illumination, completing high-fidelity image restoration without transmitting background image data.

[0048] While reconstructing the visual image, the terminal synthesis and display module 50 loads the additional specification file received during the initialization phase and constructs the interactive response logic. The terminal synthesis and display module 50 parses the set of interactive hotspots recorded in the specification file, which includes a unique identifier for each intangible cultural heritage target object and the bounding box parameters defined by the coordinates of the top-left vertex, width, and height.

[0049] When the terminal synthesis and display module 50 detects a screen touch event, it immediately obtains the horizontal and vertical coordinates of the touch point. The terminal synthesis and display module 50 iterates through the set of interactive hotspots, performing a geometric inclusion determination on the touch point. Specifically, the terminal synthesis and display module 50 sequentially determines whether the horizontal coordinate of the touch point is greater than or equal to the horizontal coordinate of the top-left corner of the target object and less than or equal to the sum of the horizontal coordinate of the top-left corner and the width; simultaneously, it determines whether the vertical coordinate of the touch point is greater than or equal to the vertical coordinate of the top-left corner of the target object and less than or equal to the sum of the vertical coordinate of the top-left corner and the height. If both conditions are met, the terminal synthesis and display module 50 determines that the interaction has been hit and retrieves intangible cultural heritage metadata from the local database based on the unique identifier of the target object. The terminal synthesis and display module 50 then calls the graphics rendering interface to overlay and render a floating display layer containing text descriptions or multimedia content on top of the current display frame.

[0050] The data packet decompression (such as JSON parsing and entropy decoding), graphics rendering pipeline (such as OpenGL or Vulkan interface calls), and operating system event listening mechanisms involved in the above process can be implemented by those skilled in the art using existing mobile development frameworks or graphics processing libraries. These are well-known technologies in the field and will not be elaborated here.

[0051] See attached document Figure 6 , Figure 6 This is a flowchart illustrating a low-latency image communication acquisition, editing, and display method for intangible cultural heritage tourism news according to an embodiment of the present invention. The present invention provides a low-latency image communication acquisition, editing, and display method for intangible cultural heritage tourism news, comprising the following steps: S10: Control the camera acquisition device to acquire images of the on-site environment, and generate a clear initial baseline map after stability analysis. Divide the initial baseline map into several logical detection areas, and use a target detection neural network to identify intangible cultural heritage target objects in the baseline map, generating an interactive description file containing the target object's position coordinates and related information. Send the initial baseline map and interactive description file to the receiving end for storage.

[0052] S20: Real-time acquisition of the live video stream and extraction of luminance components. For all logical detection areas, the linear luminance transformation relationship between the current video frame and the initial reference base map is calculated. Through error minimization calculation, a sparse illumination parameter matrix composed of luminance gain coefficients and luminance bias values ​​is obtained.

[0053] S30 performs spatial interpolation on the sparse illumination parameter matrix, mapping the discrete grid data into a continuous gain map and a continuous bias map covering the entire screen, in order to quantify the spatial distribution characteristics of ambient illumination at the pixel level.

[0054] S40: Linearly modulate the initial reference base map using a continuous gain map and a continuous bias map to generate a virtual reference base map consistent with the current real-time illumination. Calculate the pixel difference between the current real-time video frame and the virtual reference base map, and extract dynamic foreground image fragments based on the difference threshold. Construct the sparse illumination parameter matrix, dynamic foreground image fragments, and their position information into a transmission data packet and send it to the receiving end.

[0055] S50: The receiving end parses the transmitted data packets, reconstructs the continuous gain map and continuous offset map locally, and renders the locally stored initial reference base map as a virtual background. Decoded dynamic foreground image fragments are superimposed onto the corresponding coordinate positions of the virtual background to synthesize the final high-fidelity display. Simultaneously, it monitors screen touch events and responds to user operations according to the coordinate definitions in the interaction specification file, displaying corresponding intangible cultural heritage tourism information.

Claims

1. A low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news, characterized in that: include: The background image storage module is configured to acquire static environmental images of the performance venue as initial reference images and pre-send them to the receiving end for storage, while recognizing target entities in the background image to generate interactive instruction files containing coordinates and navigation information; The regional illumination detection module is connected to the background base map storage module and is configured to perform partition analysis on the real-time video image, calculate the linear transformation relationship of brightness of each region relative to the initial reference base map, and generate a sparse illumination parameter matrix characterizing the changes in light and shadow on site. The illumination smoothing module is connected to the regional illumination detection module and is configured to perform spatial interpolation transition processing on the sparse illumination parameter matrix to map and generate a continuous pixel-level illumination adjustment map covering the entire screen. The dynamic foreground extraction and differential transmission module is connected to the background base map storage module and the illumination smoothing processing module, respectively. It is configured to use the continuous pixel-level illumination adjustment map to modulate the initial reference base map to construct a virtual reference base map, separate dynamic foreground image fragments based on the difference between the real-time video and the virtual reference base map, and construct a transmission data packet containing a sparse illumination parameter matrix and dynamic foreground image fragments to send to the receiving end. The terminal synthesis and display module, located at the receiving end, is configured to re-render the local initial reference base map using the received sparse illumination parameter matrix, and superimpose the dynamic foreground image fragments to synthesize the display screen, while responding to touch interaction according to the interaction specification document.

2. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, When generating the interactive description file, the background map storage module is configured to perform the following operations: use a pre-trained target detection neural network model to extract features and predict the classification of the initial baseline map, identify target objects with specific attributes and obtain their bounding box coordinates; retrieve associated cultural and tourism metadata based on the identified target object identifier, and bind the bounding box coordinates with the cultural and tourism metadata to generate the interactive description file.

3. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, When generating the sparse illumination parameter matrix, the region illumination detection module is configured to perform the following operations: for all logical detection regions, extract the pixel brightness set of the current real-time video frame and the pixel brightness set of the initial reference base map; use the least squares method to solve the brightness gain coefficient and brightness bias value of the region, so as to minimize the linear transformation error between the two sets of pixel brightness sets in the region.

4. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, The continuous pixel-level illumination adjustment map generated by the illumination smoothing processing module includes a continuous gain map and a continuous bias map; the illumination smoothing processing module is configured to: for any target pixel in the entire image, determine the center point of the adjacent logical detection region surrounding the target pixel, and use a bilinear interpolation algorithm to calculate the smoothing gain value and smoothing bias value of the target pixel according to the distance weight.

5. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 4, characterized in that, When generating a virtual reference base map, the dynamic foreground extraction and differential transmission module is configured to perform the following operations: for all pixels in the initial reference base map, obtain their original brightness values, multiply them by the smoothed gain value of the corresponding coordinate in the continuous gain map, and accumulate the smoothed offset value of the corresponding coordinate in the continuous offset map. The calculated result is used as the virtual brightness value of the virtual reference base map at that pixel.

6. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, When extracting dynamic foreground image fragments, the dynamic foreground extraction and differential transmission module is configured to perform the following operations: calculate the absolute value of the brightness difference between the current real-time video frame and the virtual reference base map at the same coordinates; compare the absolute value with a preset tolerance threshold; if the absolute value is greater than the tolerance threshold, determine that the pixel is a dynamic foreground, otherwise determine that it is a static background; generate a binarized foreground mask based on the determination result, and crop out the image region containing the dynamic foreground pixel from the current real-time video frame.

7. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, The transmitted data packet contains only the sparse illumination parameter matrix, the compressed data stream of dynamic foreground image fragments, and the vertex coordinate information of the foreground fragments, and does not contain any static background image data.

8. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, When restoring the display screen, the terminal synthesis and display module is configured to perform the following operations: reconstruct a continuous pixel-level illumination adjustment map locally at the receiving end; determine whether all pixel coordinates of the target display image are within the location range of the dynamic foreground image fragment; if so, extract the pixel value of the foreground fragment at that location as the display value; if not, use the reconstructed continuous pixel-level illumination adjustment map to perform linear adjustment calculations on the locally stored initial reference base map, and use the calculation result as the display value.

9. The low-latency image communication acquisition, editing, and display system for intangible cultural heritage tourism news according to claim 1, characterized in that, When responding to a touch operation, the terminal synthesis and display module is configured to perform the following operations: monitor screen touch events and obtain the coordinates of the touch point; traverse the bounding box parameters of the target object recorded in the interaction specification file and determine whether the coordinates of the touch point fall within the range of any target object; If a match is found, the cultural and tourism metadata associated with the target object is invoked and displayed.

10. A method for low-latency image communication acquisition, editing, and display of intangible cultural heritage tourism news, as described in any one of claims 1-9, characterized in that, Includes the following steps: The system controls the camera acquisition device to obtain the initial reference base map, generates an interactive specification file containing information about the target object, and sends the initial reference base map and the interactive specification file to the receiving end. The video stream is acquired in real time and divided into regions. The linear transformation relationship of brightness of each region relative to the initial reference base map is calculated to generate a sparse illumination parameter matrix. Spatial interpolation is performed on the sparse lighting parameter matrix to generate a continuous pixel-level lighting adjustment map covering the entire image; The initial reference base map is corrected using a continuous pixel-level illumination adjustment map to generate a virtual reference base map. The difference between the real-time video and the virtual reference base map is calculated to extract dynamic foreground image fragments. A transmission data packet containing a sparse illumination parameter matrix and dynamic foreground image fragments is constructed and sent. The receiving end parses the transmitted data packets, uses lighting parameters to render the local base map, and overlays dynamic foreground image fragments to synthesize the display screen. At the same time, it responds to the user's touch operations according to the interaction specification document.

Images