An e-commerce live broadcast holographic imaging device based on environmental interference resistance

Abstract

The application discloses an e-commerce live broadcast holographic imaging device based on environmental interference resistance, and relates to the technical field of e-commerce live broadcast holographic imaging.The device comprises a data preprocessing module, a decoupling processing module, a fusion output module and a state feedback module.The data preprocessing module constructs a dynamic data grid, receives and synchronizes original input streams of imaging sensors, and generates a mixed feature tensor;the decoupling processing module inputs the mixed feature tensor into a cyclic evolution decoupling network, the network comprises interwoven but parameter-independent illumination, structure and anti-noise paths, and outputs three types of feature vectors after multiple rounds of iteration;the fusion module combines the three types of vectors by weighting, and generates an anti-interference imaging data block;and the state feedback module outputs the data block to a holographic projection module, and simultaneously writes back key feature updates to the dynamic data grid.The application can realize three types of feature separation and extraction, optimize imaging clarity and restoration, enhance environmental interference adaptability, and ensure holographic imaging stability in e-commerce live broadcast scenarios.

Description

Technical Field

[0001] This invention belongs to the field of e-commerce live streaming holographic imaging technology, specifically an e-commerce live streaming holographic imaging device based on resistance to environmental interference. Background Technology

[0002] In e-commerce live streaming scenarios, holographic imaging devices need to output clear and stable imaging effects in complex environments to meet the core requirements of product display and interactive presentation. Existing e-commerce live streaming holographic imaging devices typically use a single-path feature extraction network to process the raw input stream collected by the imaging sensor, jointly extracting illumination features, structural features, and noise features during the imaging process. After extraction, fused feature data is directly generated to drive the holographic projection module to form an image. At the same time, existing devices mostly use a static data caching mechanism to temporarily store the raw input stream, only for temporarily storing data to be processed. After processing, only the final imaging data is output, without subsequent updating and reuse of the cached data.

[0003] Existing single-path feature extraction networks cannot independently separate illumination, structure, and noise features. These features overlap and interfere with each other during extraction, resulting in redundant information in the extracted illumination features, loss of structural details, and ineffective filtering of noise features, thus affecting the clarity and fidelity of holographic imaging. Static data caching mechanisms mean that each imaging process is based solely on the initial raw input stream, making it impossible to utilize imaging-related features from the previous processing cycle to optimize the current process. This leads to poor adaptability of the device to environmental interference such as sudden changes in illumination and noise fluctuations in e-commerce live streaming scenarios, making it difficult to continuously output stable holographic imaging effects and failing to meet the core requirement of environmental interference resistance for holographic imaging devices in e-commerce live streaming scenarios. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; To this end, the present invention proposes a holographic imaging device for e-commerce live streaming based on resistance to environmental interference, comprising: The data preprocessing module constructs an initialized dynamic data grid, receives and temporarily stores the raw input stream from the imaging sensor, performs synchronous processing on the raw input stream in the dynamic data grid, and generates a hybrid feature tensor. The decoupling processing module inputs the hybrid feature tensor into a cyclically evolving decoupling network. The decoupling network includes intertwined but parameter-independent illumination, structural, and noise suppression paths. Within the decoupling network, through multiple rounds of iterative data exchange, the illumination path outputs an illumination purification vector, the structural path outputs a structural enhancement vector, and the noise suppression path outputs a noise suppression vector. The fusion output module weights and combines the illumination purification vector, structure enhancement vector, and noise suppression vector according to a preset fusion rule to generate the final anti-interference imaging data block for this processing cycle. The state feedback module outputs the final anti-interference imaging data block to the holographic projection module for imaging, and simultaneously writes back the key features to the corresponding area of ​​the dynamic data grid to update the initial state of the next processing cycle.

[0005] Furthermore, the original input stream in the dynamic data grid is synchronously processed to generate a hybrid feature tensor, including: After the holographic imaging device is started, an initial dynamic data grid is constructed to receive and temporarily store the raw input stream from the imaging sensor in real time. The original input stream in the dynamic data grid is synchronously processed, and each frame of input data is decomposed into illumination component stream, structure component stream and noise component stream. Create an independent illumination history sequence to continuously record the temporal evolution trajectory of the illumination component stream; The topological skeleton information representing the contour of the live object is extracted from the structural component stream, and this topological skeleton information is mapped to a dedicated topological feature space. For the noise component stream, an online adaptive noise spectrum analysis window is activated to calculate the distribution pattern of noise energy in the frequency domain in real time; Within each processing cycle, the structural components of the current frame are extracted from the dynamic data grid and correlated with the historical illumination sequence and real-time noise spectrum to generate a hybrid feature tensor containing multidimensional contextual information.

[0006] Furthermore, the raw input stream in the dynamic data grid is synchronized, including: A synchronization controller is assigned to the dynamic data grid, and the synchronization controller generates timing pulses according to the sampling frequency of the imaging sensor; At the effective edge of each timing pulse, the synchronization controller locks a fixed-size data block in the dynamic data grid, the data block containing the latest, unprocessed raw input data at the current moment; The locked data block is copied to three parallel processing pipelines; In the first processing pipeline, a brightness separation operator based on logarithmic domain transformation is applied to extract the components that mainly reflect changes in ambient light intensity from the copied data blocks, forming the light component stream; In the second processing pipeline, a multi-scale edge detection and contour connection algorithm is applied to extract components reflecting the geometry and surface texture of the live object from the copied data block, forming the structural component stream; In the third processing pipeline, differential operations are applied between the original data block and the data block that has undergone preliminary processing in the first and second pipelines, and combined with high-frequency component filtering, to obtain the residual random fluctuation components, forming the noise component stream.

[0007] Further, topological skeleton information characterizing the contour of the live-streaming object is extracted from the structural component stream, including: For each frame of the binary contour map in the structural component stream, perform a morphological thinning operation until the contour line width is one pixel to obtain the initial skeleton map. On the initial skeleton map, the endpoints and intersections of all branches are identified, and a continuous chain of pixels connecting two key points is defined as a skeleton segment. Calculate the geometric properties of each skeleton segment, including length, orientation, average curvature, and area of ​​the original contour region it covers; Establish a hierarchical relationship tree for skeleton segments. The root node of the tree is the main skeleton segment of the object, and the child nodes are the secondary skeleton segments branched from the main segment. All skeleton segments, their geometric attributes, and hierarchical relationships are encoded into a fixed-dimensional feature vector, which serves as the topological skeleton information and is then fed into the dedicated topological feature space.

[0008] Furthermore, within each processing cycle, the structural components of the current frame are extracted from the dynamic data grid and correlated with the historical illumination sequence and real-time noise spectrum to generate a hybrid feature tensor containing multi-dimensional contextual information, including: The feature vector of topological skeleton information is extracted from the current frame structure components of the dynamic data grid; From the illumination history sequence, the moving average of the illumination purification vector for the most recent several consecutive periods is read as the historical illumination context; From the online adaptive noise spectrum analysis window, obtain the pattern code of the noise energy distribution at the current moment, which identifies the type of dominant noise; The feature vector of the topological skeleton information, the historical illumination context, and the pattern encoding of the noise energy distribution are concatenated to form a one-dimensional joint feature vector. The one-dimensional joint feature vector is reshaped according to a preset dimensional rule to form a three-dimensional hybrid feature tensor, where the three dimensions correspond to the abstract feature channels of structure, illumination history and noise spectrum, respectively.

[0009] Furthermore, the decoupling processing module is configured to generate the output vectors of each path in the following manner: At the beginning of each iteration of the decoupled network, the hybrid feature tensor is simultaneously fed into the input layer of the illumination path, the structure path, and the noise reduction path. Each path performs a forward propagation computation once, generating the intermediate potential representation of the corresponding path in the current iteration round; At the end of each iteration, a cross-path attention exchange module is set up. The cross-path attention exchange module calculates the attention weight of the intermediate representation of the illumination path to the intermediate representation of the structure path, calculates the attention weight of the intermediate representation of the structure path to the intermediate representation of the anti-noise path, and calculates the attention weight of the intermediate representation of the anti-noise path to the intermediate representation of the illumination path. Based on the calculated attention weights, the intermediate latent representations of the three pathways are fused and exchanged using information weighting to generate an enhanced input for each pathway for the next iteration. The forward propagation calculation is performed once for each of the illumination path, structure path, and noise reduction path, and an iterative process of information weighted fusion and exchange is carried out in the cross-path attention exchange module until the preset number of iterations is reached. After the final iteration round is completed, the illumination purification vector is read from the output layer of the illumination path, the structure enhancement vector is read from the output layer of the structure path, and the noise suppression vector is read from the output layer of the noise reduction path.

[0010] Furthermore, the specific operation of the cross-path attention exchange module is as follows: The intermediate latent representations of the illumination path, the structural path, and the noise-resistant path are respectively used as the first feature matrix, the second feature matrix, and the third feature matrix. Calculate the product of the first feature matrix and the transpose of the second feature matrix, and input the product result into the normalization function to obtain the first attention weight matrix. The first attention weight matrix represents the association weight of illumination information with structural information. Calculate the product of the second feature matrix and the transpose of the third feature matrix, and input the product result into the normalization function to obtain the second attention weight matrix. The second attention weight matrix represents the association weight of structural information with noise information. Calculate the product of the third feature matrix and the transpose of the first feature matrix, and input the product result into the normalization function to obtain the third attention weight matrix, which represents the association weight of noise information with illumination information. The second feature matrix is ​​weighted and fused using the first attention weight matrix, and the fusion result is added to the first feature matrix to generate an enhanced intermediate representation of the illumination path for the next iteration. The third feature matrix is ​​weighted and fused using the second attention weight matrix, and the fusion result is added to the second feature matrix to generate an enhanced intermediate representation of the structural path for the next iteration. The first feature matrix is ​​weighted and fused using the third attention weight matrix, and the fusion result is added to the third feature matrix to generate an enhanced intermediate representation for the next iteration of the noise-resistant path.

[0011] Furthermore, the illumination purification vector, structure enhancement vector, and noise suppression vector are weighted and combined according to a preset fusion rule, including: A dynamic weight is assigned to the illumination purification vector, which is obtained by mapping the absolute value of the difference between the average intensity of the current frame illumination component stream and the average intensity of the historical illumination sequence through a monotonically increasing function. Assign a fixed weight to the structure enhancement vector; A dynamic weight is assigned to the noise suppression vector. This dynamic weight is obtained by mapping the proportion of high-frequency noise energy to the total noise energy in the current noise spectrum analysis window through another monotonically increasing function. Ensure that the sum of the dynamic weights of the illumination purification vector, the fixed weights of the structure enhancement vector, and the dynamic weights of the noise suppression vector is one. Multiply the illumination purification vector by its dynamic weight, the structure enhancement vector by its fixed weight, and the noise suppression vector by its dynamic weight. The three weighted vectors are added element by element to obtain the vector representation of the final anti-interference imaging data block.

[0012] Furthermore, the final anti-interference imaging data block is output to the holographic projection module for imaging, and simultaneously, the key features are written back to the corresponding area of ​​the dynamic data grid, including: From the final anti-interference imaging data block, the spatially corresponding block is extracted; Calculate the difference map between the parsed data block and the old data at the corresponding position in the current dynamic data grid; A low-pass filter is applied to the difference map to remove abrupt differences at high frequencies and retain gradual changes at low frequencies. The filtered difference plot is weighted and averaged with the old data in the current dynamic data grid, where the weight of the old data decays over time and the weight of the new differences increases over time. The weighted average result is used to update the corresponding data region in the dynamic data grid, which serves as the input basis for the synchronous processing step in the next processing cycle.

[0013] Furthermore, the construction of the initialized dynamic data grid includes: Based on the resolution of the imaging sensor, a three-dimensional data buffer is allocated in memory, with its width and height matching the sensor resolution and its depth being the preset number of channels. Initialize all elements of the three-dimensional data buffer to zero; Define a virtual mesh structure that covers the three-dimensional data buffer. Each cell of the virtual mesh is associated with a two-dimensional position index of the buffer and contains pointers to all depth channel data of the corresponding column in the buffer. The virtual grid structure is equipped with a circular indexing mechanism, which allows newly written data to overwrite old data in chronological order and maintains a fixed-length historical context window.

[0014] Compared with the prior art, the beneficial effects of the present invention are: A decoupled network with cyclic evolution is employed, consisting of interwoven but parameter-independent illumination, structure, and noise suppression paths. Through multiple rounds of iterative data exchange, each path outputs an illumination purification vector, a structure enhancement vector, and a noise suppression vector, respectively. The parameter independence of the three paths enables the separate extraction of illumination, structure, and noise features, fundamentally avoiding mutual interference between these features during the extraction process. The illumination path outputs a vector retaining only pure illumination information, the structure path accurately extracts the structural features of the imaging target and forms an enhancement vector, and the noise suppression path effectively separates noise components and outputs a corresponding suppression vector. This significantly optimizes the illumination reproduction, structural clarity, and noise filtering effect of the imaging data.

[0015] An initial dynamic data grid is constructed to temporarily store the raw input stream of the imaging sensor. Simultaneously, while outputting the final anti-interference imaging data block to the holographic projection module, key features from the data block are written back to the corresponding area of ​​the dynamic data grid, thus updating the initial state for the next processing cycle. The closed-loop iterative update mode of the dynamic data grid ensures that the imaging processing in each cycle is based on the effective imaging features of the previous cycle. This allows the imaging device to continuously adapt to environmental changes during e-commerce live streaming while processing continuous raw input streams, gradually optimizing the initial data foundation for imaging processing. This ensures that the continuously output imaging data blocks maintain good continuity and consistency, and that the output state of the holographic imaging remains stable even under sudden changes in ambient light and fluctuations in ambient noise. Attached Figure Description

[0016] Figure 1 This is a timing diagram of an e-commerce live streaming holographic imaging device based on anti-environmental interference, as described in this invention. Figure 2 A flowchart for extracting topological skeleton information; Figure 3 A flowchart for generating association and concatenation of hybrid feature tensors; Figure 4 This is a diagram showing the changes in the feature dimensions of each pathway during the decoupling network iteration process; Figure 5 This diagram illustrates the weight update mechanism of the dynamic data grid in the state feedback module. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 The data preprocessing module constructs an initialized dynamic data grid to receive and temporarily store the raw input stream from the imaging sensor. The preprocessing module performs synchronous processing on the raw input stream in the dynamic data grid, transforming it into a hybrid feature tensor containing multi-dimensional information. The decoupling processing module receives this hybrid feature tensor and feeds it into a cyclically evolving decoupling network. This decoupling network includes an illumination path, a structure path, and a noise suppression path; these three paths are intertwined but their parameters are independent. Within the network, through multiple iterations and data exchanges over a preset number of rounds, the illumination path outputs an illumination purification vector, the structure path outputs a structure enhancement vector, and the noise suppression path outputs a noise suppression vector. The fusion output module, according to preset fusion rules, dynamically and with fixed weights combines these three vectors to generate the final anti-interference imaging data block for this processing cycle. The state feedback module outputs this final anti-interference imaging data block to the holographic projection module to drive imaging, and simultaneously writes back the key features of this data block to the corresponding region of the dynamic data grid to update the grid state, providing an initial input basis for the calculation of the next processing cycle.

[0019] In one embodiment of the present invention, during the specific implementation after the holographic imaging device is activated, the system constructs an initialized dynamic data grid in memory. This dynamic data grid is used to receive and temporarily store the raw input stream containing the live scene acquired by the imaging sensor in real time. The imaging sensor outputs an RGB image data stream with a resolution of 1920*1080 at a rate of 60 frames per second. The dynamic data grid allocates a corresponding buffer to temporarily store the image data of the most recent 5 frames. The step of synchronizing the raw input stream in the dynamic data grid is managed by a dedicated synchronization controller. The synchronization controller generates periodic timing pulses according to the 60Hz sampling frequency of the imaging sensor. At the rising edge of each timing pulse, the synchronization controller locks a fixed-size data block in the dynamic data grid. This data block contains the latest, unprocessed raw input data at the current moment, and its size is consistent with the data volume of one frame of image. Subsequently, the synchronization controller copies the locked data block to three parallel processing pipelines, and the three pipelines start working simultaneously.

[0020] In the first processing pipeline, a luminance separation operator based on logarithmic domain transformation is applied to extract the illumination component stream from the copied data block. The luminance separation operator converts the data from the RGB color space to the HSL color space and extracts the luminance component L. A logarithmic transformation is applied to the luminance component L to compress the dynamic range of bright areas and enhance the contrast of dark details. The transformed luminance channel data forms the illumination component stream reflecting changes in ambient light intensity. In the second processing pipeline, a multi-scale edge detection and contour connection algorithm is applied to extract the structure component stream from the same copied data block. The algorithm first smooths the image using Gaussian kernels with different standard deviations, then calculates the gradient magnitude and direction at each scale. Through non-maximum suppression and double-threshold hysteresis, multi-scale edge information is fused to finally obtain a binary contour map reflecting the geometry and surface texture of the live object, forming the structure component stream. In the third processing pipeline, the noise component stream is extracted by differential operation between the original data block and the data block after preliminary processing by the first and second pipelines. Specifically, the brightness field reconstructed by the illumination component stream and the contour skeleton reconstructed by the structure component stream are subtracted from the original RGB data block, and then the residual signal is subjected to high-frequency bandpass filtering to filter out the random fluctuation components mainly composed of sensor thermal noise and random electromagnetic interference in the environment, thus forming the noise component stream.

[0021] In one operational example, the imaging sensor captures a frame containing an image of the anchor holding a product. This frame's data is locked and copied in the raw input stream by the synchronization controller. In the first pipeline, the logarithmically transformed brightness data shows an illumination value of 0.85 for the background window area and 0.60 for the anchor's face area, forming the illumination component stream. In the second pipeline, multi-scale edge detection successfully extracts the product outline and the anchor's arm outline, connecting them to form a complete structural boundary, forming the structure component stream. In the third pipeline, after subtracting the illumination and structural information from the raw data, the residual signal undergoes high-frequency filtering, displaying a uniformly distributed, finely granular pattern, forming the noise component stream.

[0022] An independent illumination history sequence is created to continuously record the temporal evolution of the illumination component stream. This sequence stores the statistical characteristics of the most recent 100 frames of the illumination component stream in a first-in, first-out (FIFO) queue structure, such as the global average brightness value for each frame. For the noise component stream, an online adaptive noise spectrum analysis window is initiated. This window performs a Fast Fourier Transform on each frame of the noise component stream, calculating the distribution pattern of noise energy in the frequency domain in real time, such as calculating the energy proportions of the 0-10Hz low-frequency band, the 10-30Hz mid-frequency band, and the high-frequency band above 30Hz. Extracting the topological skeleton information representing the contour of the live-streamed object from the structural component stream involves subsequent processing, and this topological skeleton information is mapped to a dedicated topological feature space. In practice, the changes in the average illumination value recorded in the illumination history sequence can reflect the process of sudden indoor lighting activation, while the noise spectrum analysis window can identify periodic low-frequency noise patterns caused by air conditioning activation.

[0023] In one embodiment of the present invention, see [reference] Figure 2 The process involves extracting topological skeleton information representing the contour of a live-stream object from the structural component stream. This is done by unfolding a binarized contour map of each frame provided by the structural component stream. In the binarized contour map, white pixels represent the object contour, and black pixels represent the background. Morphological thinning operations are performed on each frame of the binarized contour map in the structural component stream, repeatedly removing contour boundary pixels until the contour line width is only one pixel, resulting in an initial skeleton map composed only of lines with a single pixel width. In some embodiments, the live-stream object is a handheld lipstick; after thinning, the lipstick's elongated shape results in an initial skeleton map that appears as an approximately straight central axis.

[0024] Identify the endpoints and intersections of all branches on the initial skeleton map. An endpoint is defined as a pixel with only one adjacent skeleton pixel, and an intersection is defined as a pixel with three or more adjacent skeleton pixels. Define a continuous chain of pixels connecting two keypoints as a skeleton segment. Calculate the geometric properties of each skeleton segment, including segment length, segment orientation, average curvature, and the area of ​​the original contour region covered by the segment. The segment length is obtained by counting the number of pixels contained in the segment; the segment orientation is obtained by calculating the tilt angle of the line connecting the two endpoints of the segment; the average curvature is obtained by calculating and averaging the rate of change of the local tangent direction at each point on the segment; the area of ​​the original contour region covered by the segment is obtained by mapping back to the original binary contour map and counting all contour pixels within a certain threshold range from the pixels of the skeleton segment. Calculate the average curvature of the skeleton segment. One alternative approach is described by the following relation: ; in: This represents the total number of pixels on the skeleton segment. Representative at the The estimated local tangent direction angles at each pixel are summed to calculate the absolute values ​​of the directional changes between adjacent points, and the average value reflects the overall curvature of the skeleton segment.

[0025] A hierarchical tree of skeleton segments is established, with the root node representing the main skeleton segment of the object and child nodes representing secondary skeleton segments branching from the main segment. In some embodiments, for a human skeleton, the central axis of the torso is determined as the main skeleton segment, and the skeleton lines of the arms and head extending from the torso are established as secondary skeleton segments and linked to the root node as child nodes. All skeleton segments, their geometric attributes, and hierarchical relationships are encoded into a fixed-dimensional feature vector. This fixed-dimensional feature vector serves as the topological skeleton information and is fed into a dedicated topological feature space. In a specific implementation, the encoding process may involve concatenating the length, direction, curvature, area attributes of each skeleton segment, as well as its depth and parent node index in the tree structure, into a one-dimensional array in a predetermined order.

[0026] In one embodiment of the present invention, see [reference] Figure 3The process of generating a hybrid feature tensor within each processing cycle begins with extracting the structural components of the current frame from the dynamic data grid. These structural components are binarized contour maps after synchronous processing. Feature vectors of topological skeleton information are extracted from the current frame structural components of the dynamic data grid. These feature vectors are fixed-dimensional arrays obtained by performing morphological thinning, keypoint recognition, skeleton segment attribute calculation, and hierarchical encoding on the current frame structural components. In an example scenario, the live-streaming object is a water glass. The extracted feature vectors of the topological skeleton information encode the cylindrical skeleton segments of the glass's main body, the curved skeleton segments of the handle, and their length, curvature, and area attributes.

[0027] The historical lighting context is derived by reading the moving average of the lighting purification vectors from the most recent consecutive cycles in the lighting history sequence. The lighting history sequence stores the lighting purification vectors output by the decoupled network lighting path in past processing cycles. When reading the moving average, a fixed-length window is defined, and the mean of all lighting purification vectors within the window is calculated. The historical lighting context reflects recent trends in the lighting environment; for example, as the broadcaster moves from a darker location to in front of the lightbox, the historical lighting context will show a sequence of increasing brightness values. The moving average of the lighting purification vectors... The calculation can be expressed as: ; in: Represents the size of the sliding window. The index representing the current processing cycle. Representing the The illumination purification vectors output by the illumination path in each historical processing cycle were summed element-wise within the window to obtain the average vector. .

[0028] The pattern code for the noise energy distribution at the current moment is obtained from an online adaptive noise spectrum analysis window, which continuously performs spectral analysis on the noise component stream. The pattern code identifies the type of dominant noise and can be a discrete category label or a multi-dimensional feature vector used to distinguish different types such as "high-frequency random noise," "low-frequency periodic hum," or "impulse noise." In some embodiments, when a fan is running in the environment, the noise spectrum analysis window outputs the pattern code corresponding to low-frequency periodic noise.

[0029] The feature vector of the topological skeleton information, the historical illumination context, and the pattern encoding of the noise energy distribution are concatenated to form a one-dimensional joint feature vector. The concatenation operation combines the values ​​of the three parts end-to-end in a predetermined order to form a longer one-dimensional array. This concatenation method can be understood as integrating heterogeneous information from object structure, illumination history, and noise environment. The one-dimensional joint feature vector is then reshaped according to a preset dimensionality rule to form a three-dimensional hybrid feature tensor. The preset dimensionality rule defines the specific dimensions of the three-dimensional hybrid feature tensor in height, width, and number of channels. In a specific implementation, if the joint feature vector length is 768, the preset dimensionality rule can reshape it into a three-dimensional hybrid feature tensor of size 16x16x3, where the three channel dimensions correspond to the abstract feature channels of structure, illumination history, and noise spectrum, respectively. Optionally, the reshaped three-dimensional hybrid feature tensor can be directly used as input to the subsequent decoupling processing module.

[0030] In one embodiment of the invention, the decoupling processing module operates based on a cyclically evolving decoupling network structure. The decoupling network comprises interwoven but parameter-independent illumination, structural, and noise-resistant paths. At the start of each iteration of the decoupling network, a hybrid feature tensor generated by the data preprocessing module is simultaneously fed into the input layers of the illumination, structural, and noise-resistant paths. Each path performs a complete forward propagation computation, which includes a combination of linear transformation layers, activation function layers, and normalization layers, generating the intermediate latent representation for the corresponding path in the current iteration. For example, when the hybrid feature tensor dimension is 16x16x3, a single forward propagation of the illumination path may output an intermediate latent representation with a dimension of 8x8x64, the structural path may output an intermediate latent representation with a dimension of 8x8x64, and the noise-resistant path may output an intermediate latent representation with a dimension of 8x8x64.

[0031] At the end of each iteration, a cross-path attention exchange module is set up to perform information fusion. The cross-path attention exchange module uses the intermediate latent representations of the illumination path, the structure path, and the noise-resistant path as the first, second, and third feature matrices, respectively. These feature matrices are flattened into two-dimensional matrices before entering the attention computation, where rows represent spatial or feature locations and columns represent feature channels. See Table 1 for an example of the dimensionality of the feature matrices in one iteration round.

[0032] Table 1: Example of feature matrix dimensions in cross-path attention exchange module The product of the first feature matrix and the transpose of the second feature matrix is ​​calculated, and the result is input into a normalization function to obtain the first attention weight matrix. This first attention weight matrix represents the association weight of illumination information with structural information. The product of the second feature matrix and the transpose of the third feature matrix is ​​also calculated, and the result is input into a normalization function to obtain the second attention weight matrix. This second attention weight matrix represents the association weight of structural information with noise information. The product of the third feature matrix and the transpose of the first feature matrix is ​​calculated, and the result is input into a normalization function to obtain the third attention weight matrix. This third attention weight matrix represents the association weight of noise information with illumination information. The normalization function is typically the Softmax function, ensuring that the sum of each row or column of the weight matrix is ​​1. (First attention weight matrix) One calculation method can be described as: ; in: This represents the query matrix obtained by linear projection of the first feature matrix (light path representation). The key matrix represents the linear projection of the second characteristic matrix (structural path representation). Represents the dimension of the key vector. The function is normalized along the last dimension to obtain the weight matrix. .

[0033] The second feature matrix is ​​weighted and fused using a first attention weight matrix. The result of the weighted fusion is then added element-wise to the original first feature matrix to generate an enhanced intermediate representation for the illumination path used in the next iteration. The third feature matrix is ​​weighted and fused using a second attention weight matrix. The result of the weighted fusion is then added element-wise to the original second feature matrix to generate an enhanced intermediate representation for the structure path used in the next iteration. The first feature matrix is ​​weighted and fused using a third attention weight matrix. The result of the weighted fusion is then added element-wise to the original third feature matrix to generate an enhanced intermediate representation for the noise-resistant path used in the next iteration. In some embodiments, after attention-weighted fusion, the features representing product edges in the structure path are strengthened in the enhanced representation of the illumination path, reflecting the influence of illumination on edge perception. The above iterative process of performing forward propagation calculations once for each of the illumination path, structure path, and noise-resistant path, and then performing information weighted fusion and exchange in the cross-path attention exchange module, is repeated until a preset number of iterations is reached. The preset number of iterations can be 3 or 5 rounds. After the final iteration, the illumination purification vector is read from the output layer of the illumination path, the structure enhancement vector is read from the output layer of the structure path, and the noise suppression vector is read from the output layer of the noise reduction path. It can be understood that through multiple iterations of cross-path attention exchange, the information from each path can be fully interacted and purified within the decoupled network.

[0034] See Figure 4 This is a graph showing the change in feature dimensions of each pathway during the iterative process of the decoupled network. It illustrates the trend of feature dimensions (number of columns after flattening) of the illumination, structure, and noise resistance pathways with the number of iterations during the forward propagation phase of the decoupled network. As the number of iterations increases, the feature dimensions of all three pathways show a decreasing trend, indicating that the network filters and refines features during the iteration process. The illumination pathway shows the largest decrease in dimension, from 256 to 128, a reduction of 50%; the structure pathway shows the most gradual decrease in dimension, consistently remaining at the highest level; the noise resistance pathway's decrease falls in between. The illumination pathway's fastest decrease in dimension indicates that its main task is to quickly filter ambient light interference and focus on effective lighting information. The structure pathway's consistently highest dimension indicates that it needs to retain rich structural details to ensure that the outline and texture information of the live object are not lost. The noise resistance pathway's moderate change in dimension reflects its need to balance information integrity while suppressing noise.

[0035] In one embodiment of the present invention, the fusion output module weights and combines the illumination purification vector, structure enhancement vector, and noise suppression vector according to a preset fusion rule. A dynamic weight is assigned to the illumination purification vector, which is obtained by mapping the absolute value of the difference between the average intensity of the current frame's illumination component stream and the average intensity of the historical illumination sequence using a monotonically increasing function. The function mapping can be scaled using the Sigmoid function, ensuring the dynamic weight ranges from 0 to 1. A fixed weight is assigned to the structure enhancement vector; this fixed weight is a pre-set constant based on experience, reflecting the fundamental importance of structural information in the final imaging. A dynamic weight is assigned to the noise suppression vector, which is obtained by mapping the proportion of high-frequency noise energy to total noise energy in the current noise spectrum analysis window using another monotonically increasing function. It is ensured that the sum of the dynamic weight of the illumination purification vector, the fixed weight of the structure enhancement vector, and the dynamic weight of the noise suppression vector is equal to one. The dynamic weight of the illumination purification vector is then calculated. The formula can be expressed as: ; in: Represents the Sigmoid activation function. It is an adjustable scaling factor. This represents the average intensity scalar value of the current frame's illumination component stream. This represents the historical average intensity scalar value calculated from the historical illumination history sequence. Calculate the absolute value of the difference between the two. Dynamic weights of the noise suppression vector. The fixed weights of the structure enhancement vector, calculated in a similar manner based on the proportion of high-frequency noise energy, are denoted as... And satisfy .

[0036] The illumination purification vector is multiplied by its dynamic weight, the structure enhancement vector by its fixed weight, and the noise suppression vector by its dynamic weight. These three weighted vectors are then summed element-wise to obtain the vector representation of the final anti-interference imaging data block. In some embodiments, when a sudden change in ambient illumination causes an increase in the dynamic weight of the illumination purification vector, the illumination compensation effect in the final imaging data block will be correspondingly enhanced.

[0037] The state feedback module outputs the final anti-interference imaging data block to the holographic projection module for imaging, and simultaneously writes back the key features to the corresponding region of the dynamic data grid. The block corresponding spatially to the original input data is parsed from the final anti-interference imaging data block, the parsing process based on a preset spatial coordinate mapping relationship. A difference map is calculated between the parsed data block and the corresponding old data in the current dynamic data grid, recording the difference between the new and old values ​​for each data point. A low-pass filter is applied to the difference map, using Gaussian kernel convolution to filter out high-frequency abrupt differences while retaining low-frequency gradual changes. The filtered difference map is then weighted and averaged with the old data in the current dynamic data grid, where the weight of the old data decays over time, while the weight of the new differences increases over time. One possible weighting method is that the weight coefficient of the new differences increases linearly with the processing cycle, while the weight coefficient of the old data decreases linearly accordingly. The weighted average result is used to update the corresponding data region in the dynamic data grid, serving as the input basis for the synchronous processing steps in the next processing cycle. Understandably, this update method enables the adaptive and smooth evolution of the system state.

[0038] Constructing the initial dynamic data grid involves: allocating a three-dimensional data buffer in memory based on the imaging sensor's resolution. The buffer's width and height match the sensor's resolution, and its depth is the preset number of channels. For example, for a 1920x1080 sensor, a three-dimensional data buffer with a width of 1920, a height of 1080, and a depth of 3 (corresponding to RGB channels) can be constructed. All elements of the three-dimensional data buffer are initialized to zero. A virtual grid structure covering the three-dimensional data buffer is defined. Each cell of the virtual grid is associated with a two-dimensional position index in the buffer and contains pointers to all depth channel data in the corresponding column of the buffer. In some embodiments, the cell located in the i-th row and j-th column of the virtual grid has a pointer to all channel data in the i-th row and j-th column of the three-dimensional data buffer. A circular indexing mechanism is provided for the virtual grid structure. This mechanism maintains a current write pointer, ensuring that newly written data overwrites the oldest data in chronological order, and maintains a fixed-length historical context window, for example, always retaining the valid context of the most recent 5 frames of data. It can be understood that the initialization and update mechanism of the dynamic data grid provides an iterable data base with historical memory for the entire processing flow.

[0039] See Figure 5 This diagram illustrates the weight update mechanism of the dynamic data grid in the state feedback module. It shows the changing trends of old data weights, new difference weights, and the data update rate over the processing cycle during the update process. It visually reflects how the system balances historical and new information to achieve stable and efficient state updates. The old data weight decreases exponentially with increasing processing cycles, rapidly dropping from an initial 0.95 to 0.08, indicating that the system gradually reduces its reliance on historical states. The new difference weight increases exponentially with increasing processing cycles, rising from an initial 0.05 to 0.92, indicating that the system increasingly values ​​newly generated difference information. The data update rate increases slowly with increasing processing cycles, eventually stabilizing, a result of the combined effect of old and new data weights.

[0040] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A holographic imaging device for e-commerce live streaming based on resistance to environmental interference, characterized in that, include: The data preprocessing module constructs an initialized dynamic data grid, receives and temporarily stores the raw input stream from the imaging sensor, performs synchronous processing on the raw input stream in the dynamic data grid, and generates a hybrid feature tensor. The decoupling processing module inputs the hybrid feature tensor into a cyclically evolving decoupling network. The decoupling network includes intertwined but parameter-independent illumination, structural, and noise suppression paths. Within the decoupling network, through multiple rounds of iterative data exchange, the illumination path outputs an illumination purification vector, the structural path outputs a structural enhancement vector, and the noise suppression path outputs a noise suppression vector. The fusion output module weights and combines the illumination purification vector, structure enhancement vector, and noise suppression vector according to a preset fusion rule to generate the final anti-interference imaging data block for this processing cycle. The state feedback module outputs the final anti-interference imaging data block to the holographic projection module for imaging, and simultaneously writes back the key features to the corresponding area of ​​the dynamic data grid to update the initial state of the next processing cycle.

2. The holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 1, characterized in that, The original input stream in the dynamic data grid is synchronously processed to generate a hybrid feature tensor, including: After the holographic imaging device is started, an initial dynamic data grid is constructed to receive and temporarily store the raw input stream from the imaging sensor in real time. The original input stream in the dynamic data grid is synchronously processed, and each frame of input data is decomposed into illumination component stream, structure component stream and noise component stream. Create an independent illumination history sequence to continuously record the temporal evolution trajectory of the illumination component stream; The topological skeleton information representing the contour of the live object is extracted from the structural component stream, and this topological skeleton information is mapped to a dedicated topological feature space. For the noise component stream, an online adaptive noise spectrum analysis window is activated to calculate the distribution pattern of noise energy in the frequency domain in real time; Within each processing cycle, the structural components of the current frame are extracted from the dynamic data grid and correlated with the historical illumination sequence and real-time noise spectrum to generate a hybrid feature tensor containing multidimensional contextual information.

3. The holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 2, characterized in that, Synchronization processing of the raw input stream in the dynamic data grid includes: A synchronization controller is assigned to the dynamic data grid, and the synchronization controller generates timing pulses according to the sampling frequency of the imaging sensor; At the effective edge of each timing pulse, the synchronization controller locks a fixed-size data block in the dynamic data grid, the data block containing the latest, unprocessed raw input data at the current moment; The locked data block is copied to three parallel processing pipelines; In the first processing pipeline, a brightness separation operator based on logarithmic domain transformation is applied to extract the components that mainly reflect changes in ambient light intensity from the copied data blocks, forming the light component stream; In the second processing pipeline, a multi-scale edge detection and contour connection algorithm is applied to extract components reflecting the geometry and surface texture of the live object from the copied data block, forming the structural component stream; In the third processing pipeline, differential operations are applied between the original data block and the data block that has undergone preliminary processing in the first and second pipelines, and combined with high-frequency component filtering, to obtain the residual random fluctuation components, forming the noise component stream.

4. The holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 3, characterized in that, Extracting topological skeleton information representing the contour of the live-stream object from the structural component stream includes: For each frame of the binary contour map in the structural component stream, perform a morphological thinning operation until the contour line width is one pixel to obtain the initial skeleton map. On the initial skeleton map, the endpoints and intersections of all branches are identified, and a continuous chain of pixels connecting two key points is defined as a skeleton segment. Calculate the geometric properties of each skeleton segment, including length, orientation, average curvature, and area of ​​the original contour region it covers; Establish a hierarchical relationship tree for skeleton segments. The root node of the tree is the main skeleton segment of the object, and the child nodes are the secondary skeleton segments branched from the main segment. All skeleton segments, their geometric attributes, and hierarchical relationships are encoded into a fixed-dimensional feature vector, which serves as the topological skeleton information and is then fed into the dedicated topological feature space.

5. The holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 4, characterized in that, Within each processing cycle, the structural components of the current frame are extracted from the dynamic data grid and correlated with the historical illumination sequence and real-time noise spectrum to generate a hybrid feature tensor containing multi-dimensional contextual information, including: The feature vector of topological skeleton information is extracted from the current frame structure components of the dynamic data grid; From the illumination history sequence, the moving average of the illumination purification vector for the most recent several consecutive periods is read as the historical illumination context; From the online adaptive noise spectrum analysis window, obtain the pattern code of the noise energy distribution at the current moment, which identifies the type of dominant noise; The feature vector of the topological skeleton information, the historical illumination context, and the pattern encoding of the noise energy distribution are concatenated to form a one-dimensional joint feature vector. The one-dimensional joint feature vector is reshaped according to a preset dimensional rule to form a three-dimensional hybrid feature tensor, where the three dimensions correspond to the abstract feature channels of structure, illumination history and noise spectrum, respectively.

6. The e-commerce live streaming holographic imaging device based on anti-environmental interference as described in claim 5, characterized in that, The decoupling processing module is configured to generate the output vectors of each path in the following ways: At the beginning of each iteration of the decoupled network, the hybrid feature tensor is simultaneously fed into the input layer of the illumination path, the structure path, and the noise reduction path. Each path performs a forward propagation computation once, generating the intermediate potential representation of the corresponding path in the current iteration round; At the end of each iteration, a cross-path attention exchange module is set up. The cross-path attention exchange module calculates the attention weight of the intermediate representation of the illumination path to the intermediate representation of the structure path, calculates the attention weight of the intermediate representation of the structure path to the intermediate representation of the anti-noise path, and calculates the attention weight of the intermediate representation of the anti-noise path to the intermediate representation of the illumination path. Based on the calculated attention weights, the intermediate latent representations of the three pathways are fused and exchanged using information weighting to generate an enhanced input for each pathway for the next iteration. The forward propagation calculation is performed once for each of the illumination path, structure path, and noise reduction path, and an iterative process of information weighted fusion and exchange is carried out in the cross-path attention exchange module until the preset number of iterations is reached. After the final iteration round is completed, the illumination purification vector is read from the output layer of the illumination path, the structure enhancement vector is read from the output layer of the structure path, and the noise suppression vector is read from the output layer of the noise reduction path.

7. The e-commerce live streaming holographic imaging device based on anti-environmental interference according to claim 6, characterized in that, The specific operation of the cross-path attention exchange module is as follows: The intermediate latent representations of the illumination path, the structural path, and the noise-resistant path are respectively used as the first feature matrix, the second feature matrix, and the third feature matrix. Calculate the product of the first feature matrix and the transpose of the second feature matrix, and input the product result into the normalization function to obtain the first attention weight matrix. The first attention weight matrix represents the association weight of illumination information with structural information. Calculate the product of the second feature matrix and the transpose of the third feature matrix, and input the product result into the normalization function to obtain the second attention weight matrix. The second attention weight matrix represents the association weight of structural information with noise information. Calculate the product of the third feature matrix and the transpose of the first feature matrix, and input the product result into the normalization function to obtain the third attention weight matrix, which represents the association weight of noise information with illumination information. The second feature matrix is ​​weighted and fused using the first attention weight matrix, and the fusion result is added to the first feature matrix to generate an enhanced intermediate representation of the illumination path for the next iteration. The third feature matrix is ​​weighted and fused using the second attention weight matrix, and the fusion result is added to the second feature matrix to generate an enhanced intermediate representation of the structural path for the next iteration. The first feature matrix is ​​weighted and fused using the third attention weight matrix, and the fusion result is added to the third feature matrix to generate an enhanced intermediate representation for the next iteration of the noise-resistant path.

8. The e-commerce live streaming holographic imaging device based on anti-environmental interference as described in claim 7, characterized in that, The illumination purification vector, structure enhancement vector, and noise suppression vector are weighted and combined according to a preset fusion rule, including: A dynamic weight is assigned to the illumination purification vector, which is obtained by mapping the absolute value of the difference between the average intensity of the current frame illumination component stream and the average intensity of the historical illumination sequence through a monotonically increasing function. Assign a fixed weight to the structure enhancement vector; A dynamic weight is assigned to the noise suppression vector. This dynamic weight is obtained by mapping the proportion of high-frequency noise energy to the total noise energy in the current noise spectrum analysis window through another monotonically increasing function. Ensure that the sum of the dynamic weights of the illumination purification vector, the fixed weights of the structure enhancement vector, and the dynamic weights of the noise suppression vector is one. Multiply the illumination purification vector by its dynamic weight, the structure enhancement vector by its fixed weight, and the noise suppression vector by its dynamic weight. The three weighted vectors are added element by element to obtain the vector representation of the final anti-interference imaging data block.

9. A holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 8, characterized in that, The final anti-interference imaging data block is output to the holographic projection module for imaging, and the key features are simultaneously written back to the corresponding area of ​​the dynamic data grid, including: From the final anti-interference imaging data block, the spatially corresponding block of the original input data is parsed out; Calculate the difference map between the parsed data block and the old data at the corresponding position in the current dynamic data grid; A low-pass filter is applied to the difference map to remove abrupt differences at high frequencies and retain gradual changes at low frequencies. The filtered difference plot is weighted and averaged with the old data in the current dynamic data grid, where the weight of the old data decays over time and the weight of the new differences increases over time. The weighted average result is used to update the corresponding data region in the dynamic data grid, which serves as the input basis for the synchronous processing step in the next processing cycle.

10. A holographic imaging device for e-commerce live streaming based on anti-environmental interference as described in claim 9, characterized in that, The construction of the initialized dynamic data grid includes: Based on the resolution of the imaging sensor, a three-dimensional data buffer is allocated in memory, with its width and height matching the sensor resolution and its depth being the preset number of channels. Initialize all elements of the three-dimensional data buffer to zero; Define a virtual mesh structure that covers the three-dimensional data buffer. Each cell of the virtual mesh is associated with a two-dimensional position index of the buffer and contains pointers to all depth channel data of the corresponding column in the buffer. The virtual grid structure is equipped with a circular indexing mechanism, which allows newly written data to overwrite old data in chronological order and maintains a fixed-length historical context window.

Images