A multi-dimensional perception driven digital cultural exhibit interactive processing method and system

Abstract

The application discloses a kind of multi-dimensional perception driven digital cultural exhibits interactive processing method and system, method includes the following steps: S1, the cultural semantic event that digital cultural exhibits is exported in the process of demonstration is acquired, the cultural semantic event carries output time and semantic label;S2, in the reaction time window established after the output time, the facial high frame rate image stream of audience is collected, micro-motion tracking is carried out to multiple preset facial key areas, and the spatiotemporal micro-expression features of audience under the induction of the cultural semantic event are extracted.Compared with prior art, the present application can effectively determine whether the captured micro-expression indeed has causal relationship with the current exhibit content, so that the triggering of interactive action is reliable and quantifiable, significantly reduces the false trigger rate, and ensures the continuity and stability of the immersive exhibition experience of the audience.

Description

Technical Field

[0001] This invention belongs to the field of interactive processing of digital cultural exhibits, specifically relating to a multi-dimensional perception-driven method and system for interactive processing of digital cultural exhibits. Background Technology

[0002] With the deep application of digital technology in the field of cultural exhibitions, digital cultural exhibits such as holographic artifacts and interactive digital scrolls are gradually becoming the mainstream display formats. In order to enhance the viewing experience, existing technologies have attempted to collect facial images of visitors, identify their emotional states, and adjust the presentation of exhibits accordingly, thereby achieving intelligent interaction between exhibits and visitors.

[0003] However, in practical applications of digital cultural exhibits, the above solutions face a core challenge: audience micro-expressions are not always induced by the exhibit content. The exhibition environment contains numerous distractions, such as conversations with companions, mobile phone notifications, and environmental noise, all of which can trigger changes in audience micro-expressions. Current technology lacks effective means to determine whether captured micro-expressions are indeed causally related to the current exhibit content, leading to unreliable triggering criteria for interactive actions. Frequent false triggers severely disrupt the audience's immersive viewing experience. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-dimensional perception-driven interactive processing method and system for digital cultural exhibits, so as to solve the technical problems mentioned in the background above.

[0005] To achieve the above objectives, this invention provides a multi-dimensional perception-driven interactive processing method for digital cultural exhibits, comprising the following steps: S1, acquiring cultural semantic events output by the digital cultural exhibits during the display process, wherein the cultural semantic events carry an output time and a semantic label; S2, within a reaction time window established after the output time, acquiring a high frame rate image stream of the audience's face, performing micro-motion tracking on multiple preset key facial regions, and extracting the spatiotemporal micro-expression features of the audience induced by the cultural semantic events; S3, based on the spatiotemporal micro-expression features, combined with the spatial matching degree between the audience's gaze and the exhibit's area of ​​interest, and the time offset between the micro-expression occurrence time and the output time, calculating the exhibit-induced confidence of the micro-expression; S4 S5. When the exhibit-induced confidence level is higher than a preset threshold, composite emotion decoupling is performed based on the spatiotemporal micro-expression features to generate a fine emotional state vector of the audience containing multiple independent psychological dimension parameter values, and a cultural understanding load index reflecting the audience's cognitive input state is calculated; S6. Based on the audience's fine emotional state vector, cultural understanding load index, and exhibit-induced confidence level, a graded interaction parameter set is generated to drive the exhibit presentation device to perform corresponding level interactive actions; S7. After the interactive action is executed, a second facial high frame rate image stream within a preset feedback time window is collected to generate a fine emotional state vector of the audience after intervention, and the generation rules of the graded interaction parameter set are adjusted according to the difference results of the audience's fine emotional state vector before and after intervention.

[0006] In one or more embodiments of the present invention, S2 extracts the spatiotemporal micro-expression features of the audience under the cultural semantic event, specifically including: performing peak detection on the motion trajectory of each of the preset facial key regions to determine the pulse start time, peak time and pulse amplitude of each of the preset facial key regions; constructing a micro-expression phase propagation matrix describing the propagation sequence of facial micro-movements based on the peak time delay relationship between different preset facial key regions; and calculating the motion amplitude asymmetry index based on the motion trajectory of the same preset action unit on the left and right sides of the face.

[0007] In one or more embodiments of the present invention, constructing a micro-expression phase propagation matrix describing the propagation time sequence of facial micro-movements specifically includes: dividing the plurality of preset key facial regions into the glabella region, eyelid region, nasal ala region, corner of mouth region, and cheekbone region; normalizing the motion trajectory of each region and extracting the pulse start time, peak time, and pulse amplitude of each region; calculating the peak time difference between any two regions to form the peak propagation delay between regions; and combining the peak propagation delays of all region pairs according to the division of the plurality of preset key facial regions to form a multidimensional matrix describing the propagation time sequence relationship of micro-movements between regions, which serves as the micro-expression phase propagation matrix.

[0008] In one or more embodiments of the present invention, calculating the exhibit-induced confidence level of the micro-expression in S3 specifically includes: calculating a first confidence component based on whether the peak-delay relationship in the spatiotemporal micro-expression features conforms to a preset expected propagation pattern corresponding to the semantic label of the cultural semantic event; calculating a second confidence component based on whether the time offset between the occurrence time of the micro-expression pulse and the output time is within a preset effective response window; calculating a third confidence component based on whether the viewer's gaze point overlaps with the exhibit's area of ​​interest corresponding to the current cultural semantic event; and weightedly fusing the first, second, and third confidence components to obtain the exhibit-induced confidence level.

[0009] In one or more embodiments of the present invention, the calculation of the first confidence component specifically includes: performing similarity matching between the micro-expression phase propagation matrix in the spatiotemporal micro-expression features and the preset expected propagation pattern corresponding to the semantic label, and calculating the Pearson correlation coefficient; when the Pearson correlation coefficient is higher than a first preset threshold, the first confidence component takes a high value, when it is lower than a second preset threshold, it takes a low value, and when it is in between, it takes a median value; wherein, different semantic labels correspond to different expected propagation patterns, visual detail events correspond to the short-term peak propagation pattern from the eyelid area to the glabella area, narrative conflict events correspond to the composite propagation pattern of the glabella area, the nasal alar area, and the corner of the mouth area, and aesthetic pleasure events correspond to the peak propagation pattern from the corner of the mouth area to the cheekbone area.

[0010] In one or more embodiments of the present invention, the calculation of the cultural understanding load index reflecting the audience's cognitive engagement state in S4 further includes: determining the audience's cognitive engagement state as exploratory load, obstructive load, or low-participation load based on the combined relationship between the focus and confusion levels and the entropy value of the eye saccade path; wherein, when the focus level is higher than a first threshold, the confusion level is in the middle range, and the entropy value of the eye saccade path is lower than the entropy value threshold, it is determined to be exploratory load; when the confusion level is higher than a second threshold and the entropy value of the eye saccade path is higher than the entropy value threshold, it is determined to be obstructive load; when the focus level is lower than a third threshold, it is determined to be low-participation load; and, a low-salience prompting strategy is matched for the exploratory load, an auxiliary explanation presentation strategy is matched for the obstructive load, and an information density reduction or content guidance switching strategy is matched for the low-participation load.

[0011] In one or more embodiments of the present invention, the duration of the reaction time window in S2 is initially set according to the type of cultural semantic event that triggers the reaction time window. Visual detail events correspond to a first duration window, and narrative turning point events correspond to a second duration window. The second duration window is longer than the first duration window. Furthermore, based on the historical micro-expression peak moment distribution of the same audience under the same type of cultural semantic event, the start offset and end offset of the reaction time window corresponding to this type of event are adaptively adjusted.

[0012] In one or more embodiments of the present invention, generating a graded interaction parameter set in S5 includes: increasing the grade of the interaction action when the exhibit's evoked confidence level is higher than a preset confidence threshold within multiple consecutive reaction time windows; and decreasing the grade of the interaction action when, after the interaction action is executed, a decrease in focus or a viewer's gaze leaving the exhibit's area of ​​interest is detected in the audience's fine emotional state vector.

[0013] To achieve the above objectives, this invention provides a multi-dimensional perception-driven interactive processing system for digital cultural exhibits, comprising a digital cultural exhibit presentation device, a cultural semantic event output module, a viewer perception acquisition module, a micro-expression processing module, a gaze matching module, an exhibit-induced confidence calculation module, a composite emotion decoupling module, a cultural understanding load calculation module, an interaction parameter generation module, and a feedback correction module. The cultural semantic event output module acquires cultural semantic events output by the digital cultural exhibits during the display process, each cultural semantic event carrying an output time and semantic label. The viewer perception acquisition module acquires a high-frame-rate facial image stream of the viewer within a reaction time window established after the output time. The micro-expression processing module tracks micro-motions in multiple preset key facial regions to extract spatiotemporal micro-expression features of the viewer induced by the cultural semantic events. The gaze matching module acquires the spatial matching degree between the viewer's gaze and the exhibit's area of ​​interest. The exhibit-induced confidence calculation module calculates the spatiotemporal micro-expression features, spatial matching degree, and micro-expression load based on the cultural semantic events. The time offset between the occurrence and output of the expression is used to calculate the exhibit-induced confidence level of the micro-expression; the composite emotion decoupling module is used to perform composite emotion decoupling based on the spatiotemporal micro-expression features when the exhibit-induced confidence level is higher than a preset threshold, generating a fine emotional state vector of the audience containing multiple independent psychological dimension parameter values; the cultural understanding load calculation module is used to calculate the cultural understanding load index reflecting the audience's cognitive input state based on the audience's fine emotional state vector and the audience's gaze scanning path entropy value; the interaction parameter generation module is used to generate a graded interaction parameter set based on the audience's fine emotional state vector, the cultural understanding load index, and the exhibit-induced confidence level, so as to drive the digital cultural exhibit presentation device to perform corresponding level interactive actions; the feedback correction module is used to collect a second facial high frame rate image stream within a preset feedback time window after the interactive action is executed, generate a fine emotional state vector of the audience after intervention, and adjust the generation rules of the graded interaction parameter set based on the difference results of the audience's fine emotional state vector before and after intervention.

[0014] In one or more embodiments of the present invention, the audience perception acquisition module includes at least one high frame rate image acquisition device with a frame rate of 120fps to 240fps. The image acquisition device is mounted on the front or side front of the exhibit via a rigid bracket, and a ring-shaped near-infrared supplementary light and a frosted diffuser are arranged around the outer periphery of the image acquisition device. A polarizing filter is also arranged in front of the image acquisition device. The plurality of preset facial key areas include the glabella area, eyelid area, nasal wing area, corner of mouth area, and cheekbone area. The micro-expression processing module performs peak detection on the motion trajectory of each preset facial key area to determine the pulse start time, peak time, and pulse amplitude of each preset facial key area. Based on the peak time delay relationship between different preset facial key areas, a micro-expression phase propagation matrix is ​​constructed. At the same time, based on the motion trajectory of the same preset action unit on the left and right sides of the face, the motion amplitude asymmetry index is calculated.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: By using the cultural semantic events actually output by the exhibits as the time benchmark and causal anchor point, a reaction time window bound to semantic tags is established after the output time. This limits the collection of micro-expressions to an interval with reasonable psychological reaction delays. Furthermore, a micro-expression phase propagation matrix for multiple key facial regions is constructed. The exhibit-induced confidence level is calculated by combining expected propagation pattern matching, time offset verification, and gaze space matching degree with triple weighted fusion. Interactive actions are only triggered when the confidence level meets the threshold. This accurately distinguishes between micro-expressions induced by exhibits with propagation patterns and facial movements caused by external interference. It effectively determines whether the captured micro-expressions are indeed causally related to the current exhibit content, making the triggering basis of interactive actions reliable and quantifiable. This significantly reduces the false trigger rate and ensures the continuity and stability of the audience's immersive viewing experience. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall process of a multi-dimensional perception-driven digital cultural exhibit interaction processing method in one embodiment of the present invention.

[0017] Figure 2 This is a flowchart illustrating the extraction of spatiotemporal micro-expression features in a multi-dimensional perception-driven digital cultural exhibit interaction processing method according to an embodiment of the present invention.

[0018] Figure 3 This is a flowchart illustrating the calculation of exhibit-induced confidence in a multi-dimensional perception-driven digital cultural exhibit interaction processing method according to an embodiment of the present invention.

[0019] Figure 4 This is a flowchart illustrating the calculation of the cultural understanding load index and the determination of cognitive state in a multi-dimensional perception-driven digital cultural exhibit interaction processing method according to an embodiment of the present invention.

[0020] Figure 5 This is a schematic diagram illustrating the adaptive adjustment of the reaction time window and the upgrading and downgrading of the interaction level in a multi-dimensional perception-driven digital cultural exhibit interaction processing method according to an embodiment of the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0022] like Figures 1-5 As shown, a multi-dimensional perception-driven interactive processing method and system for digital cultural exhibits is presented. The system is used to execute the multi-dimensional perception-driven interactive processing method for digital cultural exhibits. This solution is applied to digital cultural exhibition scenarios, such as holographic artifact display cases, immersive digital scrolls, digital murals, virtual restored ruins, and interactive artifact explanation screens. The system includes a digital cultural exhibit presentation device, a cultural semantic event output module, a visitor perception acquisition module, a micro-expression processing module, a gaze matching module, an exhibit-induced confidence calculation module, a complex emotion decoupling module, a cultural understanding load calculation module, an interaction parameter generation module, and a feedback correction module. The above modules can be integrated into the edge computing host of the exhibition hall, or they can be implemented collaboratively by the local controller of the exhibit and the cloud server.

[0023] Digital cultural exhibit presentation devices may include transparent holographic displays, projection fusion screens, LED display walls, spatial audio speakers, controllable light sources, tactile prompts, and interactive explanatory screens. The audience perception acquisition module includes at least one high-frame-rate image acquisition device facing the audience's face. This device preferably uses an industrial camera or global shutter camera with a frame rate of 120fps to 240fps and a resolution of no less than 1920×1080, and is mounted on a rigid bracket in front of or to the side of the exhibit. To reduce the impact of changes in exhibition hall lighting on micro-expression recognition, a ring-shaped near-infrared supplementary light is installed around the high-frame-rate image acquisition device. The supplementary light wavelength can be selected as 850nm or 940nm. A frosted diffuser is installed outside the supplementary light to ensure uniform illumination of the audience's face without producing obvious glare.

[0024] A polarizing filter can also be installed in front of the image acquisition device to reduce reflection interference from the glass display case or holographic film surface.

[0025] During the playback or presentation of exhibits, the cultural semantic event output module does not merely record video frames or playback progress, but rather breaks down the exhibit content into event units with cultural interpretive significance. For example, in a digital scroll, "figures looking back," "warhorses leaping," "seals appearing," and "inscriptions unfolding" can each be considered cultural semantic events; in a holographic artifact display, "partial magnification of patterns," "inscription interpretation appearing," "restoration of missing parts," and "historical scene switching" can each be considered cultural semantic events. Each cultural semantic event carries the output time, semantic tag, corresponding exhibit interest area, recommended response time window, and interactive candidate strategies.

[0026] The output time can be generated by the exhibit playback controller when the rendering engine completes the keyframe output, or it can be pre-marked in the script timeline by the content orchestration system.

[0027] Specifically, when a digital cultural exhibit display device outputs a cultural semantic event, the cultural semantic event output module generates an event data packet. The event data packet includes at least the event number, output time, semantic tag, coordinates of the exhibit's area of ​​interest, event intensity level, and event type. The event data packet can be represented as:

[0028]

[0029] In the formula, This represents the k-th cultural semantic event; Indicates the event number; This indicates the moment when the cultural semantic event is actually output to a state that is visible or audible to the audience; These represent semantic tags, such as visual details, narrative conflict, aesthetic pleasure, or narrative turning points. This indicates the area of ​​interest for the exhibit corresponding to the event, which can be a region in the screen coordinate system or the exhibit's three-dimensional coordinate system. Indicates the intensity level of the event, used to distinguish between ordinary prompts, key narrative nodes, or highly salient visual nodes; This indicates the content channel associated with the event, such as images, text, audio, or spatial lighting effects.

[0030] This event data packet serves as a time reference for subsequent micro-expression collection, gaze matching, and interaction triggering, enabling the system to establish a judgment chain based on "what the exhibit actually outputs" rather than simply "what happened on the audience's face," thereby reducing false triggers caused by environmental interference from the source.

[0031] After the event data packet is generated, the system outputs the time. A reaction time window is established as a baseline. The reaction time window is not a fixed length, but rather an initial window is set based on the type of cultural semantic event, and adaptively adjusted according to the historical distribution of peak micro-expression moments for the same audience member under similar events. For visual detail events, since audience members typically produce micro-movements in areas such as the eyelids and brows within a short period, the first duration window can be set to 0.2s to 1.2s. For narrative transition or conflict events, since audience members need to complete semantic understanding before showing an expression, the second duration window can be set to 0.4s to 2.5s, and the second duration window is longer than the first duration window.

[0032] The reaction time window can be expressed as:

[0033]

[0034] In the formula, This represents the reaction time window corresponding to the k-th cultural semantic event; Representation and semantic tags The corresponding starting offset; Representation and semantic tags The corresponding termination offset. This setting is because different cultural content triggers different psychological processing times in viewers. If micro-expressions are captured using a fixed window, facial movements caused by exhibition hall noise, peer interaction, or mobile phone notifications can easily be mistaken for exhibit reactions. By binding time windows with semantic tags, micro-expression analysis can be limited to an interval with a reasonable delay in psychological response, improving the reliability of subsequent causal judgments.

[0035] Furthermore, the system updates the reaction time window based on the historical reaction data of the same audience member. Let the semantic label of the same audience member be... The set of offsets of micro-expression peaks detected in historical events relative to the event output time is: Then, the window boundaries can be updated using a sliding statistics method:

[0036]

[0037]

[0038] In the formula, This represents the average historical peak offset for similar semantic events. The standard deviation of the historical peak offset; and This is the window expansion factor, preferably ranging from 1 to 2; and These represent the minimum starting offset and the maximum ending offset allowed by the system, respectively. The value is 0.15s. The value is 0.3s. and The value is determined by the longest psychological response delay of the human eye to visual stimuli. This model enables the system to adapt to the differences in reaction speed among different viewers, reducing missed detections, while avoiding the introduction of interference caused by infinitely expanding the window.

[0039] Within the reaction time window, the audience perception acquisition module continuously acquires a high frame rate image stream of the audience's face. The acquired image stream first undergoes face detection, facial pose estimation, key point localization, and region registration. Key facial regions include at least the glabella, eyelids, nasal alae, corners of the mouth, and cheekbones, each enclosed by several facial key points. The system can employ an active shape model, a convolutional neural network key point model, or a lightweight Transformer key point model for key point detection. To ensure the tracking of minute movements, the system can pause the current event's micro-expression reliability assessment when it detects a facial yaw angle greater than 30°, a pitch angle greater than 25°, or an occlusion ratio greater than 40%, avoiding erroneous micro-motion extraction due to pose changes.

[0040] For each preset key facial region, the system uses a short-term stable image before the event output as a reference frame. The reference frame can be selected from... to The average of several frames between them. Then, within the reaction time window, optical flow tracing or feature point displacement tracing is performed on the local texture blocks of each region to obtain the region motion trajectory. Let the set of key point coordinates of the r-th facial key region in the m-th frame be . The set of reference frame coordinates is Then the normalized motion amplitude in this region can be expressed as:

[0041]

[0042] In the formula, This represents the normalized motion amplitude of the r-th facial key region in the m-th frame; This indicates the number of key points involved in the calculation within the region; This indicates the position of the i-th keypoint in the r-th region in the m-th frame; This indicates the location of the key point in the reference frame; This represents the distance between the outer canthi of the viewer's eyes, used to eliminate scale differences caused by varying distances between the viewer and the camera. Through this normalization process, the system obtains regional motion trajectories that are correlated with real micro-motion but weakly correlated with shooting distance, providing stable input for subsequent peak detection.

[0043] Subsequently, the micro-expression processing module performs bandpass filtering and peak detection on the motion trajectory of each region. The frequency range of the bandpass filter can be set from 0.5Hz to 15Hz to reduce slow head movements and high-frequency image noise. For each region, the system identifies the pulse start time, peak time, and pulse amplitude. If the motion trajectory exceeds the mean of the background noise plus a preset multiple of the standard deviation for several consecutive frames and then falls back within a short time, it is determined that a micro-expression pulse has appeared in that region. Correspondingly, the pulse feature of the r-th region can be represented as:

[0044]

[0045] In the formula, This represents the micro-expression pulse characteristics of the r-th key facial region; Indicates the pulse start time; Indicates the peak moment of the pulse; Indicates the pulse amplitude; This indicates the duration of the pulse rise. This feature does not simply record "whether an expression appears," but rather records the temporal structure of facial micro-movements occurring, propagating, and intensifying in different areas. This allows subsequent systems to distinguish between transient, localized micro-expressions induced by exhibits and large facial movements caused by ordinary speaking, turning the head, or external disturbances.

[0046] After obtaining the pulse characteristics of each facial region, the system constructs a micro-expression phase propagation matrix. Specifically, the glabella, eyelid, nasal alar, corner of mouth, and cheekbone regions are denoted as region sets. Calculate the peak time difference between any two regions and combine them into a matrix:

[0047]

[0048]

[0049] In the formula, This represents the peak propagation delay between the i-th and j-th regions within the reaction time window corresponding to the k-th cultural semantic event; when When, it means that the peak value of the i-th region is later than that of the j-th region; when The term "time" indicates that the peak value of the i-th region precedes that of the j-th region. This matrix serves to preserve the temporal propagation relationship of micro-expressions. For example, aesthetic pleasure reactions may typically manifest as a slight upward tilt in the corner of the mouth, followed by a subtle correlation in the cheekbone area; narrative conflict reactions may manifest as a complex tension pattern forming in the glabella, nasal alae, and corner of the mouth areas within a short period. Compared to judging emotions solely based on the amplitude of a single region's movement, this invention describes "how expressions unfold" using a propagation matrix, thereby improving the ability to identify reactions evoked by real exhibits.

[0050] Simultaneously, the system calculates the asymmetry index of motion amplitude based on the motion trajectories of the same preset motion unit on the left and right sides of the face. Let the amplitude of a certain motion unit on the left side be... The amplitude on the right is Then the asymmetry index can be expressed as:

[0051]

[0052] In the formula, Indicates the asymmetry index of motion amplitude; and These represent the amplitude of micro-movements in the corresponding facial regions on the left and right sides, respectively. To prevent extremely small constants with a denominator of zero, this index is used to assist in the identification of spontaneous micro-expressions. Typically, short-term spontaneous reactions induced by exhibit content may exhibit slight asymmetry in certain areas, while expressions caused by deliberate performance or continuous talking may have different amplitude distribution characteristics. By incorporating this index as part of the spatiotemporal micro-expression features, the system can further reduce misjudgments caused by non-exhibit factors.

[0053] While extracting micro-expression features, the gaze matching module obtains the spatial matching degree between the viewer's gaze point and the exhibit's area of ​​interest. Gait data can be acquired using an infrared eye tracker or jointly estimated from the pupil center, eye corner position, and head posture in a facial image. The system projects the viewer's gaze direction onto the exhibit's coordinate system to obtain the gaze point. and the interest areas corresponding to current cultural semantic events. Perform matching. Spatial matching degree can be expressed as:

[0054]

[0055] In the formula, This indicates the spatial alignment between the line of sight and the areas of interest in the exhibits. Indicates the point of focus To the area of ​​interest Distance from the boundary or center; This represents the gaze tolerance parameter, the value of which can be set according to screen size, viewer distance, and eye movement estimation error. If the viewer's gaze falls within the area corresponding to the cultural semantic event, then... Approaching zero Approaching 1; if the viewer's gaze is significantly diverted from the current exhibit, then Approaching 0. The significance of this matching degree is that even if a visitor's face shows micro-expressions, if their gaze is not on the current event area, the micro-expressions are more likely to be caused by companions, mobile phones, or external factors in the exhibition hall, and should not directly drive interaction with exhibits.

[0056] After calculating the micro-expression phase propagation matrix, time offset, and gaze matching degree, the exhibit-induced confidence calculation module determines whether the micro-expression is induced by the exhibit content. This module first calls the corresponding expected propagation pattern based on semantic tags. The expected propagation pattern can be pre-trained by curatorial experts, experimental sample statistics, or historical audience data. For example, visual detail events correspond to a short-term peak propagation pattern from the eyelid area to the glabella area, narrative conflict events correspond to a composite propagation pattern from the glabella area, the nasal alar area, and the corner of the mouth area, and aesthetic pleasure events correspond to a peak propagation pattern from the corner of the mouth area to the cheekbone area.

[0057] The system will actually obtain the micro-expression phase propagation matrix The expected propagation pattern corresponding to this semantic tag Similarity matching is performed to obtain the first confidence component. The first confidence component... The micro-expression phase propagation matrix With the desired propagation pattern The Pearson correlation coefficient between them is calculated using the following formula:

[0058]

[0059] In the formula, This represents the first confidence component corresponding to the similarity of propagation patterns. This represents the micro-expression phase propagation matrix corresponding to the k-th cultural semantic event; Representation and semantic tags The corresponding expected propagation pattern; This represents the Pearson correlation coefficient function, used to measure the degree of linear correlation between two matrices on their effective elements.

[0060] Wherein, the Pearson correlation coefficient function It can be defined as: taking the matrix sum matrix After expanding the effective elements involved in the comparison into vectors, the standardized covariance between the two vectors is calculated. Its value ranges from [−1, 1]. When The closer the value is to 1, the more consistent the actual micro-expression propagation pattern is with the expected propagation pattern; when... The closer the correlation is to 0, the weaker the correlation between the two; when... When the value is less than 0, it indicates that the two have opposite propagation trends.

[0061] Subsequently, the system calculates the second confidence component based on the time offset between the occurrence time of the micro-expression pulse and the output time of the cultural semantic event. Let the main peak time be... Then the time offset is The second confidence component can be expressed as:

[0062]

[0063] In the formula, The second confidence component represents the time consistency. This represents the time offset of the micro-expression peak relative to the output time of the cultural semantic event; This represents the mean of the expected response delay under this semantic label; This indicates the degree of dispersion of the expected response delay under this semantic label; This represents the reaction time window for the current cultural semantic event. The model combines "whether it is within the time window" with "whether it is close to the expected delay," which can avoid the errors caused by making hard judgments based solely on the window boundaries.

[0064] The third confidence component is given by the aforementioned line-of-sight spatial matching degree, namely: Finally, the system performs a weighted fusion of the first confidence component, the second confidence component, and the third confidence component to obtain the exhibit-induced confidence score:

[0065]

[0066] In the formula, The exhibit-induced confidence level represents the micro-expression corresponding to the k-th cultural semantic event; , , These represent the weights of similarity in propagation patterns, temporal consistency, and spatial matching of gaze, respectively, with the sum of the three being 1; The second confidence component represents the time consistency. This represents the third confidence component corresponding to the line-of-sight spatial matching degree; This represents the mapping function that maps the Pearson correlation coefficient to the interval [0,1].

[0067] Due to Pearson correlation coefficient The value range is [−1, 1], while the second confidence component and the third confidence component The values ​​of all are in the range [0,1]. Therefore, before performing weighted fusion, it is necessary to use the mapping function. right Normalization is performed.

[0068] Preferably, the mapping function can be:

[0069]

[0070] Therefore, when the actual micro-expression phase propagation matrix is ​​highly consistent with the desired propagation pattern, When the two are close to 1; when there is no significant correlation between them, Approaching 0.5; when the two show opposite trends, Approaching 0. This mapping process avoids the first confidence component from directly destroying the numerical stability of the fusion result due to taking a negative value, and allows the propagation pattern similarity, temporal consistency, and line-of-sight spatial matching degree to participate in the calculation of exhibit-induced confidence under a unified dimension.

[0071] When exhibits induce confidence Higher than the preset threshold When, the system initiates complex emotion decoupling; when At this time, the system marks the micro-expression as a low-confidence response, using it only for non-trigger statistical recording and not driving the exhibit presentation device to perform interactive actions. Preset threshold. The threshold can be set according to the noise level of the exhibition hall and the sensitivity of the exhibits to interaction, with an optimal value of 0.65 to 0.85. For scenarios that allow for a higher frequency of interaction, such as children's exhibition areas and science popularization interactive exhibition areas, the threshold can be appropriately lowered; for scenarios that emphasize the continuity of the exhibition, such as digital restoration of original cultural relics and immersive narrative theaters, the threshold can be appropriately increased.

[0072] The input to the composite emotion decoupling module includes spatiotemporal micro-expression features, motion amplitude asymmetry index, gaze duration, number of gaze re-looks, pupil change trends, and event semantic labels. The output is a fine-grained emotional state vector for the audience. This vector no longer uses coarse-grained classifications such as "happy," "sad," or "surprised," but instead includes multiple independent or weakly correlated psychological dimension parameters, such as focus, confusion, aesthetic pleasure, tension, and novelty. The fine-grained emotional state vector for the audience can be represented as:

[0073]

[0074] In the formula, This represents the detailed emotional state vector of the audience after the k-th event; Indicates focus; Indicates the level of confusion; Indicates aesthetic pleasure; Indicates the level of tension; This represents the novelty level. Each parameter value can be normalized to 0 or 1. The above vector can be generated using a multi-output regression model, a Bayesian network, or a lightweight neural network.

[0075] Preferably, a quantized and compressed multilayer perceptron model is used on the edge computing device in the exhibition hall to ensure that the single inference delay does not exceed 50ms, so as to guarantee the real-time performance of interactive response.

[0076] Specifically, the decoupling of complex emotions can be achieved using the following model:

[0077]

[0078] In the formula, It represents the feature vector formed by splicing together the micro-expression phase propagation matrix, pulse amplitude, asymmetry index, gaze duration, and pupil change. Semantic tags The embedding vector is used to enable the same facial response to have different interpretations under different cultural semantic events; and Represents the model weight matrix; Indicates the bias term; This represents the Sigmoid function, used to map the output to the range of 0 to 1. The design principle of this model is that the slight twitching of the eyebrows when faced with "magnified visual details" may represent careful observation, while the slight twitching of the eyebrows when faced with "narrative conflict" may represent tension or confusion. By adding semantic label embedding, the system can avoid mechanically mapping the same facial movement to a fixed emotion.

[0079] After obtaining the audience's detailed emotional state vector, the cultural understanding load calculation module further calculates the cultural understanding load index. This index reflects whether the audience is currently in a state of active exploration, understanding blockage, or low engagement. During calculation, the system uses not only focus and confusion levels but also incorporates the gaze saccade entropy value. The gaze saccade entropy value is calculated based on the distribution of audience dwell time among different interest areas of the exhibit. Assuming the exhibit is divided into n interest sub-areas, the proportion of audience dwell time in the i-th sub-area is... Then the entropy value of the gaze path is:

[0080]

[0081] In the formula, This represents the entropy value of the gaze path after the k-th event; This represents the percentage of time that the viewer's attention is focused on the i-th interest sub-region. This represents the number of sub-regions of interest; ε is a very small constant to prevent taking the logarithm of zero. If the viewer's gaze is focused on a few key areas, the entropy value is low, usually indicating that they are concentrating on the target content; if the viewer's gaze jumps randomly between multiple areas, the entropy value is high, which may indicate that their comprehension path is unstable or their attention is wandering.

[0082] The cultural understanding load index can be further expressed as:

[0083]

[0084] In the formula, Indicates the cultural understanding load index; Indicates focus; Indicates the level of confusion; This represents the normalized entropy value of the gaze path. , , The three factors represent weighting coefficients, and their sum is 1. It's important to note that a high cultural understanding load does not necessarily indicate a negative experience. When focus is high, confusion is in the middle range, and gaze entropy is low, the audience may be in an exploratory load state, indicating they are actively understanding the cultural content. When confusion is high and gaze entropy is high, the audience may be in a hindered load state, indicating they have difficulty establishing an effective understanding path. When focus is low, it indicates a low-participation load state.

[0085] The system determines the cognitive engagement state based on the aforementioned parameters. If the focus level is above the first threshold, the confusion level is in the middle range, and the entropy value of the gaze path is below the entropy threshold, it is determined to be an exploratory load, and a low-salience cue strategy is applied. In this case, the exhibit presentation device can display semi-transparent short phrases, softly highlighted edges, or low-volume spatial voice supplements near the relevant artifact patterns without interrupting the viewer's immersion. If the confusion level is above the second threshold and the entropy value of the gaze path is above the entropy threshold, it is determined to be an obstructive load, and an auxiliary explanation presentation strategy is applied. In this case, the system can reduce the narrative speed, pop up local magnified explanations, add background explanations of artifacts, or provide an explanation layer with both text and images. If the focus level is below the third threshold, it is determined to be a low-engagement load, and a strategy of reducing information density or content guidance switching is applied. In this case, the system can reduce the amount of text explanations, switch the exhibit image to a more visually appealing section, or guide the viewer to refocus on key areas through soft lighting effects. All of the above strategies are tied to the viewer's current state and are not applied uniformly to all viewers, thereby improving the targeting of interactive actions.

[0086] The interactive parameter generation module generates a tiered set of interactive parameters based on the audience's detailed emotional state vector, cultural understanding load index, and exhibit-induced confidence level. The interactive parameter set can include content level, cue salience, playback speed, local magnification, explanatory text length, audio volume, light effect intensity, and interaction duration. Its expression format can be:

[0087]

[0088] In the formula, This represents the hierarchical interaction parameter set corresponding to the k-th event; Indicates the level of interactive action; This indicates the length or information density of the explanatory text; Indicates the magnification of a specific part of the exhibit; Indicates luminous efficacy intensity; Indicates audio cue volume or spatial audio weight; Indicates the duration of the interactive action. The interactive action level can be set from 0 to 3, where level 0 means no interaction is triggered, level 1 means a low-salience prompt, level 2 means auxiliary explanations are presented, and level 3 means content path adjustment or switching of key areas.

[0089] Specifically, if the exhibit's evoked confidence level consistently exceeds a preset confidence threshold across multiple response time windows, and the audience's focus remains high, the system increases the level of the interactive action. For example, if an audience responds with high confidence twice consecutively to "enlarged ornamentation" and "inscription interpretation," it indicates sustained interest in such cultural details, and the system can upgrade from level 1 edge highlighting to level 2 local enlargement with short text description. Conversely, if a decrease in focus or the audience's gaze leaves the exhibit's area of ​​interest is detected after the interactive action, the system decreases the level of the interactive action. For example, if the audience's gaze quickly leaves after the system displays excessively long explanatory text, subsequent similar events will have their text length reduced or be replaced with low-salience annotations. Thus, the interaction level is not a one-time static decision but is continuously adjusted based on audience feedback.

[0090] Preferably, the interactive parameter generation module can also set an interactive cooldown time. That is, after an interactive action is executed, unless a high-priority cultural semantic event occurs, no new significant interactive actions will be triggered within 0.8s to 3s. This preferred solution can further prevent exhibits from changing frequently due to the continuous capture of multiple micro-expressions, ensuring that the audience can receive cultural content stably.

[0091] After the interactive action is performed, the feedback correction module acquires a high-frame-rate image stream of the second face within a preset feedback time window, and uses the same or similar micro-expression feature extraction, complex emotion decoupling, and cultural understanding load calculation process as described above to generate a refined emotional state vector of the audience after intervention. Let the emotional state vector before interaction be... The emotional state vector after interaction is The difference result is:

[0092]

[0093] In the formula, This indicates subtle changes in the audience's emotional state before and after the interaction; This represents a vector of the audience's detailed emotional state before the interactive action is performed. This represents the detailed emotional state vector of the audience after the interactive action is performed. If If the interaction parameters are effective for the audience, the following are possible outcomes: increased focus, decreased confusion, or increased aesthetic pleasure. If the audience's focus decreases, confusion increases, or their gaze leaves the area of ​​interest in the exhibit, the interaction may be too intense, too long, or have an inappropriate information density.

[0094] The feedback correction module can adjust the generation rules of the hierarchical interaction parameter set using an incremental update method. For example, for a certain type of semantic event and a certain type of audience state, if the current interaction action receives positive feedback, the weight of the corresponding parameter combination is increased; if negative feedback is received, the weight of the parameter combination is decreased. Parameter updates can be represented as:

[0095]

[0096] In the formula, This indicates the updated interaction generation rule parameters; This indicates the interaction generation rule parameters before the update; This represents the learning rate, with a preferred value between 0.001 and 0.05. This represents the feedback reward value calculated based on the changes in state before and after the interaction; This represents a strategy function that generates a set of interaction parameters given the audience's emotional state, cultural understanding load index, and exhibit evoked confidence level. This represents the gradient with respect to the rule parameters. The purpose of this feedback correction is to enable the system not only to recognize audience reactions, but also to determine whether its own interaction truly improves the viewing experience, thereby avoiding the shortcomings of existing technologies that "trigger interaction immediately upon recognizing emotions and then no longer evaluate the effect."

[0097] In practical deployment, the feedback correction module can also be implemented using a lookup table-based weight update method. For example, the system can establish a parameter weight table based on semantic tags, cultural understanding load type, interactive action level, and feedback results. When a certain combination of interactive parameters increases audience focus, decreases confusion, or increases aesthetic pleasure, the weight of that parameter combination in the corresponding state is increased; when a certain combination of interactive parameters causes audience focus to decrease, their gaze to leave the area of ​​interest of the exhibit, or their confusion to further increase, the weight of that parameter combination is decreased. This lookup table-based weight update method can also adjust the generation rules of the graded interactive parameter set based on the difference between the audience's fine emotional state vectors before and after the intervention.

[0098] Preferably, to facilitate rapid deployment across different exhibition halls, the system can maintain an anonymous visitor session identifier locally without storing the visitor's real identity information. This session identifier is only used during a single visit to record the distribution of the same visitor's reaction time and interactive feedback results to similar cultural semantic events. After the visit, it can be automatically cleared or only the de-identified statistical parameters can be retained, thus helping to balance personalized interaction and privacy protection.

[0099] Preferably, the audience perception acquisition module can be further connected to an environmental noise sensor, a mobile phone Bluetooth beacon detector, or an exhibition hall crowd density sensor. When the environmental noise suddenly increases, a significant crowd gathers nearby, or an audience's mobile phone screen lights up, the system can temporarily increase the exhibit-induced confidence threshold, or adjust the weights of propagation mode similarity, temporal consistency, and line-of-sight spatial matching in the fusion calculation. This preferred solution can further adapt to complex exhibition hall environments.

[0100] For example, the digital cultural exhibit is an interactive digital scroll. During playback, the digital scroll outputs the cultural semantic event "warhorse leaps," which belongs to the narrative conflict category of cultural semantic events, and the output time is... The semantic tag is "narrative conflict type", and the corresponding interest area of ​​the exhibit is the area of ​​action of the warhorse and the surrounding figures in the picture.

[0101] After the "horse leaps" scene is rendered and displayed, the cultural semantic event output module generates an event data packet. The system establishes a reaction time window based on the "narrative conflict" semantic tag, which is [0.4s, 2.5s]. Within this reaction time window, a high frame rate image acquisition device continuously acquires a stream of facial images of the audience at a frame rate of 120fps to 240fps, and the micro-expression processing module performs micro-motion tracking on the glabella, eyelids, nasal alae, corners of the mouth, and cheekbones.

[0102] exist At that time, the system detected micro-expression pulses in the audience's brow area, with a pulse amplitude of 0.12; At that time, micro-expression pulses were detected in the nasal alar region, with a pulse amplitude of 0.09; At that time, a micro-expression pulse was detected in the corner of the mouth area, with a pulse amplitude of 0.07. Based on the peak time differences of the aforementioned key facial regions, the system constructed a micro-expression phase propagation matrix. Matching revealed that the micro-expression phase propagation matrix exhibited a propagation order of response from the glabella area, nasal alae area, and corner of the mouth area, which highly matched the expected propagation pattern corresponding to "narrative conflict type" cultural semantic events. The system calculated:

[0103]

[0104] Simultaneously, the gaze matching module detected that the viewer's gaze was located within the area where the warhorse leaped, and the spatial matching degree between the viewer's gaze and the area of ​​interest of the exhibit corresponding to the current cultural semantic event was: .

[0105] The main peak time for micro-expressions is Its time offset relative to the event output time is 0.95s, which falls within the effective reaction time window of the narrative conflict event. Based on this, the system calculates the second confidence component. After performing interval mapping on the first confidence component, the exhibit-induced confidence calculation module performs weighted fusion based on propagation pattern similarity, temporal consistency, and line-of-sight spatial matching to obtain the exhibit-induced confidence:

[0106] Since the exhibit evoked a confidence level higher than the preset threshold of 0.7, the system determined that the micro-expression was a credible reaction induced by the cultural semantic event of "horse leaping" in the current digital scroll, rather than an irrelevant facial change caused by external factors such as exhibition hall noise, companion communication, or mobile phone notifications.

[0107] Subsequently, the composite emotion decoupling module generates a refined emotional state vector for the audience based on the micro-expression phase propagation matrix, pulse amplitude in each region, motion amplitude asymmetry index, gaze dwell time, and semantic tags. Calculations show that the audience's focus level is 0.82, confusion level is 0.55, and aesthetic pleasure level is 0.68. Gaze analysis results show that the audience's gaze is mainly concentrated on the area where the warhorse leaps and the surrounding character movement areas, with a gaze path entropy value of 0.32, indicating a low entropy state. Based on this, the system determines that the audience is currently in an exploratory load state, meaning they have high focus, moderate confusion, and their gaze is focused on key cultural content areas, indicating they are actively trying to understand the narrative conflict scene.

[0108] In response to the exploratory load state, the interactive parameter generation module generates a Level 1 interactive parameter set. Specifically, the exhibit presentation device applies low-salience edge highlighting to the "horse leaping" area and displays a short-term semi-transparent prompt near this area without obscuring the main content of the original image. This prompt guides the viewer's attention to the narrative relationship between the horse's movements and the character's posture. The prompt lasts for 1.5 seconds, and the light intensity is below a preset high-salience prompt threshold to avoid disrupting the viewer's immersive understanding process.

[0109] After the interactive action was executed, the feedback correction module continued to acquire a second high-frame-rate image stream of the face within a feedback time window of 0.5s to 2s, and generated a detailed emotional state vector of the audience after the intervention. Comparison showed that the audience's focus increased from 0.82 to 0.87, confusion decreased from 0.55 to 0.48, and their gaze remained within the area of ​​the leaping horse and its associated figures. Based on this, the system determined that the Level 1 low-salience interactive action was effective and increased the weight of the parameter combination of "narrative conflict events, exploratory load state, and low-salience edge highlighting." If the same audience or similar audience members exhibit an exploratory load state again under similar cultural semantic events, the system will prioritize the use of this type of low-salience cue strategy.

[0110] In the complete workflow, firstly, the exhibit presentation device outputs cultural semantic events, forming event data packets with time and semantic constraints. Secondly, the system acquires high-frame-rate image streams of the audience's faces within a reaction time window matching the event type, and extracts micro-motion trajectories, pulse features, phase propagation matrices, and asymmetry indices for each key facial region. Subsequently, the system calculates the exhibit-induced confidence level by combining the micro-expression propagation pattern, the time offset of the micro-expression peak relative to the event output time, and the spatial matching degree between the audience's gaze and the exhibit's area of ​​interest. Only when this confidence level meets a threshold condition does the system further perform composite emotion decoupling and cultural understanding load calculation. After that, the interaction parameter generation module generates a hierarchical interaction parameter set based on the audience's fine emotional state, cultural understanding load index, and exhibit-induced confidence level, and drives the exhibit presentation device to execute corresponding levels of interactive actions. Finally, the feedback correction module acquires a second high-frame-rate image stream of the face after the interaction, compares the changes in fine emotional state before and after the intervention, and adjusts the subsequent interaction parameter generation rules accordingly.

[0111] Through the above implementation methods, this invention improves the interactive processing of digital cultural exhibits from the traditional "collecting facial expressions, recognizing emotions, and directly triggering interaction" to a complete technical chain of "cultural semantic event-driven, micro-expression causal attribution, refined emotional decoupling, cognitive load judgment, hierarchical interactive execution, and feedback closed-loop correction." Each interactive action has a clear data source, triggering conditions, processing model, and feedback basis, which can effectively distinguish between genuine micro-expression reactions induced by the exhibit content and irrelevant facial changes caused by external interference, thereby reducing false triggers, minimizing interference with the immersive viewing experience, and enabling digital cultural exhibits to present a more refined and stable interactive experience based on the audience's true understanding and emotional state.

[0112] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0113] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A multi-dimensional perception-driven interactive processing method for digital cultural exhibits, characterized in that, Includes the following steps: S1. Obtain the cultural semantic events output by digital cultural exhibits during the display process, wherein the cultural semantic events carry the output time and semantic tags; S2. Within the reaction time window established after the output time, a high frame rate image stream of the audience's face is acquired, and micro-motion tracking is performed on multiple preset key facial regions to extract the spatiotemporal micro-expression features of the audience under the inducement of the cultural semantic event. S3. Based on the spatiotemporal micro-expression features, combined with the spatial matching degree between the audience's gaze and the area of ​​interest of the exhibit, and the time offset between the time of micro-expression occurrence and the time of output, calculate the exhibit-induced confidence of the micro-expression. S4. When the exhibit induces confidence higher than a preset threshold, composite emotion decoupling is performed based on the spatiotemporal micro-expression features to generate a refined emotional state vector of the audience containing multiple independent psychological dimension parameter values, and the cultural understanding load index reflecting the audience's cognitive input state is calculated. S5. Based on the audience's detailed emotional state vector, cultural understanding load index, and exhibit-induced confidence, generate a hierarchical interaction parameter set to drive the exhibit presentation device to perform corresponding level interactive actions. S6. After the interactive action is executed, a second high-frame-rate image stream of the face within a preset feedback time window is acquired to generate a fine emotional state vector of the audience after intervention. Based on the difference between the fine emotional state vectors of the audience before and after intervention, the generation rules of the hierarchical interactive parameter set are adjusted.

2. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 1, characterized in that, S2 extracts the spatiotemporal micro-expression features of the audience under the influence of the cultural semantic event, specifically including: Peak detection is performed on the motion trajectories of each preset key facial region to determine the pulse start time, peak time, and pulse amplitude of each preset key facial region; based on the peak time delay relationship between different preset key facial regions, a micro-expression phase propagation matrix describing the propagation sequence of facial micro-movements is constructed; simultaneously, based on the motion trajectories of the same preset action unit on the left and right sides of the face, the motion amplitude asymmetry index is calculated.

3. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 2, characterized in that, The construction of the micro-expression phase propagation matrix describing the propagation time sequence of facial micro-movements specifically includes: The multiple preset key facial areas are divided into the glabella area, eyelid area, nasal wing area, corner of mouth area, and cheekbone area; The motion trajectory of each region is normalized, and the pulse start time, peak time and pulse amplitude of each region are extracted; Calculate the peak time difference between any two regions to form the peak propagation delay between regions; The peak propagation delays of all region pairs are combined according to the division of the multiple preset key facial regions to form a multidimensional matrix describing the temporal relationship of micro-motion propagation between regions, which serves as the micro-expression phase propagation matrix.

4. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 1, characterized in that, S3 calculates the exhibit-evoked confidence level of the micro-expressions, specifically including: The first confidence component is calculated based on whether the peak time delay relationship in the spatiotemporal micro-expression features conforms to the preset expected propagation pattern corresponding to the semantic label of the cultural semantic event. The second confidence component is calculated based on whether the time offset between the occurrence time of the micro-expression pulse and the output time is within a preset effective response window. The third confidence component is calculated based on whether the viewer's gaze coincides with the exhibit's area of ​​interest corresponding to the current cultural semantic event. The first, second, and third confidence components are then weighted and fused to obtain the exhibit-induced confidence.

5. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 4, characterized in that, The calculation of the first confidence component specifically includes: The micro-expression phase propagation matrix in the spatiotemporal micro-expression features is matched with the preset expected propagation pattern corresponding to the semantic label, and the Pearson correlation coefficient is calculated. When the Pearson correlation coefficient is higher than the first preset threshold, the first confidence component takes a high value; when it is lower than the second preset threshold, it takes a low value; and when it is in between, it takes a median value. Among them, different semantic tags correspond to different expected propagation patterns. Visual detail events correspond to a short-term peak propagation pattern from the eyelid area to the glabella area, narrative conflict events correspond to a composite propagation pattern from the glabella area, the nasal wing area, and the corner of the mouth area, and aesthetic pleasure events correspond to a peak propagation pattern from the corner of the mouth area to the cheekbone area.

6. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 1, characterized in that, The calculation of the cultural understanding load index, which reflects the audience's cognitive engagement, as described in S4, also includes: Based on the combined relationship between focus and confusion, as well as the entropy value of the eye saccade path, the audience's cognitive engagement state is determined as exploratory load, blocking load, or low-participation load. Specifically, when the focus level is higher than the first threshold, the confusion level is in the middle range, and the entropy value of the gaze saccade path is lower than the entropy value threshold, it is determined to be an exploratory load; when the confusion level is higher than the second threshold and the entropy value of the gaze saccade path is higher than the entropy value threshold, it is determined to be an obstructive load; when the focus level is lower than the third threshold, it is determined to be a low-participation load. Furthermore, the strategy includes a low-salience prompt for the exploratory load matching, an auxiliary explanation presentation strategy for the obstructive load matching, and a strategy to reduce information density or guide content switching for the low-participation load matching.

7. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 1, characterized in that, The duration of the reaction time window described in S2 is initially set according to the type of cultural semantic event that triggers the reaction time window. Visual detail events correspond to the first duration window, and narrative turning point events correspond to the second duration window. The second duration window is longer than the first duration window. Furthermore, based on the distribution of peak moments of micro-expressions of the same audience under similar cultural semantic events, the starting and ending offsets of the reaction time window corresponding to this type of event are adaptively adjusted.

8. The method for multi-dimensional perception-driven interactive processing of digital cultural exhibits according to claim 1, characterized in that, The generation of hierarchical interaction parameter sets described in S5 includes: When the exhibit's evoked confidence level is higher than a preset confidence threshold for multiple consecutive reaction time windows, the level of the interactive action is increased; when, after the interactive action is executed, a decrease in focus is detected in the audience's fine emotional state vector or the audience's gaze leaves the exhibit's area of ​​interest, the level of the interactive action is decreased.

9. A multi-dimensional perception-driven interactive processing system for digital cultural exhibits, characterized in that, include: Digital cultural exhibit display installations; The cultural semantic event output module is used to acquire cultural semantic events output by digital cultural exhibits during the display process. The cultural semantic events carry the output time and semantic tags. The audience perception acquisition module is used to acquire a high frame rate image stream of the audience's face within a reaction time window established after the output time. The micro-expression processing module is used to track micro-movements in multiple preset key facial regions and extract spatiotemporal micro-expression features of the audience under the influence of the cultural semantic events. The gaze matching module is used to obtain the spatial matching degree between the audience's gaze and the areas of interest of the exhibits; The exhibit-induced confidence calculation module is used to calculate the exhibit-induced confidence of the micro-expression based on the spatiotemporal micro-expression features, spatial matching degree, and time offset between the micro-expression occurrence time and the output time. The composite emotion decoupling module is used to perform composite emotion decoupling based on the spatiotemporal micro-expression features when the confidence level induced by the exhibit is higher than a preset threshold, and generate a fine emotional state vector of the audience containing multiple independent psychological dimension parameter values. The cultural understanding load calculation module is used to calculate the cultural understanding load index, which reflects the audience's cognitive engagement state, based on the audience's fine emotional state vector and the audience's gaze path entropy value. The interactive parameter generation module is used to generate a hierarchical interactive parameter set based on the audience's fine emotional state vector, cultural understanding load index and exhibit induced confidence, so as to drive the digital cultural exhibit presentation device to perform corresponding level interactive actions. The feedback correction module is used to acquire a second high-frame-rate facial image stream within a preset feedback time window after the interactive action is executed, generate a fine emotional state vector of the audience after intervention, and adjust the generation rules of the graded interactive parameter set according to the difference between the fine emotional state vectors of the audience before and after intervention.

10. The multi-dimensional perception-driven digital cultural exhibit interactive processing system according to claim 9, characterized in that, The audience perception acquisition module includes at least one high frame rate image acquisition device with a frame rate of 120fps to 240fps. The image acquisition device is installed in front of or to the side of the exhibit by a rigid bracket. The outer periphery of the image acquisition device is provided with a ring-shaped near-infrared supplement light and a frosted diffuser. A polarizing filter is also provided in front of the image acquisition device. The multiple preset key facial areas include the glabella area, eyelid area, nasal wing area, corner of mouth area, and cheekbone area; The micro-expression processing module performs peak detection on the motion trajectory of each preset key facial region to determine the pulse start time, peak time and pulse amplitude of each preset key facial region. Based on the peak time delay relationship between different preset key facial regions, a micro-expression phase propagation matrix is ​​constructed; at the same time, based on the motion trajectory of the same preset action unit on the left and right sides of the face, the motion amplitude asymmetry index is calculated.

Images