Elevator multi-mode crowd feature perception and intelligent advertisement putting method and system

Abstract

The invention provides an elevator multi-mode crowd feature perception and intelligent advertisement putting method and system, and relates to the technical field of intelligent advertisement putting, and the method comprises the steps: collecting multi-mode perception data in an elevator through an edge computing terminal; performing feature decoupling on the data, performing cross-modal semantic alignment, establishing a directed association relationship between modals, and constructing a scene feature map; calculating the topology importance degree of map nodes, screening feature nodes, and extracting context information for semantic coding; mapping the scene semantic code and the advertisement audience semantic code to a two-dimensional coordinate system to construct a semantic matching graph, and extracting an optimal matching path to form a candidate set; predicting a scene evolution trend based on the scene characteristic spectrum evolution trajectory, and calculating an advertisement adaptive score to generate a playing sequence; putting and collecting user interaction data feedback according to the sequence to update the graph structure. According to the invention, accurate crowd feature recognition and advertisement dynamic matching are realized, and the advertisement putting efficiency and the user experience are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent advertisement delivery, and in particular to an elevator multi-modal crowd feature perception and intelligent advertisement delivery method and system. BACKGROUND

[0002] With the rapid development of digital advertising technology and the popularity of smart Internet of Things devices, elevator advertising, as an important media form in offline scenarios, is gradually transforming from the traditional static poster delivery mode to a digital and intelligent direction. As a special scenario with high passenger flow and closed characteristics in urban buildings, passengers have a relatively fixed stay time in the elevator and their attention is relatively concentrated, making elevator advertising have high reach and attention.

[0003] In recent years, with the maturity of computer vision, natural language processing, edge computing and other technologies, advertising delivery has begun to introduce face recognition, behavior analysis and other technical means to identify audience characteristics. Some elevator advertisements install cameras and other perception devices in elevators to collect basic demographic characteristics such as age and gender of passengers, and match simple advertisement content according to these characteristics. At the same time, some systems have also begun to try to use multi-sensor fusion to combine visual, sound, environmental parameters and other multi-dimensional data to more comprehensively perceive the scene in the elevator, in order to improve the accuracy and personalization level of advertising delivery.

[0004] However, the existing elevator advertising intelligent delivery technology still has many deficiencies. When processing multi-modal perception data, a simple feature splicing or weighted fusion method is often used, lacking the mining of deep semantic association relationships between different modal data, resulting in the complementary advantages of multi-modal information not being fully utilized, the feature expression ability being limited, and complex crowd characteristics and scene states being difficult to accurately depict. The existing advertisement matching method mainly recommends content based on static audience portraits, ignoring the dynamic change characteristics of the crowd composition in the elevator scene, failing to capture the evolution trend of the scene, resulting in a lack of foresight and coherence in advertising delivery. There is a lack of effective feedback mechanism and self-adaptive optimization capability, the user interaction data generated during the delivery process is not fully utilized, and the matching strategy cannot be dynamically adjusted according to the actual delivery effect, making it difficult to continuously optimize the delivery accuracy and the efficiency of advertising delivery has limited room for improvement. SUMMARY

[0005] The present application provides an elevator multi-modal crowd feature perception and intelligent advertisement delivery method and system, which can solve the problems in the prior art.

[0006] In a first aspect of the embodiments of the present application, an elevator multi-modal crowd feature perception and intelligent advertisement delivery method is provided, comprising: collecting multi-modal perception data in the elevator through an edge computing terminal; feature decoupling on multi-modal perception data to obtain a decoupled feature set, cross-modal semantic alignment on the decoupled feature set, establishment of a directed association relationship between features of different modalities, and construction of a scene feature graph; topological importance of each node in the scene feature graph, filtering of feature nodes based on a preset importance threshold, extraction of context information of the feature nodes and semantic encoding, and generation of scene semantic encoding; mapping of the scene semantic encoding and audience semantic encoding of the advertisement material to a two-dimensional coordinate system, construction of a semantic matching graph, extraction of an optimal matching path, selection of advertisement materials along the optimal matching path to construct a candidate set; extraction of an evolution trajectory of the scene feature graph, generation of a scene evolution trend, calculation of an adaptation score of the advertisement materials in the candidate set, and sorting to generate a play sequence according to the adaptation score; According to the play sequence, the advertisement is put into operation, and user interaction data in the process of putting into operation is collected, and the user interaction data is fed back to the scene feature graph for graph structure updating.

[0007] In an optional embodiment, feature decoupling on multi-modal perception data to obtain a decoupled feature set, cross-modal semantic alignment on the decoupled feature set, establishment of a directed association relationship between features of different modalities, and construction of a scene feature graph include: modal separation on multi-modal perception data, extraction of initial feature vectors of each modality, decoupling transformation on the initial feature vectors of each modality, generation of independent decoupled feature components, and aggregation of the decoupled feature components to form a decoupled feature set; projection of each decoupled feature component to a unified continuous semantic manifold space, identification of a plurality of semantic cluster centers in the continuous semantic manifold space through adaptive density clustering, determination of a cross-modal alignment reference anchor point, calculation of the geodesic distance of each decoupled feature component in the continuous semantic manifold space along the manifold surface to each reference anchor point, construction of an affinity matrix of cross-modal semantic similarity according to the geodesic distance, Laplacian spectral decomposition on the affinity matrix to obtain a feature subspace, and establishment of a bidirectional semantic mapping relationship between each decoupled feature component in the feature subspace by minimizing reconstruction error, to complete cross-modal semantic alignment; Based on the bidirectional semantic mapping relationship, identifying decoupled feature component pairs with semantic association, performing causal structure learning on the decoupled feature component pairs, and determining the directed association relationship between different modalities; Taking each decoupled feature component in the decoupled feature set as a node and the directed association relationship as a directed edge, a scene feature graph is constructed.

[0008] In an optional embodiment, based on the bidirectional semantic mapping relationship, a pair of decoupled feature components with semantic association is identified, a causal structure learning is performed on the pair of decoupled feature components, and the directed association relationship between different modalities is determined, including: Based on the bidirectional semantic mapping relationship, a pair of decoupled feature components belonging to different modalities is screened out, and a cross-modal feature component pair set is formed; Evolution trajectory data in a plurality of time windows is extracted for each group of decoupled feature component pairs in the cross-modal feature component pair set, the prediction contribution degree of the preceding evolution trajectory of the first decoupled feature component to the subsequent evolution trajectory of the second decoupled feature component and the prediction contribution degree of the preceding evolution trajectory of the second decoupled feature component to the subsequent evolution trajectory of the first decoupled feature component are calculated respectively, and any prediction contribution degree exceeding a preset contribution threshold is marked as a causal candidate pair; For each causal candidate pair, a disturbance amount is superimposed on the first decoupled feature component, a calculated response value is obtained by mapping to the semantic space of the second decoupled feature component based on the bidirectional semantic mapping relationship, the deviation amplitude of the calculated response value from the actual observation value is calculated to determine the strength of the positive causal effect, a disturbance amount is superimposed on the second decoupled feature component and mapped to the semantic space of the first decoupled feature component, the deviation amplitude is calculated to determine the strength of the reverse causal effect, and when the strength of the positive causal effect is greater than the strength of the reverse causal effect, a directed causal edge from the first decoupled feature component to the second decoupled feature component is established, otherwise a directed causal edge from the second decoupled feature component to the first decoupled feature component is established; The directed causal edges meeting the preset strength requirement are screened out to determine the directed association relationship between different modalities.

[0009] In an optional embodiment, the topological importance of each node in the scene feature graph is calculated, the feature nodes are screened based on a preset importance threshold, the context information of the feature nodes is extracted and semantically encoded, and the scene semantic code is generated, including: The nodes in the scene feature graph are traversed, the degree value of each node is calculated based on the number of connection edges, the local topological importance is determined based on the ratio of the sum of the degree values of all adjacent nodes of the node to the degree value of the node, and the global topological importance is determined based on the ratio of the number of nodes reachable from the node within a preset hop number to the total number of nodes; the neighborhood stability indicator is calculated based on the variance value of the connection edge weight of the node, and when the neighborhood stability indicator is less than or equal to a preset stability threshold, the local topological importance is taken as the node topological importance, and when the neighborhood stability indicator is greater than the stability threshold, the global topological importance is taken as the node topological importance; The nodes exceeding the preset importance threshold are screened out according to the node topological importance to obtain the feature nodes; For the feature node, a directly adjacent node is extracted, an edge type identifier and an edge weight between the feature node and the directly adjacent node are obtained, attributes of the directly adjacent node, the edge type identifier and the edge weight are combined to construct a context description, the edge weight is corrected based on a shortest path length between the feature node and the directly adjacent node, and the context information is generated by sorting the context description according to the corrected edge weight in series; and semantic coding is performed on the context information to obtain a semantic coding vector, and the semantic coding vector is fused to generate scene semantic coding.

[0010] In an optional embodiment, the scene semantic coding and audience semantic coding of the advertisement material are mapped to a two-dimensional coordinate system to construct a semantic matching graph, an optimal matching path is extracted, and the advertisement material is selected along the optimal matching path to construct a candidate set, which includes: The audience semantic coding of all advertisement materials in the local advertisement content library is extracted. The scene semantic coding is taken as a query vector, an attention score between the query vector and each audience semantic coding is calculated, and a semantic space mapping reference vector is obtained by weighted summation of the audience semantic coding according to the attention score. For each advertisement material, a projection length of the corresponding audience semantic coding in the direction of the semantic space mapping reference vector and a deviation distance perpendicular to the direction of the semantic space mapping reference vector are calculated, which correspond to determining a first-dimensional coordinate and a second-dimensional coordinate respectively, to obtain a node position. Based on the node position, an Euclidean distance between nodes is calculated, a directed edge is established when the Euclidean distance is less than a preset distance threshold, and a semantic matching graph is formed. A position of the query vector in the semantic matching graph is calculated as a starting position, and a node closest to the starting position is selected as a path starting point. Adjacent nodes are traversed from the path starting point, a difference value between the first-dimensional coordinates of the adjacent nodes and the path starting point is calculated to determine a forward direction score, an adjacent node with the maximum forward direction score is selected to join the path and serve as a new starting point, and the process is repeated until a preset path length is reached, to obtain an optimal matching path. Advertisement materials corresponding to nodes on the optimal matching path are extracted, and a candidate set is constructed by arranging the advertisement materials in order of joining.

[0011] In an optional embodiment, an evolution trajectory of the scene feature graph is extracted, a scene evolution trend is generated, an adaptation score of the advertisement material in the candidate set is calculated, and a playing sequence is generated by sorting according to the adaptation score, which includes: A historical snapshot sequence of the scene feature graph within a preset time window is extracted, for each historical snapshot, node attribute features and edge weights are extracted, a node attribute feature difference vector between adjacent historical snapshots and an edge weight change rate are calculated, the node attribute feature difference vector and the edge weight change rate are spliced to form a snapshot evolution feature, and the snapshot evolution features are arranged in time sequence to form an evolution trajectory. The evolutionary trajectory is time series decomposed to obtain a trend component and a periodic component, a prediction difference vector is determined based on the trend component, and a prediction change rate is determined based on the periodic component; All the advertising materials in the candidate set are traversed to extract audience semantic encoding and historical delivery data; A semantic matching degree is determined by calculating the cosine similarity between the audience semantic encoding and the prediction difference vector, a correlation coefficient is determined by calculating the Pearson correlation coefficient between the historical delivery data and the prediction change rate, and a fitting score is obtained by weighted summation of the semantic matching degree and the correlation coefficient; The advertising materials are ranked in descending order according to the fitting score to generate a playing sequence.

[0012] In an optional embodiment, the time series decomposition of the evolutionary trajectory obtains a trend component and a periodic component, the prediction difference vector is determined based on the trend component, and the prediction change rate is determined based on the periodic component, which includes: The evolutionary trajectory is time series decomposed to extract a trend component and a periodic component; A fitting straight line is obtained by least squares fitting of the trend component, a transition vector set between adjacent snapshot evolutionary features in the evolutionary trajectory is constructed, transition mode clusters are determined by clustering the transition vector set, a centroid vector with the highest similarity to the slope vector of the fitting straight line is selected, a dominant transition vector is determined, and a prediction difference vector is determined in combination with a preset time step; Autocorrelation analysis is performed on the periodic component, the first peak position in the autocorrelation coefficient sequence is searched to determine the period length, the edge weight change rate waveform within the period length is extracted, the corresponding amplitude change gradient is calculated, the amplitude decay coefficient is determined based on the amplitude change gradient, the edge weight change rate waveform is shifted backward by the product of the period length and a preset multiple, and the prediction change rate is determined in combination with the correction of the amplitude decay coefficient.

[0013] In a second aspect of the embodiment of the application, an elevator multi-modal crowd feature perception and advertising intelligent delivery system is provided, which includes: A first unit is configured to collect multi-modal perception data in the elevator through an edge computing terminal; A second unit is configured to decouple features of the multi-modal perception data to obtain a decoupled feature set, perform cross-modal semantic alignment on the decoupled feature set, establish a directed association relationship between different modal features, and construct a scene feature map; A third unit is configured to calculate the topological importance of each node in the scene feature map, filter feature nodes based on a preset importance threshold, extract context information of the feature nodes and perform semantic encoding to generate scene semantic encoding; The fourth unit is configured to map the scene semantic code to the audience semantic code of the advertisement material to a two-dimensional coordinate system, construct a semantic matching graph, extract an optimal matching path, and select advertisement materials along the optimal matching path to construct a candidate set; The fifth unit is configured to extract an evolution track of the scene feature graph, generate a scene evolution trend, calculate an adaptation score of the advertisement material in the candidate set, and generate a playing sequence according to the adaptation score. The sixth unit is configured to perform advertisement delivery according to the playing sequence, collect user interaction data in the delivery process, and feed back the user interaction data to the scene feature graph for graph structure updating.

[0014] In a third aspect, an electronic device is provided, including: a processor; a memory configured to store processor-executable instructions; The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0015] In a fourth aspect, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0016] In the embodiment of the present application, by deploying edge computing terminals in the elevator scene to collect multi-modal perception data, and decoupling features and aligning cross-modal semantics, a complete scene feature graph is constructed, which can fully capture multi-dimensional feature information of the crowd in the elevator, breaking through the limitations of traditional single-modal perception, and realizing accurate recognition and deep understanding of crowd features. By calculating the topological importance of nodes in the scene feature graph and performing feature screening, scene semantic codes are generated, which are then mapped to a two-dimensional coordinate system with audience semantic codes of advertisement materials and matched with optimal paths, realizing accurate matching of advertisement materials and target audiences. In combination with the adaptation score of the advertisement material calculated based on the scene evolution trend, a playing sequence is dynamically generated, making the advertisement delivery more in line with real-time scene needs, significantly improving the targeting and effectiveness of the advertisement delivery, and improving the advertisement conversion rate. A real-time feedback mechanism for user interaction data is established, and the user interaction data collected in the delivery process is fed back to the scene feature graph for graph structure updating, forming a closed-loop optimization. This continuous learning and adaptive adjustment mechanism can continuously optimize the scene understanding ability and the accuracy of advertisement matching, realizing continuous improvement of the advertisement delivery strategy and continuous improvement of the effect. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 FIG. 1 is a flowchart of an elevator multi-modal crowd feature perception and intelligent advertisement delivery method according to an embodiment of the present application; Figure 2A flowchart is provided for cross-modal data causal relationship mining. DETAILED DESCRIPTION

[0018] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0019] The technical solutions of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and some embodiments can not be described again for the same or similar concepts or processes.

[0020] Figure 1 A flowchart of the elevator multi-modal crowd feature perception and intelligent advertisement delivery method of the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps. Figure 1 collecting multi-modal perception data in the elevator through an edge computing terminal; decoupling features of the multi-modal perception data to obtain a decoupled feature set, performing cross-modal semantic alignment on the decoupled feature set, establishing a directed association relationship between different modal features, and constructing a scene feature graph; calculating a topological importance of each node in the scene feature graph, filtering feature nodes based on a preset importance threshold, extracting context information of the feature nodes and performing semantic encoding, and generating scene semantic encoding; mapping the scene semantic encoding and audience semantic encoding of an advertisement material to a two-dimensional coordinate system, constructing a semantic matching graph, extracting an optimal matching path, selecting advertisement materials along the optimal matching path to construct a candidate set; extracting an evolution trajectory of the scene feature graph, generating a scene evolution trend, calculating an adaptation score of the advertisement materials in the candidate set, and generating a play sequence in order of the adaptation score; performing advertisement delivery according to the play sequence, collecting user interaction data in the delivery process, and feeding back the user interaction data to the scene feature graph for graph structure updating.

[0021] In an optional embodiment, decoupling features of the multi-modal perception data to obtain a decoupled feature set, performing cross-modal semantic alignment on the decoupled feature set, establishing a directed association relationship between different modal features, and constructing a scene feature graph comprises: ​Modal separation is performed on multimodal sensing data, initial feature vectors of each modality are extracted, decoupling transformation is performed on the initial feature vectors of each modality to generate mutually independent decoupled feature components, and the decoupled feature components are aggregated to form a decoupled feature set; Each decoupled feature component is projected onto a unified continuous semantic manifold space. In the continuous semantic manifold space, multiple semantic cluster centers are identified through adaptive density clustering to determine the reference anchor points for cross-modal alignment. The geodesic distance from each decoupled feature component to each reference anchor point along the manifold surface in the continuous semantic manifold space is calculated. Based on the geodesic distance, an affinity matrix for cross-modal semantic similarity is constructed. The affinity matrix is ​​subjected to Laplace spectral decomposition to obtain a feature subspace. In the feature subspace, a bidirectional semantic mapping relationship between each decoupled feature component is established by minimizing the reconstruction error, thus completing the cross-modal semantic alignment. Based on the bidirectional semantic mapping relationship, decoupled feature component pairs with semantic association are identified, causal structure learning is performed on the decoupled feature component pairs, and directed association relationships between different modalities are determined. Using each decoupled feature component in the decoupled feature set as a node and the directed association relationship as a directed edge, a scene feature graph is constructed.

[0022] In one specific implementation, the multimodal perception data comes from multiple sensors installed inside the elevator, including but not limited to cameras, microphones, and gravity sensors. These sensors can collect multimodal data such as images, sounds, and weights of people inside the elevator. The multimodal perception processing module processes this data to achieve comprehensive perception of the characteristics of people inside the elevator.

[0023] When performing modal separation on multimodal sensing data, a hierarchical sensing strategy is adopted. For example, visual data captured by cameras is processed through a convolutional feature extraction network to extract visual feature vectors of the crowd; audio data captured by microphones is processed through spectral analysis to extract acoustic feature vectors; and load feature vectors are obtained from gravity sensor data through temporal feature extraction. These initial feature vectors contain multidimensional information about the crowd, including visual, auditory, and load information.

[0024] In the decoupling transformation process of the initial feature vectors of each modality, a variational autoencoder structure is used to decompose the feature vectors of each modality. Taking visual features as an example, they can be decoupled into feature components representing independent semantic dimensions such as age, gender, and dressing style. Audio features can be decoupled into dimensions such as tone, volume, and emotion. Load features can be decoupled into dimensions such as the number of people and density. The decoupling process is achieved by maximizing the mutual information difference between the feature components to ensure the independence between the components. In practical applications, visual features can be decoupled into 10 independent semantic components, audio features can be decoupled into 6 independent components, and load features can be decoupled into 3 independent components, forming a total of 19 decoupled feature components, which together constitute the decoupled feature set.

[0025] When projecting each decoupled feature component to a unified continuous semantic manifold space, a manifold learning method is used to construct a mapping function from high-dimensional features to a low-dimensional semantic space. Specifically, the t-SNE dimension reduction technique can be used to map decoupled feature components of different modalities to the same 50-dimensional semantic manifold space, preserving the local structure relationship between the original feature components. In this semantic manifold space, feature components with similar semantics will be mapped to nearby areas, laying the foundation for subsequent cross-modal alignment.

[0026] When identifying multiple semantic cluster centers in the continuous semantic manifold space through adaptive density clustering, a density peak clustering algorithm is used to automatically determine the semantic cluster centers based on the local density and relative distance of each point in the manifold space. In the elevator scenario, typical semantic cluster centers such as "young white-collar group", "middle-aged and elderly group", and "student group" can be identified. These cluster centers serve as reference anchor points for cross-modal alignment. For the elevator advertising delivery scenario, eight typical people semantic cluster centers can be determined as reference anchor points, representing different people image categories.

[0027] When calculating the geodesic distance of each decoupled feature component in the continuous semantic manifold space along the manifold surface to each reference anchor point, the Dijkstra algorithm is used to solve the shortest path on the manifold graph structure. By establishing a neighbor graph in the manifold space, the shortest distance along the manifold surface between any two points is calculated, rather than the simple Euclidean distance, which can more accurately reflect the semantic similarity on the high-dimensional manifold. For example, when calculating the geodesic distance between the "dressing style" feature component in the visual modality and the "young white-collar group" anchor point, the resulting value reflects the semantic relevance of this feature to the people category.

[0028] When constructing the affinity matrix of cross-modal semantic similarity based on geodesic distance, the Gaussian kernel function is used to convert the geodesic distance into affinity. For any two decoupled feature components, the higher the cosine similarity between their geodesic distance vectors and the reference anchor points, the more similar their distributions in the semantic space, and the higher the affinity. In practical applications, the constructed affinity matrix has a dimension of 19x19, representing the semantic similarity relationship between 19 decoupled feature components.

[0029] When performing Laplacian spectral decomposition on the affinity matrix to obtain the feature subspace, first construct the Laplacian matrix of the affinity matrix, and then solve the eigenvalues and eigenvectors of the Laplacian matrix. Select the eigenvectors corresponding to the smallest non-zero eigenvalues to form the feature subspace. In the elevator advertising scenario, the eigenvectors corresponding to the first 8 non-zero eigenvalues can be selected to form an 8-dimensional feature subspace, which contains the main structural information of the cross-modal semantic relationship.

[0030] When minimizing the reconstruction error in the feature subspace to establish the bidirectional semantic mapping relationship between decoupled feature components, a multilayer perceptron network is used to construct the mapping function. For each pair of decoupled feature components, a mapping network is trained to enable it to predict the other component from one component while minimizing the mean square error between the predicted value and the true value. For example, a mapping relationship can be established from the "age" feature component in the visual modality to the "tone" feature component in the audio modality, reflecting the differences in voice features of different age groups. For 19 decoupled feature components, a total of 19x18=342 bidirectional mapping relationships need to be constructed.

[0031] Based on the bidirectional semantic mapping relationship, when identifying decoupled feature component pairs with semantic association, the mutual information and conditional mutual information between each pair of feature components are calculated. When the mutual information between two feature components exceeds a predetermined threshold, they are considered to have semantic association. In the elevator advertising scenario, it may be found that "dressing style" in the visual modality and "voice style" in the audio modality have a high degree of semantic association, reflecting the social attribute characteristics of the crowd.

[0032] When learning the causal structure of decoupled feature component pairs to determine the directed association relationship between different modalities, a score-based causal discovery algorithm is used. This algorithm compares the likelihood scores of different causal models to determine the optimal causal structure. For example, it may be found that the "age" feature component in the visual modality has a causal effect on the "volume" feature component in the audio modality, indicating that there are systematic differences in the speaking volume of people of different ages in the elevator environment.

[0033] In constructing the scene feature graph, the decoupling feature components in the decoupling feature set are taken as nodes, and the directed association relationship is taken as a directed edge. Each node represents a decoupling feature component, and the node attributes include modal type, semantic label, importance, etc. The directed edge represents the causal relationship between the nodes, and the edge attributes include relationship strength, confidence, etc. Through the graph, the multi-modal semantic association structure of the crowd features in the elevator environment can be intuitively expressed, providing a decision basis for subsequent intelligent advertisement delivery. The elevator scene feature graph can include 19 feature nodes and about 50 directed association edges, forming a complex multi-modal semantic network. Based on the graph, comprehensive understanding of the crowd features in the elevator can be achieved, and then a precise advertisement delivery strategy can be realized.

[0034] In an optional implementation, based on the bidirectional semantic mapping relationship, a pair of decoupling feature components with semantic association is identified, a causal structure learning is performed on the pair of decoupling feature components, and the directed association relationship between different modalities is determined. Based on the bidirectional semantic mapping relationship, a pair of decoupling feature components belonging to different modalities is screened out, forming a cross-modal feature component pair set; Evolution trajectory data in a plurality of time windows are extracted for each group of decoupling feature component pairs in the cross-modal feature component pair set, the prediction contribution degree of the preceding evolution trajectory of the first decoupling feature component to the subsequent evolution trajectory of the second decoupling feature component, and the prediction contribution degree of the preceding evolution trajectory of the second decoupling feature component to the subsequent evolution trajectory of the first decoupling feature component are respectively calculated, and any prediction contribution degree exceeding a preset contribution threshold is marked as a causal candidate pair; For each causal candidate pair, a disturbance amount is superimposed on the first decoupling feature component, a calculated response value is obtained by mapping to the semantic space of the second decoupling feature component based on the bidirectional semantic mapping relationship, a deviation amplitude between the calculated response value and the actual observation value is calculated to determine the strength of the positive causal effect, a disturbance amount is superimposed on the second decoupling feature component and mapped to the semantic space of the first decoupling feature component, and the deviation amplitude is calculated to determine the strength of the reverse causal effect, and when the strength of the positive causal effect is greater than the strength of the reverse causal effect, a directed causal edge from the first decoupling feature component to the second decoupling feature component is established, otherwise a directed causal edge from the second decoupling feature component to the first decoupling feature component is established; The directed causal edges meeting the preset strength requirement are screened out to determine the directed association relationship between different modalities.

[0035] In one specific implementation, the process of filtering decoupled feature component pairs based on bidirectional semantic mapping relationships employs a relevance threshold filtering mechanism. For established bidirectional semantic mapping relationships, the mutual information value of each mapping pair is calculated. When the mutual information value exceeds a preset threshold of 0.6, the pair of feature components is marked as having significant semantic association. Specifically, only feature component pairs belonging to different modalities are retained, forming a cross-modal feature component pair set. In an elevator scenario, a cross-modal pair might be selected consisting of the "dressing style" feature component in the visual modality and the "voice style" feature component in the audio modality, or a cross-modal pair consisting of the "people density" feature component in the visual modality and the "weight change" feature component in the load modality. In this way, approximately 30 pairs of cross-modal feature component pairs with significant semantic associations are selected from 19 decoupled feature components.

[0036] When extracting evolutionary trajectory data for each feature component pair in the cross-modal feature component pair set, the feature change sequence is collected within multiple consecutive time windows, with time as the axis. In specific implementation, a base time window length of 5 seconds and a sliding step size of 1 second can be set, continuously collecting data from 30 time windows to form a feature evolutionary trajectory of 150 seconds. Taking the elevator's operation from the first to the tenth floor as an example, the dynamic changes in the visual characteristics "age distribution" and audio characteristics "conversation volume" of people inside the elevator can be recorded throughout the entire elevator operation cycle. These evolutionary trajectory data are organized into a time series format, with each time point corresponding to a feature value vector.

[0037] Granger causality analysis framework was used to calculate the predictive contribution. For the first and second feature components in a feature component pair, the time series data were divided into a preceding part and a subsequent part, respectively. The preceding part was defined as time window 1 to 20, and the subsequent part was defined as time window 11 to 30, with 10 overlapping windows. By constructing a predictive model, the preceding evolutionary trajectory of the first feature component was used to predict the subsequent evolutionary trajectory of the second feature component, and the prediction accuracy was calculated; similarly, the preceding evolutionary trajectory of the second feature component was used to predict the subsequent evolutionary trajectory of the first feature component, and the prediction accuracy was also calculated. The predictive contribution was quantified by the proportion of variance explained by the predictive model. When the predictive contribution exceeded a preset contribution threshold of 0.3, the feature component pair was marked as a causal candidate pair. In the elevator scenario, for example, it was found that the change in the "proportion of young people" feature component could effectively predict the subsequent change in the "frequency of conversational voices" feature component, with a predictive contribution reaching 0.42, exceeding the threshold, and therefore marked as a causal candidate pair.

[0038] The intervention experiment is performed for each causal candidate pair to verify the true causal relationship. When superimposing the perturbation amount on the first decoupled feature component, a distribution-based perturbation generation method is adopted. Specifically, based on the historical distribution characteristics of the first feature component, a Gaussian perturbation with a mean of 0 and a standard deviation of 20% of the standard deviation of the feature component is generated, and the perturbation is superimposed on the original feature value. In the elevator scene instance, a random perturbation of ±15% may be superimposed on the "young people proportion" feature component. Based on the established bidirectional semantic mapping relationship, the first feature component after superimposing the perturbation is mapped to the semantic space of the second feature component to obtain the calculated response value. For example, the perturbed "young people proportion" feature component is converted to the expected change value of the "conversation sound frequency" through the mapping relationship.

[0039] When calculating the deviation amplitude of the calculated response value and the actual observation value, the standardized root mean square deviation metric is adopted. The deviation amplitude value is standardized to a value between 0 and 1 as a positive causal effect strength indicator. Similarly, the same distribution characteristic perturbation amount is superimposed on the second decoupled feature component, and the deviation amplitude of the calculated response value and the actual observation value is calculated through the bidirectional semantic mapping relationship to determine the reverse causal effect strength. In the above elevator scene instance, the positive causal effect strength of "young people proportion" on "conversation sound frequency" may be measured as 0.65, while the reverse causal effect strength of "conversation sound frequency" on "young people proportion" is 0.28.

[0040] The causal direction is determined by comparing the positive and reverse causal effect strengths. When the positive causal effect strength is greater than the reverse causal effect strength, a directed causal edge from the first decoupled feature component to the second decoupled feature component is established; otherwise, a directed causal edge from the second decoupled feature component to the first decoupled feature component is established. As in the above example, since the positive causal effect strength 0.65 is greater than the reverse causal effect strength 0.28, a directed causal edge from "young people proportion" to "conversation sound frequency" is established, indicating that the change in the proportion of young people in the elevator will lead to the change in the conversation sound frequency, not vice versa.

[0041] When screening directed causal edges that meet the preset strength requirement, the causal strength threshold is set to 0.4. Only directed edges with causal effect strength exceeding the threshold are retained, and weak causal relationships are filtered out to ensure that the identified causal relationships have practical significance. In the elevator multi-modal crowd feature perception scene, a total of 15 strong causal relationships may be finally identified to constitute a directed association network across modalities. These relationships include a causal edge from the "crowd density" feature component of the visual modality to the "background noise level" feature component of the audio modality with a causal effect strength of 0.72, a causal edge from the "total weight" feature component of the load modality to the "body posture" feature component of the visual modality with a causal effect strength of 0.56, and so on.

[0042] In practical applications, to improve the robustness of causal discovery, data augmentation techniques can be used to generate more samples by adding different levels of random noise to the original feature evolution trajectory to generate 100 groups of simulated trajectory data. The above causal analysis process is repeated for each group of data, and stable causal relationships are determined through statistical significance tests.

[0043] In this embodiment, the influence of accidental factors can be effectively reduced, and the accuracy of causal relationship identification can be improved. For the elevator advertising scenario, the identified cross-modal causal relationship can reveal the internal relationship between the characteristics of the crowd, for example, it is found that the "dressing style" feature will affect the "shopping preference" feature, and the causal effect strength is 0.61, which provides an important basis for subsequent precise advertising. Through these directed association relationships, a complete causal network of crowd characteristics in the elevator scenario can be constructed to support precise matching of advertising content and target crowd characteristics and improve advertising effectiveness.

[0044] As shown in Figure 2 The cross-modal data causal relationship mining process diagram is shown.

[0045] In an optional implementation, the topological importance of each node in the scene feature graph is calculated, the feature nodes are filtered based on a preset importance threshold, the context information of the feature nodes is extracted and semantically encoded, and the scene semantic code is generated, including: Traverse the nodes in the scene feature graph, calculate the number of connected edges for each node to determine the node degree value, determine the local topological importance based on the ratio of the sum of the node degree values of all adjacent nodes to the node degree value, and determine the global topological importance based on the ratio of the number of nodes reachable from the node within a preset hop number to the total number of nodes; calculate the edge weight variance value of the node to determine the neighborhood stability indicator, when the neighborhood stability indicator is less than or equal to a preset stability threshold, the local topological importance is taken as the node topological importance, and when the neighborhood stability indicator is greater than the stability threshold, the global topological importance is taken as the node topological importance; According to the node topological importance, the nodes exceeding the preset importance threshold are filtered to obtain the feature nodes; For the feature nodes, the directly adjacent nodes are extracted, the edge type identifier and edge weight between the feature nodes and the directly adjacent nodes are obtained, the attributes, edge type identifier and edge weight of the directly adjacent nodes are combined to construct the context description, the edge weight is corrected based on the shortest path length between the feature nodes and the directly adjacent nodes, and the context information is generated by sorting and concatenating the context description according to the corrected edge weight; the context information is semantically encoded to obtain a semantic encoding vector, and the semantic encoding vector is fused to generate a scene semantic code.

[0046] In one specific implementation, all nodes in the scene feature graph are traversed and a topology importance analysis is performed for each node. Illustratively, the scene feature graph contains 19 nodes, each representing a decoupled feature component of a different modality. For each node, the number of edges connected to it is calculated as the node degree value. For example, the "age distribution" feature node in the visual modality is connected to 6 edges, and thus has a node degree value of 6; while the "emotion fluctuation" feature node in the audio modality is connected to 4 edges, and thus has a node degree value of 4.

[0047] The local topology importance is calculated based on the adjacent node degree values of the node. Specifically, the sum of the degree values of all adjacent nodes directly connected to the current node is calculated and divided by the degree value of the current node to obtain the local topology importance. Taking the "age distribution" node as an example, the degree values of its 6 adjacent nodes are 5, 4, 3, 7, 2, and 3, respectively, and the sum of the degree values is 24, so the local topology importance is 24 divided by 6, which is equal to 4.0. This index reflects the connection importance of the node in the local network structure, and the larger the value, the more highly valued nodes the node is connected to, and the stronger the hub role it plays in the local network.

[0048] When calculating the global topology importance, the preset hop number is set to 2, the number of nodes reachable from the current node in no more than 2 hops is counted, and divided by the total number of nodes in the graph. Taking the "dressing style" node as an example, the node is reachable in one hop to 5 nodes, and these nodes are further reachable in one hop to another 7 nodes, so a total of 13 nodes (including itself) are reachable, and the total number of nodes is 19, so the global topology importance is 13 / 19 ≈ 0.684. This index reflects the reachability and influence range of the node in the entire graph, and the larger the value, the more extensive the feature space that the node can influence.

[0049] The neighborhood stability index is calculated to determine which topology importance to use. The neighborhood stability index is determined by the variance value of the edge weights connected to the node. The edge weight represents the correlation strength between feature components, and its value ranges from 0 to 1. For example, the edge weights of the 6 connections of the "age distribution" node are 0.72, 0.65, 0.83, 0.56, 0.79, and 0.68, respectively. The variance of this set of values is calculated to obtain the neighborhood stability index of 0.0095. A preset stability threshold is set to 0.01, and when the neighborhood stability index is less than or equal to the threshold, it indicates that the connection strength between the node and its neighbors is relatively balanced, and the local topology importance is used as the node topology importance; otherwise, the global topology importance is used. In the example of the "age distribution" node, since the neighborhood stability index 0.0095 is less than the stability threshold 0.01, the local topology importance 4.0 is used as the topology importance of the node.

[0050] For the "dressing style" node, its 5 connection edge weights are 0.45, 0.82, 0.38, 0.91, and 0.53, and the neighborhood stability index is calculated to be 0.0465, which is greater than the stability threshold 0.01, so the global topology importance 0.684 is used as the topology importance of the node. This dynamic selection method can adaptively select a more suitable topology importance calculation method according to the stability degree of the connection characteristics of the node neighborhood.

[0051] When selecting feature nodes according to node topology importance, a preset importance threshold of 0.5 is set. For all nodes in the graph, nodes with a topology importance greater than 0.5 are retained as feature nodes. In the elevator scene feature graph, 8 feature nodes are retained after screening, including "age distribution" (importance 4.0), "dressing style" (importance 0.684), "crowd density" (importance 3.2) in the visual modality; "conversation topic" (importance 0.579), "emotion fluctuation" (importance 0.632) in the audio modality; and "weight distribution" (importance 0.526) in the load modality. These feature nodes represent the most representative and influential crowd feature dimensions in the elevator scene.

[0052] For each feature node, its directly connected nodes are extracted to construct a context description. Taking the "age distribution" feature node as an example, its directly connected nodes include "dressing style", "conversation topic", "shopping preference", "occupation type", "activity period", and "education level". The edge type identifier and edge weight between the feature node and these adjacent nodes are obtained. The edge type identifier is divided into "influence" and "correlation", representing causal relationship and correlation relationship respectively. For example, the edge type between "age distribution" and "dressing style" is "influence", and the edge weight is 0.72; the edge type between "age distribution" and "conversation topic" is "influence", and the edge weight is 0.65.

[0053] The attributes, edge type identifiers, and edge weights of the directly connected nodes are combined to construct a context description. Node attributes include modality type, feature category, and value range. For example, the attribute description of the "dressing style" node is "visual modality, appearance feature, sub-type". The combined context description fragment is "age distribution influences dressing style, correlation strength 0.72, dressing style belongs to visual modality appearance feature, sub-type". Similarly, the context descriptions of all directly connected nodes are constructed.

[0054] The edge weight is corrected based on the shortest path length between the feature node and the directly connected node. The shortest path length is always 1, so the correction coefficient is 1.0, and the corrected edge weight is the same as the original weight. In a complex graph, if indirect adjacent nodes are considered, the correction coefficient decreases accordingly with the increase of the shortest path length, for example, when the path length is 2, the correction coefficient can be set to 0.8.

[0055] The context descriptions of the neighboring nodes are sorted according to the revised edge weights to generate complete context information. For the "age distribution" node, the context descriptions of the 6 neighboring nodes are sorted in descending order of weights: first is "age distribution influences shopping preference, correlation strength 0.83, shopping preference belongs to behavioral modality consumption characteristics, continuous type", second is "age distribution influences emotional fluctuation, correlation strength 0.79, emotional fluctuation belongs to audio modality emotional characteristics, continuous type", and so on. The sorted descriptions are concatenated to form complete context information.

[0056] The context information is semantically encoded, and a pre-trained text encoding model is used to convert the text information into a high-dimensional vector representation. In a specific implementation, a deep learning text encoder is used to encode the context information into a 512-dimensional semantic vector. For example, the context information of the "age distribution" node is encoded into a 512-dimensional vector, representing the semantic features of the feature node and its neighborhood structure. A corresponding semantic encoding vector is generated for each feature node.

[0057] The semantic encoding vectors of all feature nodes are fused to generate a scene semantic encoding. The fusion method uses weighted averaging, and the weight is proportional to the topological importance of the node. For example, the semantic encoding vector of the "age distribution" node has a weight of 4.0 / total importance, the semantic encoding vector of the "dressing style" node has a weight of 0.684 / total importance, and so on. After weighted averaging, a 512-dimensional scene semantic encoding vector is obtained, which comprehensively represents the multi-dimensional feature information of the crowd in the elevator scene and the correlation structure between features.

[0058] By way of example, the scene semantic encoding can be used to match the most suitable advertising content for the current crowd characteristics. For example, when the scene semantic encoding shows that the main crowd in the elevator is young white-collar workers, and the feature nodes "age distribution", "dressing style" and "conversation topic" have high weights, office supplies, high-end catering or professional training advertisements can be recommended; when the scene semantic encoding shows that the main crowd is a family shopping group, and the feature node "age distribution" has a bimodal structure (adults and children) and the feature node "shopping preference" has a high weight, home products, children's education or leisure entertainment advertisements can be recommended. Through this graph-based scene semantic understanding method, accurate placement of elevator advertisements is achieved, significantly improving the conversion effect of advertisements.

[0059] In an optional implementation, the scene semantic encoding and the audience semantic encoding of the advertising materials are mapped to a two-dimensional coordinate system to construct a semantic matching graph, and the optimal matching path is extracted, and the advertising materials along the optimal matching path are selected to construct a candidate set, which includes: Extracting the audience semantic encoding of all advertising materials in the local advertising content library; The scene semantic code is taken as a query vector, and an attention score between each audience semantic code is calculated. The audience semantic codes are weighted and summed according to the attention score to obtain a semantic space mapping reference vector. For each advertising material, the projection length of the corresponding audience semantic code in the direction of the semantic space mapping reference vector and the deviation distance perpendicular to the direction of the semantic space mapping reference vector are calculated, which correspond to determining the first dimension coordinate and the second dimension coordinate, respectively, to obtain a node position. Based on the node position, the Euclidean distance between the nodes is calculated, and when the Euclidean distance is less than a preset distance threshold, a directed edge is established to form a semantic matching graph. The position of the query vector in the semantic matching graph is taken as a starting position, and the node closest to the starting position is selected as a path starting point. From the path starting point, adjacent nodes are traversed, the difference between the first dimension coordinates of the adjacent nodes and the path starting point is calculated to determine the forward direction score, the adjacent node with the maximum forward direction score is selected to join the path and serve as a new starting point, and the process is repeated until a preset path length is reached to obtain an optimal matching path. The advertising materials corresponding to the nodes on the optimal matching path are extracted, arranged in order, and a candidate set is constructed.

[0060] In a specific embodiment, a large number of advertising materials are stored in a local advertising content library, and each advertising material is associated with an audience semantic code. The audience semantic code is obtained by vectorizing the target audience characteristics of the advertising material, and the coding dimension is 512 dimensions. For example, the audience semantic code of a certain sports shoe advertisement reflects multi-dimensional characteristic information such as age interval, sports preference, and consumption ability. The audience semantic codes of all advertising materials are extracted from the advertising content library. Assuming that the current library contains 10,000 advertising materials, 10,000 audience semantic code vectors are extracted, each of which is a 512-dimensional floating-point number array.

[0061] The scene semantic code is also a 512-dimensional vector, representing the characteristic information of the current advertising delivery scene. The scene semantic code is taken as a query vector, and attention calculation is performed with each audience semantic code. The specific calculation process is as follows: the query vector and a certain audience semantic code are multiplied element by element, and the sum of the 512 numerical values of the product is calculated to obtain an original correlation value. The operation is performed on 10,000 audience semantic codes to obtain 10,000 original correlation values. The 10,000 original correlation values are input into an exponential function for normalization processing, so that the sum of all values is equal to 1. At this time, each value is the attention score of the corresponding audience semantic code. For example, the attention score of advertising material A is 0.0023, and the attention score of advertising material B is 0.0015.

[0062] Each audience semantic encoding is multiplied by its corresponding attention score to obtain a weighted vector. The 10000 weighted vectors are numerically accumulated in the corresponding dimensions to obtain a new 512-dimensional vector, which is the semantic space mapping reference vector. The reference vector comprehensively reflects the overall mapping trend of all audience semantic encodings in the current scene.

[0063] For the audience semantic encoding of each advertisement material, the projection length in the direction of the semantic space mapping reference vector is calculated. Specifically, the audience semantic encoding is multiplied by the semantic space mapping reference vector element by element, the sum of the 512 product values is calculated, and then divided by the modulus length of the semantic space mapping reference vector to obtain the projection length value. For example, the audience semantic encoding projection length of advertisement material A is 8.7, and the projection length of advertisement material B is 6.2. The projection length reflects the matching degree of the audience semantic encoding and the scene semantics, and the value serves as the first-dimensional coordinate.

[0064] The deviation distance perpendicular to the direction of the semantic space mapping reference vector is calculated. The audience semantic encoding is subtracted from its projection vector on the reference vector to obtain a residual vector. The projection vector is obtained by multiplying the unitized reference vector by the projection length. The Euclidean norm of the residual vector is calculated, that is, the 512 elements of the residual vector are squared, summed, and then the square root is taken to obtain the deviation distance value. For example, the deviation distance of advertisement material A is 2.3, and the deviation distance of advertisement material B is 3.1. The deviation distance serves as the second-dimensional coordinate. Through the first-dimensional coordinate and the second-dimensional coordinate, each advertisement material determines a unique node position in a two-dimensional plane.

[0065] Based on the node positions, a semantic matching graph is constructed. The Euclidean distance between any two nodes is calculated. The square of the difference between the first-dimensional coordinates and the square of the difference between the second-dimensional coordinates are added and then the square root is taken to obtain the Euclidean distance. The preset distance threshold is set to 1.5 units of length. When the Euclidean distance between two nodes is less than 1.5, a directed edge is established between the two nodes, indicating that they are adjacent in the semantic space. For example, node A has coordinates (8.7, 2.3) and node C has coordinates (9.1, 2.8), and the Euclidean distance between them is calculated as 0.64, which is less than the threshold 1.5, so a directed edge is established between node A and node C. This judgment is performed for all node pairs to finally form a semantic matching graph containing 10000 nodes and several edges.

[0066] The query vector, i.e., the scene semantic code, is mapped to a two-dimensional coordinate system. The projection length of the query vector onto the reference vector in the semantic space is calculated in the same way as the audience semantic code; let's assume the projection length is 9.5. The deviation distance of the query vector is also calculated using the Euclidean norm of the residual vector; let's assume the deviation distance is 1.8. The position of the query vector in the two-dimensional coordinate system is (9.5, 1.8), which is taken as the starting position. All nodes in the semantic matching graph are traversed, and the Euclidean distance between each node and the starting position is calculated. The node with the smallest distance is selected as the path starting point. Assuming node D has coordinates (9.3, 1.6) and a distance of 0.28 from the starting position, which is the minimum among all nodes, node D becomes the path starting point.

[0067] Starting from the path's starting node D, find all its directly connected neighboring nodes. Assume node D has three neighboring nodes E, F, and G, with coordinates (9.8, 1.9), (8.9, 1.4), and (9.5, 2.3) respectively. Calculate the difference between the first-dimensional coordinates of each neighboring node and the first-dimensional coordinates of the path's starting node. The difference for node E is 0.5, for node F it is -0.4, and for node G it is 0.2. A larger difference indicates greater progress in the main matching direction; this difference is the forward direction score. Select node E with the largest forward direction score and add it to the path; node E becomes the new starting point.

[0068] Starting from node E, repeatedly search for its neighboring nodes and calculate the score for the forward direction. Assume that among the neighboring nodes of node E, node H has a first-dimensional coordinate of 10.2, a difference of 0.4, which is the maximum score; therefore, node H is added to the path. Set the preset path length to 50, and continue this traversal process, adding the node with the maximum forward direction score to the path each time, until the path contains 50 nodes. The final node sequence consists of 50 nodes, including nodes D, E, and H, forming the optimal matching path.

[0069] The ad creatives corresponding to the 50 nodes on the optimal matching path are extracted and arranged according to the order in which the nodes are added to the path, forming a candidate set containing 50 ad creatives. The order of the ad creatives in this candidate set reflects the progressive relationship from the best match to the second best match with the scenario, providing an orderly candidate solution for subsequent ad placement decisions.

[0070] In one optional implementation, the process of extracting the evolution trajectory of the scene feature map, generating a scene evolution trend, calculating the fit score of the advertising materials in the candidate set, and generating a playback sequence according to the fit score includes: Extract the historical snapshot sequence of the scene feature map within a preset time window. For each historical snapshot, extract the node attribute features and edge weights. Calculate the node attribute feature difference vector and edge weight change rate between adjacent historical snapshots. Concatenate the node attribute feature difference vector and edge weight change rate to form snapshot evolution features. Arrange them in chronological order to form an evolution trajectory. The evolution trajectory is decomposed into a time series to obtain trend components and periodic components. The predicted difference vector is determined based on the trend components, and the predicted rate of change is determined based on the periodic components. Iterate through all ad creatives in the candidate set and extract audience semantic codes and historical delivery data; The semantic matching degree is determined by calculating the cosine similarity between the audience semantic code and the predicted difference vector, and the correlation coefficient is determined by calculating the Pearson correlation coefficient between the historical delivery data and the predicted rate of change. The adaptation score is obtained by weighted summing of the semantic matching degree and the correlation coefficient. The ad creatives are sorted in descending order according to their suitability scores to generate a playback sequence.

[0071] In one specific implementation, when extracting the historical snapshot sequence of scene feature maps within a preset time window, the time window is set to the past 7 days, and feature map snapshots are collected at 24 time points each day, accumulating to 168 historical snapshots. Taking the elevator multimodal crowd feature perception scenario as an example, these snapshots record the dynamic characteristics of crowd features changing over time. For each historical snapshot, node attribute features and edge weight data are extracted. Node attribute features include specific values ​​for 19 feature dimensions such as age distribution, clothing style, occupation type, and emotional state. For example, in a snapshot at a certain time point, the attribute feature value of the "age distribution" node may be [0.25, 0.45, 0.2, 0.1], representing the proportions of children, youth, middle-aged, and elderly, respectively; the attribute feature value of the "clothing style" node may be [0.6, 0.3, 0.1], representing the proportions of casual, formal, and sporty, respectively. The edge weight data records the strength of the association between features. For example, the edge weight between "age distribution" and "dress style" is 0.72, indicating that the two have a strong association.

[0072] When calculating the node attribute feature difference vector between adjacent historical snapshots, an element-by-element difference calculation method is adopted. Taking the "age distribution" node as an example, if the attribute feature value at time t is [0.25, 0.45, 0.2, 0.1], and the attribute feature value at time t+1 is [0.2, 0.5, 0.2, 0.1], then the difference vector is [-0.05, 0.05, 0, 0], indicating that the proportion of young people increases by 5% and the proportion of children decreases by 5%, and the other age groups remain unchanged. At the same time, the change rate of the edge weight is calculated, such as the edge weight between "age distribution" and "dressing style" changes from 0.72 to 0.75, with a change rate of (0.75-0.72) / 0.72≈0.042, indicating that the association strength is improved by 4.2%. The same calculation is performed on all nodes and edges to obtain a complete set of difference vectors and change rates.

[0073] The node attribute feature difference vector and the edge weight change rate are spliced to form the snapshot evolution feature. In this embodiment, the scene feature map contains 19 nodes and 32 edges, so the evolution feature of each snapshot has 19 node attribute difference vectors (the attribute vector dimension of each node depends on the specific feature, and the total dimension is 76) and 32 edge weight change rates, totaling 108-dimensional feature vectors. For example, in the evolution feature from time t to time t+1, it may contain the "age distribution" node 4-dimensional difference vector [-0.05, 0.05, 0, 0], the "dressing style" node 3-dimensional difference vector [0.03, -0.05, 0.02], and the "age distribution-dressing style" edge weight change rate 0.042, etc. The evolution features of all snapshots are arranged in chronological order to form an evolution trajectory matrix with a length of 167 and a dimension of 108.

[0074] When performing time series decomposition on the evolution trajectory to obtain the trend component and the periodic component, the seasonal trend decomposition method is adopted. For each dimension of the 108-dimensional feature, the corresponding time series is extracted and decomposed. Taking the "age distribution" node as an example, the value at 167 time points forms a time series. After decomposition, the trend component, the periodic component and the residual component are obtained. The trend component reflects the long-term change trend, such as the continuous rise in the proportion of young people from 8am to 9am on weekdays; the periodic component reflects the regular change, such as the periodic pattern of the fluctuation in the proportion of young people around 12pm. The same decomposition operation is performed on all 108-dimensional features to obtain complete trend component matrix and periodic component matrix.

[0075] When determining the prediction difference vector based on the trend component, the time series prediction model is used to extrapolate the trend component. Specifically, an autoregressive model is used to predict the trend of each dimension of the trend component for the next 24 hours. For example, for the youth proportion difference trend component of the "age distribution" node, it is predicted that the youth proportion will increase by an average of 0.03 in the next 24 hours. The trend components of all node attribute features are predicted, and the prediction difference vector is formed by merging, which has the same dimension as the node attribute feature difference vector, i.e., 76 dimensions. This vector indicates the trend of changes in each population feature in the next 24 hours.

[0076] When determining the prediction change rate based on the periodic component, the self-similarity characteristic of the periodic component is used. The periodic component of the historical periodic time closest to the target time point is extracted as the prediction change rate. For example, to predict the periodic component at 9 am on Monday, the periodic component data at 9 am on the previous Monday is referred to. The periodic components of all edge weight change rates are predicted to form a 32-dimensional prediction change rate vector. This vector reflects the periodic characteristics of the edge weight changes at a specific time point in the future.

[0077] All advertising materials in the candidate set are traversed to extract the audience semantic encoding and historical delivery data. In the elevator advertising delivery system, the advertising material library contains 500 different types of advertising content, and each advertisement is labeled with target audience feature encoding. The audience semantic encoding is a 76-dimensional vector that describes the population characteristics suitable for the advertisement. For example, the audience semantic encoding of a coffee advertisement is [0.1, 0.5, 0.3, 0.1] in the age distribution dimension, indicating that it mainly targets young and middle-aged groups; and [0.4, 0.3, 0.2, 0.1] in the occupation type dimension, indicating that it mainly targets white-collar workers and professionals. The historical delivery data records the number of plays and interaction rates of the advertisement in different time points and different elevator scenarios, forming a 32-dimensional time series data corresponding to the dimension of the edge weight change rate.

[0078] The cosine similarity between the audience semantic encoding and the prediction difference vector is calculated to determine the semantic matching degree. The cosine similarity calculates the closeness of the directions of two vectors, with a value range of -1 to 1, and the closer to 1 indicates a higher matching degree. Taking the coffee advertisement as an example, the cosine similarity between its audience semantic encoding and the prediction difference vector in the age distribution dimension is 0.87, indicating that the target audience of the advertisement is highly matched with the predicted population feature change trend. The semantic matching degree is calculated for all 500 advertising materials.

[0079] The Pearson correlation coefficient between the historical delivery data and the predicted change rate determines the correlation coefficient. The Pearson correlation coefficient measures the degree of linear correlation between two variables, with a value ranging from -1 to 1, and the closer to 1 indicates a stronger positive correlation. By analyzing the relationship between the historical interaction rate of the advertisement and the change rate of the edge weight, it can be found that a specific advertisement performs better when the graph structure changes. For example, the Pearson correlation coefficient between the historical delivery data and the predicted change rate of a certain children's education advertisement is 0.73, indicating that the advertisement may achieve good results under the current predicted graph evolution state. The correlation coefficient is calculated for all advertisement materials respectively.

[0080] The semantic matching degree and the correlation coefficient are weighted and summed to obtain the adaptation score. The weight of the semantic matching degree is set to 0.6, and the weight of the correlation coefficient is set to 0.4, reflecting the relative importance of semantic matching in advertisement delivery decision-making. Taking a coffee advertisement as an example, the semantic matching degree is 0.87, the correlation coefficient is 0.65, and the adaptation score is calculated as 0.87x0.6+0.65x0.4=0.782. The adaptation score is calculated for all 500 advertisement materials.

[0081] The advertisement materials are ranked in descending order according to the adaptation score to generate a play sequence. In this embodiment, the top 10 advertisements with the highest adaptation score are selected into the play sequence and played in order of score. For example, the ranking result shows that the adaptation score of the coffee advertisement is 0.782, ranking third, the adaptation score of the professional training advertisement is 0.835, ranking first, and the adaptation score of the smart watch advertisement is 0.804, ranking second. According to this ranking, in the next play period, the elevator advertisement screen will play the professional training advertisement, the smart watch advertisement, the coffee advertisement, etc. in turn.

[0082] For example, in a high-rise elevator, more advertisement content can be played due to the longer ride time; in a low-rise elevator, only 1-2 advertisements with the highest adaptation score are played due to the short ride time. In addition, the weight parameters in the adaptation score calculation can be adjusted in real time according to the play statistical feedback data to further optimize the advertisement delivery effect. Through this advertisement matching mechanism based on scene feature graph evolution prediction, accurate delivery of elevator advertisements can be achieved, significantly improving user experience and advertisement conversion rate.

[0083] In an alternative embodiment, the evolution trajectory is decomposed into a trend component and a periodic component by time series decomposition, the predicted difference vector is determined based on the trend component, and the predicted change rate is determined based on the periodic component, including: The evolution trajectory is decomposed into a trend component and a periodic component by time series decomposition; The least square fitting is performed on the trend component to obtain a fitting straight line, a transition vector set between adjacent snapshot evolution characteristics in the evolution trajectory is constructed, transition mode clusters are determined by clustering the transition vector set, a centroid vector with the highest similarity to the slope vector of the fitting straight line is selected, a dominant transition vector is determined, and a prediction difference vector is determined in combination with a preset time step; The autocorrelation analysis is performed on the periodic component, a first peak position in the autocorrelation coefficient sequence is searched to determine a period length, an edge weight change rate waveform within the period length is extracted, a corresponding amplitude change gradient is calculated, an amplitude decay coefficient is determined based on the amplitude change gradient, the edge weight change rate waveform is shifted backward by a product of the period length and a preset multiple, and the prediction change rate is determined in combination with the correction of the amplitude decay coefficient.

[0084] In a specific embodiment, time series decomposition processing is performed on the extracted evolution trajectory data, and the trajectory data is separated into two independent parts, a trend component and a periodic component. When specifically implemented, a seasonal decomposition method is used to decompose the feature value at each time point in the evolution trajectory into a long-term trend item, a periodic fluctuation item, and a random noise item. The original data is smoothed by a moving average technique, and the average value within a specified window length is calculated as an estimate of the trend component. The window length is set to an integer multiple of the period length, for example, when the observed data contains 30 time snapshots, the window length is set to 6, and the smoothed trend sequence is calculated by sliding window. The original evolution trajectory data is subtracted from the extracted trend component to obtain a detrended sequence, which mainly contains periodic fluctuations and random noise. The detrended sequence is segmented according to the period length, and the average value at each period position is calculated to obtain a standardized periodic component template.

[0085] The least square fitting is performed on the extracted trend component data, and the best fitting straight line is determined by calculating the parameters of the straight line that minimizes the sum of squared errors. Assuming that the trend component contains 25 data points, the time index is from 1 to 25, and the corresponding feature value gradually increases from the initial value 100 to 150. The average value of the time index and the average value of the feature value are calculated, and the slope of the straight line is obtained by dividing the covariance of the time index and the feature value by the variance of the time index. The slope reflects the overall change rate of the trend component. The intercept of the straight line is obtained by subtracting the product of the average feature value and the average time index from the average feature value, and the construction of the fitting straight line is completed.

[0086] A set of transition vectors between adjacent snapshots in the evolution trajectory is constructed. For each pair of consecutive time points, the difference vector is calculated as the feature vector at the later time point minus the feature vector at the earlier time point. Assuming each snapshot contains a 50-dimensional feature vector, each transition vector is also a 50-dimensional vector, representing the direction and magnitude of movement in the feature space. All transition vectors corresponding to adjacent snapshots in the entire evolution trajectory are collected to form a set containing 24 transition vectors.

[0087] A clustering analysis is performed on the set of transition vectors to identify the main transition patterns. The K-means clustering algorithm is adopted, with a preset number of clusters of 5 and random initialization of 5 cluster centers. The Euclidean distance of each transition vector to each cluster center is calculated, and the transition vector is assigned to the nearest cluster center. The distance calculation is realized by summing the square of the difference in each dimension and then taking the square root. After the initial assignment of all transition vectors is completed, the average value of all vectors in each dimension within each cluster is recalculated to obtain the updated cluster center. The assignment and update steps are repeated until the cluster center position changes by less than a set threshold of 0.001 or the number of iterations reaches 100. Finally, 5 transition pattern clusters are obtained, each containing several similar transition vectors.

[0088] The centroid vector of each transition pattern cluster is calculated, which is the arithmetic mean of all transition vectors in the cluster in each dimension. For example, a cluster contains 8 transition vectors with values of 2.1, 2.3, 1.9, 2.0, 2.2, 2.4, 1.8, and 2.1 in the first dimension, the centroid value of this dimension is the sum of all values divided by 8, which is 2.1. The same calculation is performed for the 50 dimensions to obtain a 50-dimensional centroid vector for the cluster.

[0089] The slope vector is extracted from the fitted straight line, which has a component of 1 in the time dimension and a component of the slope value 2.0 in the feature dimension. The similarity between the slope vector and the centroid vectors of each transition pattern cluster is calculated. The similarity is measured by the cosine similarity, which calculates the dot product of the two vectors divided by the product of the vector magnitudes. The dot product is obtained by multiplying the corresponding dimension elements and then summing them up, and the vector magnitude is calculated by the square root of the sum of the squares of the dimension values. The similarity values of the 5 centroid vectors and the slope vector are compared, and the centroid vector with the highest similarity is selected as the dominant transition vector. Assuming that the centroid vector of the 3rd cluster has a similarity of 0.92, which is higher than that of the other clusters, then this centroid vector is determined as the dominant transition vector.

[0090] The predicted difference vector is determined in combination with the preset time step parameter. The time step is set to 3, indicating a prediction of 3 time units into the future. The dimension values of the dominant transition vector are multiplied by the time step of 3 to obtain the predicted difference vector. For example, if the dimension value of the dominant transition vector is 1.5, then the corresponding predicted difference vector has a value of 4.5 in that dimension.

[0091] Autocorrelation analysis is performed on the extracted periodic component data, which calculates the correlation degree of the periodic component sequence with itself at different lag times. The lag time starts from 1 and increases step by step. For each lag value, the correlation coefficient is calculated between the periodic component sequence and the sequence after shifting. The correlation coefficient is obtained by the average value of the product of the corresponding elements of the two sequences minus the product of the respective averages, and then divided by the product of the standard deviations of the two sequences. The autocorrelation coefficients of lags 1 to 20 are calculated, and the autocorrelation coefficient sequence is 0.85, 0.62, 0.31, -0.05, -0.28, -0.40, -0.35, -0.18, 0.10, 0.38, 0.65, 0.82, 0.88, 0.78, 0.55, and so on.

[0092] The first peak position in the autocorrelation coefficient sequence is searched to determine the period length. Starting from the second position, it is judged whether the coefficient value at the current position is greater than the coefficient values at the previous and next positions. In the above example, the coefficient value 0.88 at the 13th position is greater than 0.82 at the 12th position and 0.78 at the 14th position, constituting a local peak, so the period length is determined to be 13 time units.

[0093] The edge weight change rate waveform data within a complete period length is extracted. A continuous data segment of length 13 is cut from the periodic component, and the edge weight relative change rate value at each time point is recorded, such as 0.05, 0.12, 0.18, 0.22, 0.20, 0.15, 0.08, -0.02, -0.10, -0.15, -0.12, -0.06, 0.01, and so on.

[0094] The amplitude change gradient of the waveform is calculated, which is realized by comparing the amplitude difference of the corresponding positions of adjacent periods. The amplitudes at the same phase position of the previous period and the current period are extracted, and the difference between the current period amplitude and the previous period amplitude is calculated. Assuming that the peak amplitude of the previous period is 0.25 and the peak amplitude of the current period is 0.22, the amplitude decreases by 0.03. The amplitude difference is divided by the amplitude of the previous period to obtain the decay ratio 0.12.

[0095] The amplitude decay coefficient is determined based on the calculated amplitude decay ratio, which is equal to 1 minus the decay ratio, obtaining 0.88. The coefficient is used to correct the amplitude prediction of future periods.

[0096] The extracted edge weight change rate waveform is shifted backward by a certain distance to generate a predicted change rate. The shift distance is equal to the period length multiplied by a preset multiple, and when the preset multiple is 2, the shift distance is 26 time units. The waveform data is moved 26 positions backward on the time axis, and all the amplitudes in the waveform are multiplied by the preset multiple of the amplitude attenuation coefficient. The square of the attenuation coefficient 0.88 is 0.7744, and the amplitude of each point in the waveform is multiplied by the correction coefficient to obtain the amplitude-corrected predicted change rate waveform, which completes the prediction of future periodic changes.

[0097] In an optional embodiment, the advertisement intelligent delivery further includes interactive data feedback. In the process of executing the advertisement delivery, the advertisements are displayed according to the sequence of the descending fitting scores, and the delivery control module displays the top-ranked advertisement materials in sequence through the intelligent advertisement screen in the elevator. Taking an elevator in a certain commercial building as an example, in the morning peak period (8:00-9:30), the top five advertisement materials in the playing sequence generated according to the foregoing method are: professional training advertisement (fitting score 0.835), smart watch advertisement (fitting score 0.804), coffee advertisement (fitting score 0.782), gym promotion advertisement (fitting score 0.756), and office software advertisement (fitting score 0.739). The delivery control module determines that 3 advertisements can be played in a complete elevator running cycle (from the bottom to the top) according to the elevator running time and the average number of passengers, and each advertisement is played for 15 seconds, so the top three advertisement materials are mainly delivered in this period.

[0098] In the process of advertisement delivery, collecting user interaction data is a key link to realize closed-loop feedback. The elevator advertisement screen is equipped with a multi-modal interaction data collection device, including visual sensors, audio sensors, infrared sensors, and pressure sensors, etc. The visual sensor is used to capture the user's gaze behavior, expression changes, and body movements; the audio sensor is used to collect environmental sounds and voice reactions; the infrared sensor is used to detect the proximity and gesture operation of the user to the screen; and the pressure sensor is used to record the change of the elevator load, so as to infer the number of passengers. The multi-modal interaction data collection network composed of these sensors can monitor the user's reaction to the advertisement content in all directions.

[0099] User interaction data types include both explicit and implicit interactions. Explicit interaction data refers to user-initiated interactions with the ad content, such as touching the screen, scanning the ad QR code, or performing gestures. In this example, the smartwatch ad designed an interactive interface that allowed users to flip through product details using gestures. Statistics showed that 32% of users performed such explicit interactions. Implicit interaction data refers to user's unconscious expressions of attention and reaction to the ad, such as gaze duration, facial expression changes, and stopping behavior. Data showed that the coffee ad averaged 5.7 seconds of effective gaze duration, and 67% of users showed positive facial reactions (smiling or surprised).

[0100] The collected user interaction data is pre-processed and feature extracted to form a structured interaction feature vector. Visual data processing includes facial expression recognition, gaze point tracking, and body movement analysis, extracting user emotional tendencies, attention distribution, and interaction willingness features; audio data processing includes environmental volume analysis, speech emotion recognition, and keyword extraction, obtaining user sound feedback features; infrared and pressure data processing mainly extracts user quantity, distance change, and dwell time spatial behavior features. Taking the coffee ad as an example, the interaction feature vector contains average gaze duration (5.7 seconds), positive expression proportion (67%), active interaction rate (15%), and dwell time increase proportion (23%) multi-dimensional data.

[0101] User interaction data is associated with corresponding ad materials and scene features. Each piece of user interaction data is labeled with the ad ID that triggered the interaction, the timestamp of the interaction, the elevator location information, and the scene feature map state at the time. This association allows the system to analyze the actual effect of a specific ad in a specific scene. For example, professional training ads in a scene containing more young white-collar workers (age distribution node feature value [0.05, 0.65, 0.25, 0.05]) have an average gaze duration of 6.3 seconds, while in a scene containing more middle-aged groups, the gaze duration drops to 3.8 seconds. This difference reflects the influence of the matching degree of ad content and population characteristics on ad effectiveness.

[0102] Actual feedback indicators of advertising materials are calculated based on user interaction data. Core feedback indicators include attention indicators, sentiment indicators, and interaction indicators. Attention indicators measure the degree of user attention to the advertisement, including average gaze duration, effective gaze rate (gaze duration as a percentage of advertisement play duration), etc. Sentiment indicators measure the emotional response of users to the advertisement, including the proportion of positive expressions, emotional fluctuation amplitude, etc. Interaction indicators measure the depth of user interaction with the advertisement, including active interaction rate, interaction completion rate, etc. Analysis of the feedback indicators of five advertisements shows that the attention indicators of the professional training advertisement are 0.42 (effective gaze rate 42%), the sentiment indicators are 0.58 (58% of users show positive emotions), and the interaction indicators are 0.21 (21% of users have active interaction). The corresponding indicators of the coffee advertisement are 0.38, 0.67, and 0.15, respectively, reflecting a higher emotional resonance but weaker interaction.

[0103] The edge weights and node attributes of the scene feature graph are updated using actual feedback indicators. The update process is divided into two parts: edge weight update and node attribute adjustment. Edge weight update is based on the consistency analysis of advertisement feedback and feature association. When a specific advertisement obtains high feedback under a specific combination of population features, it indicates that there is a strong association between these features, and the edge weight between these feature nodes is increased accordingly; otherwise, the edge weight is decreased. For example, the smart watch advertisement obtains high feedback (comprehensive index 0.65) in the scene where "age distribution" is mainly young, "occupation type" is mainly white-collar, and "dressing style" is mainly formal. The "age distribution-occupation type" edge weight is increased from 0.65 to 0.68, and the "occupation type-dressing style" edge weight is increased from 0.58 to 0.62, strengthening the association between these features.

[0104] Node attribute adjustment is based on the re-evaluation of the importance of features by user feedback. When the prediction ability of a certain feature node for advertisement feedback is enhanced, the importance attribute of the node is increased; when the prediction ability is weakened, the importance is decreased. Taking the "emotion state" node as an example, analysis shows that this node has strong prediction power for the feedback of coffee advertisements and gym advertisements (correlation coefficient reaches 0.72), and its node importance is increased from 0.53 to 0.59. Through this dynamic adjustment mechanism, the scene feature graph can continuously optimize its structure and more accurately express the association between population features.

[0105] The specific algorithm for realizing the graph structure update adopts a progressive learning method. After collecting a batch of new interaction data (for example, the data volume of one day), the deviation of the data from the current graph prediction result is calculated, and the edge weight and node attribute are adjusted based on the deviation size and direction. The adjustment amplitude is controlled by the learning rate, which is initially set to 0.05, representing a maximum adjustment of 5% of the weight value each time. As the data accumulation increases, the learning rate gradually decreases to 0.01, ensuring the stability of the graph structure. In actual operation, after 30 days of feedback data accumulation and graph update, the average effective gaze rate of elevator advertising is increased from the initial 32% to 47%, and the active interaction rate is increased from 12% to 19%, significantly improving the advertising effect.

[0106] The updated scene feature graph participates in the next round of advertising placement decision-making, forming a closed-loop optimization mechanism. For example, after graph update, it is found that the edge weight between the "conversation topic" node and the "shopping preference" node has increased significantly (from 0.48 to 0.63), indicating that the correlation between the two has increased. In subsequent advertising matching, when it is detected that the conversation topic of the crowd in the elevator is around food and entertainment, the matching score of related shopping advertisements is increased, and restaurant discount and entertainment venue advertisements are preferentially pushed. Through this continuous data feedback and graph update mechanism, the advertising placement system can continuously adjust and optimize its decision-making model, improve the accuracy and effectiveness of advertising, and realize the collaborative optimization of multi-modal crowd feature perception and intelligent advertising placement in the elevator scene.

[0107] The elevator multi-modal crowd feature perception and intelligent advertising placement system of the embodiment of the application comprises: A first unit is configured to collect multi-modal perception data in the elevator through an edge computing terminal; A second unit is configured to decouple the multi-modal perception data to obtain a decoupled feature set, perform cross-modal semantic alignment on the decoupled feature set, establish a directed correlation relationship between different modal features, and construct a scene feature graph; A third unit is configured to calculate the topological importance of each node in the scene feature graph, filter feature nodes based on a preset importance threshold, extract context information of the feature nodes and perform semantic encoding, and generate a scene semantic code; A fourth unit is configured to map the scene semantic code and the audience semantic code of the advertising material to a two-dimensional coordinate system, construct a semantic matching graph, extract an optimal matching path, select advertising materials along the optimal matching path to construct a candidate set; A fifth unit is configured to extract an evolution trajectory of the scene feature graph, generate a scene evolution trend, calculate the matching score of the advertising materials in the candidate set, and generate a playback sequence in the order of the matching score; A sixth unit is configured to perform advertising placement according to the playback sequence, collect user interaction data during the placement process, and feed back the user interaction data to the scene feature graph for graph structure update.

[0108] In a third aspect, the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0109] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method described above.

[0110] The present application can be a method, apparatus, system, and / or computer program product. Computer program products can include computer-readable storage media having computer-readable program instructions loaded thereon for performing various aspects of the present application.

[0111] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An elevator multi-modal crowd feature perception and advertisement intelligent delivery method, characterized in that, include: Multimodal sensing data inside the elevator is collected through an edge computing terminal; Multimodal perception data is decoupled to obtain a decoupled feature set. Cross-modal semantic alignment is performed on the decoupled feature set to establish directed correlations between features of different modalities and construct a scene feature map. Calculate the topological importance of each node in the scene feature map, filter feature nodes based on a preset importance threshold, extract the context information of the feature nodes and perform semantic encoding to generate scene semantic encoding; The scene semantic encoding and the audience semantic encoding of the advertising material are mapped to a two-dimensional coordinate system to construct a semantic matching graph, extract the optimal matching path, and select advertising materials along the optimal matching path to construct a candidate set; Extract the evolution trajectory of scene feature maps, generate scene evolution trends, calculate the adaptation scores of advertising materials in the candidate set, and generate playback sequences according to the adaptation scores; Ads are delivered according to the playback sequence, and user interaction data is collected during the delivery process. The user interaction data is then fed back to the scene feature map for graph structure updates.

2. The method of claim 1, wherein, Feature decoupling is performed on multimodal perception data to obtain a decoupled feature set. Cross-modal semantic alignment is then performed on the decoupled feature set to establish directed correlations between features of different modalities. The construction of a scene feature map includes: Modal separation is performed on multimodal sensing data, initial feature vectors of each modality are extracted, decoupling transformation is performed on the initial feature vectors of each modality to generate mutually independent decoupled feature components, and the decoupled feature components are aggregated to form a decoupled feature set; Each decoupled feature component is projected onto a unified continuous semantic manifold space. In the continuous semantic manifold space, multiple semantic cluster centers are identified through adaptive density clustering to determine the reference anchor points for cross-modal alignment. The geodesic distance from each decoupled feature component to each reference anchor point along the manifold surface in the continuous semantic manifold space is calculated. Based on the geodesic distance, an affinity matrix for cross-modal semantic similarity is constructed. The affinity matrix is ​​subjected to Laplace spectral decomposition to obtain a feature subspace. In the feature subspace, a bidirectional semantic mapping relationship between each decoupled feature component is established by minimizing the reconstruction error, thus completing the cross-modal semantic alignment. Based on the bidirectional semantic mapping relationship, decoupled feature component pairs with semantic association are identified, causal structure learning is performed on the decoupled feature component pairs, and directed association relationships between different modalities are determined. Using each decoupled feature component in the decoupled feature set as a node and the directed association relationship as a directed edge, a scene feature graph is constructed.

3. The method of claim 2, wherein, Based on the bidirectional semantic mapping relationship, decoupled feature component pairs with semantic association are identified, and causal structure learning is performed on the decoupled feature component pairs to determine the directed association relationship between different modalities, including: Based on the bidirectional semantic mapping relationship, decoupled feature component pairs belonging to different modalities are selected to form a cross-modal feature component pair set; The evolution track data in a plurality of time windows is extracted for each group of decoupled feature component pairs in the cross-modal feature component pair set, the prediction contribution degree of the preceding evolution track of the first decoupled feature component to the subsequent evolution track of the second decoupled feature component and the prediction contribution degree of the preceding evolution track of the second decoupled feature component to the subsequent evolution track of the first decoupled feature component are respectively calculated, and any prediction contribution degree is marked as a causal candidate pair when the prediction contribution degree exceeds a preset contribution threshold; For each causal candidate pair, a perturbation amount is superimposed on the first decoupled feature component, a calculated response value is obtained by mapping to the semantic space of the second decoupled feature component based on the bidirectional semantic mapping relationship, the deviation amplitude of the calculated response value from the actual observation value is calculated to determine the strength of the positive causal effect, a perturbation amount is superimposed on the second decoupled feature component and mapped to the semantic space of the first decoupled feature component, the deviation amplitude is calculated to determine the strength of the reverse causal effect, and a directed causal edge from the first decoupled feature component to the second decoupled feature component is established when the strength of the positive causal effect is greater than the strength of the reverse causal effect, otherwise a directed causal edge from the second decoupled feature component to the first decoupled feature component is established; The directed causal edges meeting the preset strength requirement are screened to determine the directed association relationship between different modalities.

4. The method of claim 1, wherein, The topological importance of each node in the scene feature graph is calculated, the feature nodes are screened based on a preset importance threshold, the context information of the feature nodes is extracted and semantically encoded, and the scene semantic code is generated, including: The nodes in the scene feature graph are traversed, the node degree value is calculated for each node by counting the number of connection edges, the local topological importance is determined based on the ratio of the sum of the node degree values of all adjacent nodes of the node to the node degree value, and the global topological importance is determined based on the ratio of the number of nodes reachable from the node within a preset hop number to the total number of nodes; the node connection edge weight variance value is calculated to determine the neighborhood stability indicator, the local topological importance is taken as the node topological importance when the neighborhood stability indicator is less than or equal to a preset stability threshold, and the global topological importance is taken as the node topological importance when the neighborhood stability indicator is greater than the stability threshold; The nodes exceeding the preset importance threshold are screened according to the node topological importance to obtain the feature nodes; For the feature nodes, the direct adjacent nodes are extracted, the edge type identifier and edge weight between the feature nodes and the direct adjacent nodes are obtained, the attributes, edge type identifier and edge weight of the direct adjacent nodes are combined to construct the context description, the edge weight is corrected based on the shortest path length between the feature nodes and the direct adjacent nodes, the context description is sorted in accordance with the corrected edge weight to generate the context information in series, and the context information is semantically encoded to obtain the semantic encoding vector, and the semantic encoding vector is fused to generate the scene semantic code.

5. The method of claim 1, wherein, The scene semantic code and the audience semantic code of the advertisement material are mapped to a two-dimensional coordinate system to construct a semantic matching graph, and an optimal matching path is extracted, and the advertisement material is selected along the optimal matching path to construct a candidate set, including: The audience semantic codes of all advertisement materials in the local advertisement content library are extracted; The scene semantic encoding is taken as a query vector, attention scores between the query vector and audience semantic encodings are calculated, and a semantic space mapping reference vector is obtained by weighted summation of the audience semantic encodings according to the attention scores; For each advertisement material, a projection length of the corresponding audience semantic encoding in a direction of the semantic space mapping reference vector and a deviation distance of the corresponding audience semantic encoding perpendicular to the direction of the semantic space mapping reference vector are calculated, to determine a first dimension coordinate and a second dimension coordinate respectively, and a node position is obtained; An Euclidean distance between the node positions is calculated, and a directed edge is established when the Euclidean distance is less than a preset distance threshold, to form a semantic matching graph; A position of the query vector in the semantic matching graph is taken as a starting position, and a node closest to the starting position is selected as a path starting point; Adjacent nodes are traversed from the path starting point, a difference value between a first dimension coordinate of each adjacent node and the path starting point is calculated to determine a forward direction score, an adjacent node with the largest forward direction score is selected to join the path and serve as a new starting point, and the process is repeated until a preset path length is reached, to obtain an optimal matching path; Advertisement materials corresponding to the nodes on the optimal matching path are extracted, and a candidate set is constructed by arranging the advertisement materials in a joining order.

6. The method of claim 1, wherein, An evolution track of the scene feature graph is extracted, a scene evolution trend is generated, and an adaptation score of each advertisement material in the candidate set is calculated, to generate a play sequence by sorting the advertisement materials according to the adaptation scores, including: A historical snapshot sequence of the scene feature graph in a preset time window is extracted, node attribute features and edge weights are extracted for each historical snapshot, a node attribute feature difference vector between adjacent historical snapshots and an edge weight change rate are calculated, the node attribute feature difference vector and the edge weight change rate are spliced to form a snapshot evolution feature, and the snapshot evolution features are arranged in a time sequence to form an evolution track; Trend components and periodic components are obtained by time series decomposition of the evolution track, a predicted difference vector is determined based on the trend components, and a predicted change rate is determined based on the periodic components; All advertisement materials in the candidate set are traversed, and audience semantic encodings and historical delivery data are extracted; A semantic matching degree is determined by calculating a cosine similarity between the audience semantic encodings and the predicted difference vector, and a correlation coefficient is determined by calculating a Pearson correlation coefficient between the historical delivery data and the predicted change rate, and the adaptation score is obtained by weighted summation of the semantic matching degree and the correlation coefficient; The advertisement materials are arranged in a descending order according to the adaptation scores, to generate the play sequence.

7. The method of claim 6, wherein, The trend components and the periodic components are obtained by time series decomposition of the evolution track, the predicted difference vector is determined based on the trend components, and the predicted change rate is determined based on the periodic components, including: The trend components and the periodic components are extracted by time series decomposition of the evolution track; A fitting straight line is obtained by least square fitting of the trend components, a transition vector set between adjacent snapshot evolution features in the evolution track is constructed, transition mode clusters are determined by clustering of the transition vector set, centroid vectors of the transition mode clusters are calculated, a centroid vector with the highest similarity to a slope vector of the fitting straight line is selected, a dominant transition vector is determined, and the predicted difference vector is determined based on a preset time step. The autocorrelation analysis is performed on the periodic component, the first peak position in the autocorrelation coefficient sequence is searched to determine the period length, the edge weight change rate waveform in the period length is extracted, the corresponding amplitude change gradient is calculated, the amplitude attenuation coefficient is determined based on the amplitude change gradient, the edge weight change rate waveform is shifted backward by a product of the period length and a preset multiple, and the amplitude attenuation coefficient is corrected to determine the predicted change rate.

8. The elevator multi-modal crowd feature perception and advertisement intelligent delivery system for implementing the method of any one of the preceding claims 1-7, characterized in that, Comprise: A first unit for collecting multi-modal perception data in an elevator through an edge computing terminal; A second unit for feature decoupling of the multi-modal perception data to obtain a decoupled feature set, cross-modal semantic alignment of the decoupled feature set to establish a directed association relationship between different modal features, and construction of a scene feature graph; A third unit for calculating the topological importance of each node in the scene feature graph, filtering feature nodes based on a preset importance threshold, extracting context information of the feature nodes and performing semantic encoding to generate scene semantic encoding; A fourth unit for mapping the scene semantic encoding and audience semantic encoding of an advertisement material to a two-dimensional coordinate system, constructing a semantic matching graph, extracting an optimal matching path, and selecting advertisement materials along the optimal matching path to construct a candidate set; A fifth unit for extracting an evolution trajectory of the scene feature graph, generating a scene evolution trend, calculating an adaptation score of the advertisement materials in the candidate set, and generating a play sequence according to the adaptation score; A sixth unit for performing advertisement delivery according to the play sequence, collecting user interaction data during the delivery process, and feeding back the user interaction data to the scene feature graph for graph structure updating.

9. An electronic device, comprising: Comprise: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to call the instructions stored in the memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 7. The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images