Advertisement pushing method and system based on multi-modal data fusion

Abstract

The invention provides an advertisement pushing method and system based on multi-modal data fusion. The method comprises the following steps: determining a user staying event of each position according to intensity peak value distribution of a user foot heat radiation signal of each position in continuous N time windows; performing frequency domain decomposition on the user foot heat radiation signals at the plurality of positions so as to extract a people flow density component and a staying time length component which are synchronous with a timestamp of a user staying event; calculating a discrete difference degree between the people flow density component and the staying duration component by adopting a variable coefficient method; and according to the discrete difference degree, adjusting a content playing queue of an advertisement screen in the offline commercial place, so that the playing content is matched with the commodity category corresponding to the shelf vertex. According to the technical scheme provided by the invention, the user staying event can be intelligently identified, the content of the advertisement screen is dynamically matched, precise commodity marketing is realized, and a consumption scene is adapted.

Description

Technical Field

[0001] This application relates to the technical field of advertisement pushing, and particularly relates to an advertisement pushing method and system based on multi-modal data fusion. Background Art

[0002] In the intelligent retail scenario, the advertisement pushing in modern offline commercial places (such as shopping malls and supermarkets) aims to stimulate customers' consumption desire, improve the commodity conversion rate and brand awareness through accurate and real-time content display. Its core requirement lies in accurately identifying customers' interest points and instantaneously responding to the dynamic flow of people on the spot. The traditional "one-size-fits-all" or advertisement playing based on rough time period planning is inefficient and cannot capture customers' real staying behaviors and interest intensities in specific shelf areas. Therefore, there is an urgent need for a technology that can non-invasively and real-timely sense the staying states and density changes of customers in front of specific shelves, and accordingly dynamically optimize the advertisement content placement strategy to achieve a high degree of matching between the advertisement content and customers' instant interests and the on-site atmosphere.

[0003] Existing solutions adopt a passenger flow analysis system based on visual sensors. By deploying cameras in combination with image processing algorithms, it is inferred whether users stay and the goods they are interested in by analyzing face orientations, body postures or movement trajectories.

[0004] However, existing solutions are vulnerable to environmental light changes and occlusion interference, resulting in missed detection of short-term staying events; at the same time, the analysis relying on human body contours or facial features may cause privacy disputes. In addition, the calculation complexity of camera data is high, and it is difficult to real-timely extract high-frequency and fine-grained components of the dynamic flow of people, and the judgment accuracy of the staying intention is limited. Summary of the Invention

[0005] This application provides an advertisement pushing method and system based on multi-modal data fusion to solve the problem that in offline commercial scenarios with strong occlusion and high privacy requirements in the prior art, it is impossible to accurately and real-timely sense and quantify users' effective staying behaviors, which in turn leads to the deviation of the advertisement pushing strategy from the reality.

[0006] In a first aspect, this application provides an advertisement pushing method based on multi-modal data fusion, including: Synchronously acquiring the thermal radiation signals of users' feet at multiple positions through an infrared thermal imaging sensor array deployed at the vertices of shelves in an offline commercial place; Determining the user staying events at each position according to the intensity peak distribution of the thermal radiation signals of users' feet at each position within consecutive N time windows; Performing frequency-domain decomposition on the thermal radiation signals of users' feet at the multiple positions to extract the component of the flow density and the component of the staying duration synchronized with the time stamps of the user staying events; The coefficient of variation method is used to calculate the discrete difference degree between the pedestrian flow density component and the stay duration component; According to the discrete difference degree, the content playback queue of the advertising screen in the offline commercial venue is adjusted so that the played content matches the commodity category corresponding to the top of the shelf.

[0007] Optionally, the determining of the user stay event at each position according to the intensity peak distribution of the user foot thermal radiation signal within consecutive N time windows includes: Time slicing is respectively performed on the user foot thermal radiation signals at each position according to a fixed time window length to obtain a radiation intensity sequence for each time window corresponding to each position; For the radiation intensity sequence of each time window, adjacent sampling point difference calculation is performed, an intensity change rate sequence is generated according to the difference result, and candidate peak points with an intensity change rate exceeding a preset intensity change rate threshold are screened out from the sampling point sequence corresponding to the radiation intensity sequence; Hierarchical filtering is performed on the candidate peak points within each time window to obtain the final peak points; Based on the final peak points, metadata extraction operations are performed within the corresponding time windows to generate a metadata set including peak intensity values, peak timestamps, and peak durations; The metadata sets of all time windows are inserted into a preset sliding cache queue in chronological order. When the number of time windows in the sliding cache queue is equal to N, user stay events are generated based on all the metadata sets in the sliding cache queue.

[0008] Optionally, the generating of user stay events based on all the metadata sets in the sliding cache queue when the number of time windows in the sliding cache queue is equal to N includes: Based on the metadata sets of N time windows in the sliding cache queue, the spatial cumulative weight and the time domain discreteness are calculated; Based on all the peak timestamps in the sliding cache queue, peak clusters are identified; When the spatial cumulative weight value exceeds a preset weight threshold, the time domain discreteness is lower than a preset discreteness threshold, and the peak cluster satisfies the time density constraint, user stay events are generated.

[0009] Optionally, the hierarchical filtering of the candidate peak points within each time window to obtain the final peak points includes: Within each time window, the intensity differences between the candidate peak points and each sampling point within the sampling point range are calculated, and the sampling point range is the range composed of the first K sampling points and the last K sampling points of the candidate peak point; Based on the intensity difference value, retain the candidate peak points whose intensity values are greater than all sampling points within the sampling point range, and generate preliminary screening peak points; Detect the interval time between adjacent preliminary screening peak points, and merge adjacent preliminary screening peak points with an interval time less than a preset separation threshold to generate a candidate peak sequence; Perform stability verification on the candidate peak sequence to eliminate instantaneous fluctuation peak points with an intensity duration less than a preset duration threshold, and obtain the final peak points.

[0010] Optionally, the frequency domain decomposition of the user's foot thermal radiation signals at the multiple positions to extract the crowd density component and the stay duration component synchronized with the time stamp of the user's stay event includes: Perform frequency domain decomposition on the user's foot thermal radiation signals at the multiple positions to generate a steady-state amplitude-frequency component and a transient amplitude-frequency component for each position; Based on the time stamp of the user's stay event, perform spatial superposition on the steady-state amplitude-frequency components at multiple positions within the same time window to generate an aggregated component, and perform time accumulation on the transient amplitude-frequency components at a single position within the same time window to generate an integral component; According to the aggregated component, intercept the amplitude extreme value interval aligned with the time stamp of the user's stay event to obtain the crowd density component; According to the integral component, calculate the integral slope matching the time stamp of the user's stay event to obtain the stay duration component.

[0011] Optionally, the adjusting the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree includes: Generate an advertisement push priority index based on the discrete difference degree; According to the advertisement push priority index and the commodity category corresponding to the shelf vertex, adjust the content playback queue of the advertising screen in the offline commercial venue.

[0012] Optionally, the generating an advertisement push priority index based on the discrete difference degree includes: Generate a dynamic fusion weight coefficient based on the discrete difference degree; Combine with a preset scene three-dimensional model to determine the spatial topological association value between the spatial coordinates of the user's stay event and the shelf vertex; Perform a multiplication operation on the dynamic fusion weight coefficient and the spatial topological association value to obtain the advertisement push priority index.

[0013] In a second aspect, the present application provides an advertisement push system based on multi-modal data fusion, including: An acquisition module, configured to synchronously acquire user foot thermal radiation signals at multiple locations through an infrared thermal imaging sensor array deployed at the vertices of shelves in an offline commercial venue; A determination module, configured to determine user stay events at each location according to the intensity peak distribution of the user foot thermal radiation signals at each location within N consecutive time windows; An extraction module, configured to perform frequency-domain decomposition on the user foot thermal radiation signals at the multiple locations to extract a crowd density component and a stay duration component synchronized with the time stamps of the user stay events; A calculation module, configured to calculate the discrete difference degree between the crowd density component and the stay duration component by using the coefficient of variation method; A matching module, configured to adjust the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree, so that the played content matches the commodity category corresponding to the shelf vertex.

[0014] In a third aspect, the present application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an advertising push method based on multi-modal data fusion as described in any item of the first aspect.

[0015] In a fourth aspect, the present application provides a computer storage medium, storing a computer program, and when the computer program is executed by a computer, it implements an advertising push method based on multi-modal data fusion as described in any item of the first aspect.

[0016] In the present application, an advertising push method based on multi-modal data fusion is provided. The method includes: synchronously acquiring user foot thermal radiation signals at multiple locations through an infrared thermal imaging sensor array deployed at the vertices of shelves in an offline commercial venue; determining user stay events at each location according to the intensity peak distribution of the user foot thermal radiation signals at each location within N consecutive time windows; performing frequency-domain decomposition on the user foot thermal radiation signals at the multiple locations to extract a crowd density component and a stay duration component synchronized with the time stamps of the user stay events; calculating the discrete difference degree between the crowd density component and the stay duration component by using the coefficient of variation method; and adjusting the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree, so that the played content matches the commodity category corresponding to the shelf vertex.

[0017] This application non-invasively synchronously collects multi-location foot thermal radiation signals through a pre-deployed infrared thermal imaging sensor array, avoiding privacy disputes and increasing data coverage density; analyzes stay events within consecutive time windows based on the peak intensity distribution, enhancing the sensitivity of short-term behavior capture and anti-occlusion ability; extracts synchronized components of pedestrian flow density and stay duration through frequency domain decomposition to achieve fine-grained quantification of behavior characteristics; calculates the discrete difference degree using the coefficient of variation method to objectively characterize the correlation difference between pedestrian dynamics and stay intentions; finally, dynamically optimizes the advertising screen playback strategy based on the difference degree to improve the scenario matching degree between the content and the shelf products, forming a closed-loop process from perception, analysis to decision-making, capable of accurately and real-time perceiving and quantifying the effective stay behavior of users, thereby ensuring that the advertising push strategy is consistent with the actual demand and strengthening the precision marketing ability and scenario response efficiency.

[0018] Furthermore, slice the multi-location foot thermal radiation signals through a fixed time window to generate a radiation intensity sequence, and extract the intensity change rate sequence based on the difference calculation of adjacent sampling points to screen out candidate peak points; further optimize the accuracy of candidate peak points through hierarchical filtering, and extract a metadata set including peak intensity, timestamp, and duration; store the metadata of multiple time windows in a sliding cache queue. When the queue accumulates N full windows, dynamically determine the user stay event in combination with spatial cumulative weights, temporal dispersion, and the time density constraint of the peak cluster. Through the difference calculation and hierarchical filtering mechanism, effectively suppress environmental noise interference, improve the robustness and accuracy of peak detection; based on the sliding window and spatio-temporal joint criterion, achieve refined capture of short-term stay events and suppression of false touch phenomena; the lightweight calculation framework driven by metadata supports dynamic correlation analysis of multi-window data while ensuring real-time performance, enhancing the mapping reliability between stay event determination and user behavior intentions, and providing high-confidence data support for the subsequent accurate matching of advertising screen content.

[0019] These aspects or other aspects of this application will be more clearly understood in the following description of the embodiments. Brief Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of an advertising push method based on multi-modal data fusion provided by an embodiment of this application; Figure 2 It is a schematic structural diagram of an advertising push system based on multi-modal data fusion provided by an embodiment of this application; Figure 3 This is a schematic structural diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0022] In order to enable those skilled in the art of the present technology to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0023] In some processes described in the specification and claims of the present application and the above-mentioned accompanying drawings, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The operation numbers such as 11, 12, etc. are only used to distinguish different operations, and the numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are different types.

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0025] To solve the problems of missed detection of short-term stay events caused by environmental sensitivity and occlusion interference, privacy disputes caused by relying on human feature analysis, and insufficient real-time performance of fine-grained human flow dynamic feature extraction and limited accuracy of stay intention judgment under high computational complexity in the prior art, the embodiments of the present application provide an advertising push method based on multi-modal data fusion. This method adopts the following concept: Deploy an infrared thermal imaging sensor array at the apex of the shelf to synchronously capture the thermal radiation signals of multiple positions of the feet, avoiding environmental interference and the risk of human feature recognition; Based on the dynamic analysis of the intensity peak distribution within a time window, establish a short-term stay event detection model to enhance the sensitivity of behavior capture; Combine frequency domain decomposition technology to extract the components of the crowd density and stay duration synchronized with the event, and quantify the behavior features; Use the coefficient of variation method to construct a discrete difference index to objectively characterize the correlation difference between the human flow dynamics and the stay intention; Finally, dynamically optimize the advertising content playback strategy based on the difference degree, forming an intelligent adaptation framework from data perception, feature decoupling to decision-making closed-loop, and improving the real-time matching efficiency of commodity marketing and consumption scenarios.

[0026] Figure 1The flowchart of an advertising push method based on multi-modal data fusion provided by an embodiment of this application is as follows Figure 1 As shown, the method includes: S11. Synchronously obtain the thermal radiation signals of users' feet at multiple positions through an infrared thermal imaging sensor array deployed at the vertex of the shelf in an offline commercial venue.

[0027] Among them, the infrared thermal imaging sensor array refers to a network composed of multiple infrared thermal imaging sensors, which is deployed at the vertex of the shelf and generates thermal signals by detecting the thermal radiation energy of the human foot. The thermal radiation signal of the user's foot refers to the time-series data of the radiation energy corresponding to the surface temperature of the foot captured by the infrared sensor, which is used to characterize the user's position and behavior.

[0028] In an embodiment of this application, by deploying the infrared thermal imaging sensor array at the vertex position of the shelf and using its non-contact thermal radiation sensing ability, the thermal radiation signals of users' feet in multiple shelf areas are collected in real time. The sensor array synchronously obtains the signals at each position at a fixed sampling frequency and ensures the time consistency of multiple channels of signals through time alignment technology. The coverage area of each sensor corresponds to the user activity range below the vertex of the shelf, and the thermal radiation energy of the foot is characterized by the thermal imaging signal intensity value, forming multi-channel time-series data of the thermal radiation signal of the foot.

[0029] S12. Determine the user stay events at each position according to the intensity peak distribution of the thermal radiation signals of users' feet at each position within N consecutive time windows.

[0030] Among them, the intensity peak distribution refers to the statistical characteristics of the maximum signal intensity and its change trend within consecutive time windows, which is used to identify the user stay behavior. The user stay event at each position refers to determining the start and end times of the user's stay in a specific shelf area by analyzing the peak distribution of the thermal signal. N can refer to an integer greater than or equal to 2. Exemplarily, N is 3, 5, 10, etc.

[0031] In an embodiment of this application, the thermal radiation signal of each position can be divided into multiple time-series segments according to a fixed time window, and the intensity peak distribution of the signal within each window is calculated. Specifically, the maximum intensity value of each window is extracted through a sliding window algorithm, and combined with the peak change trend of adjacent windows, it is determined whether the user stays within the window. If the average peak intensity of N consecutive windows exceeds the preset threshold and the distribution is stable, the user stay event at this position is triggered, and the timestamp of the event is recorded.

[0032] S13. Perform frequency-domain decomposition on the thermal radiation signals of users' feet at multiple positions to extract the crowd density component and the stay duration component synchronized with the timestamps of the user stay events.

[0033] Among them, the timestamp of the user stay event refers to the specific time mark when the stay event occurs. The pedestrian flow density component refers to the integral of the low-frequency signal energy extracted through frequency-domain decomposition, reflecting the change trend of the number of users in the area. The stay duration component refers to the duration statistics of the high-frequency signal extracted through frequency-domain decomposition, characterizing the duration of a single stay.

[0034] In the embodiment of the present application, the fast Fourier transform is performed on the foot thermal radiation signals at multiple positions to decompose the time-domain signals into frequency-domain components. The low-frequency component and the high-frequency component synchronized with the timestamp of the user stay event are extracted through a filtering algorithm. Among them, the pedestrian flow density component is calculated through low-frequency energy integration, reflecting the dynamic change of the number of users in the area; the stay duration component quantifies the duration of a single user stay through the duration statistics of the high-frequency component.

[0035] S14. Calculate the discrete difference degree between the pedestrian flow density component and the stay duration component by using the coefficient of variation method.

[0036] Among them, the discrete difference degree refers to the quantification value of the difference in the coefficient of variation between the pedestrian flow density component and the stay duration component, which is used to evaluate the correlation of the dynamic characteristics of the two.

[0037] In the embodiment of the present application, for the pedestrian flow density component and the stay duration component within the same time window, calculate the mean value and the standard deviation of the two respectively, and quantify the difference in the degree of dispersion between the two by using the coefficient of variation method.

[0038] S15. According to the discrete difference degree, adjust the content playback queue of the advertising screen in the offline commercial venue so that the played content matches the commodity category corresponding to the top of the shelf.

[0039] Among them, the content playback queue of the advertising screen can be a priority list, which is used to reflect the order of the content to be played on the advertising screen and can be dynamically adjusted according to the real-time analysis result.

[0040] In the embodiment of the present application, when the discrete difference degree exceeds the preset discrete difference degree threshold (for example, 0.3), it is determined that the current pedestrian flow dynamics does not match the stay intention, and the advertising screen content update logic is triggered. Through the dynamic priority algorithm, the advertising content corresponding to the commodity category at the top of the shelf is inserted into the front of the playback queue, and its playback interval is shortened; if the discrete difference degree is lower than the preset discrete difference degree threshold, the order of the corresponding commodity in the original queue is maintained or the priority of the corresponding commodity is reduced to ensure the real-time matching of the advertising content with the commodities actually concerned by the users.

[0041] Here is a specific example: In the smart clothing area of ​​a large shopping mall, the infrared thermal imaging sensor array deployed at the top of the shelf captures the thermal radiation signal of the customer's feet in real time. When the customer stops in the summer light fabric display area, the sensor continuously collects signals in a 5-second time window and analyzes the intensity peak distribution. It detects that the peak mean value in three consecutive windows stably exceeds the threshold and fluctuates smoothly, and determines it as a valid user stay event and marks the timestamp. After the system decomposes the signal in the frequency domain, it extracts the low-frequency component, which shows that the density of people in the area is slowly rising, and the high-frequency component quantifies the average single stay time of about 20 seconds; at the same time, it is detected that the low-frequency component is sparse in the winter down jacket area, but the high-frequency component stays for more than 3 minutes. The discrete difference of the summer zone calculated by the coefficient of variation method is 0.7, which is higher than the threshold of 0.3, indicating that the customer flow gathers briefly but lacks in-depth interaction, which immediately triggers the hanging advertising screen to switch to a virtual fitting interactive interface and a cool fabric breathability demonstration video; while the difference in the winter zone is only 0.25, and the system maintains the original playback queue, looping to display the down filling process and cold resistance test content, accurately adapting to the customer behavior characteristics and consumption intentions of different clothing areas.

[0042] By executing S11~S15, the embodiment of the present application uses an infrared thermal imaging sensor array to non-invasively sense foot heat signals to avoid privacy risks; based on the intensity peak distribution analysis and frequency domain decomposition technology of the time window, high-sensitivity detection of short-term stay events and fine-grained quantification of dynamic characteristics of human flow are achieved; the coefficient of variation method is used to construct a discrete difference index to objectively characterize the correlation between user behavior intention and product attention; finally, by dynamically adjusting the advertising playback strategy, the matching accuracy between content and shelf products is improved, and the real-time marketing and scene adaptation capabilities are enhanced.

[0043] In a possible embodiment, S12, determining a user stay event at each location according to the intensity peak distribution of the user foot thermal radiation signal at each location in N consecutive time windows, includes: Step 121 , time slice the thermal radiation signals of the user's feet at each position according to a fixed time window length, and obtain a radiation intensity sequence of each time window corresponding to each position.

[0044] The radiation intensity sequence of each time window refers to a set of intensity values ​​of the foot thermal radiation signal arranged in time order within a fixed time period.

[0045] In the embodiment of the present application, the foot thermal radiation signal collected at each shelf position is time-series cut with a fixed time window length to generate multiple continuous time segments. For example, a 10-minute signal is cut with a 5-second window to obtain 120 time windows; the intensity value of the foot thermal radiation signal is extracted in each window according to the sampling frequency to form a radiation intensity sequence as the basic data for subsequent peak detection.

[0046] Step 122: For the radiation intensity sequence of each time window, perform differential calculation on adjacent sampling points, generate an intensity change rate sequence based on the differential result, and screen out candidate peak points with an intensity change rate exceeding a preset intensity change rate threshold from the sampling point sequence corresponding to the radiation intensity sequence.

[0047] Among them, the differential of adjacent sampling points refers to calculating the difference between the intensity values of two adjacent sampling points, which is used to characterize the signal change rate, and the sampling point is a time point. The intensity change rate sequence refers to the time series data of the instantaneous change rate of the signal intensity obtained through differential calculation. The candidate peak point with a preset intensity change rate threshold refers to the sampling point whose intensity change rate exceeds the preset critical value, which is marked as a potential peak.

[0048] In the embodiment of the present application, differential calculation of adjacent sampling points is performed on the radiation intensity sequence of each time window, and the intensity value of the latter sampling point is subtracted from the intensity value of the former sampling point to generate an intensity change rate sequence.

[0049] Step 123: Perform hierarchical filtering on the candidate peak points within each time window to obtain the final peak points.

[0050] Among them, hierarchical filtering refers to an algorithm that optimizes candidate peak points through multiple-level rules. The final peak point refers to the valid peak point confirmed after hierarchical filtering.

[0051] In the embodiment of the present application, a hierarchical filtering algorithm is used to optimize candidate peak points: if the time interval between multiple candidate peak points is less than the preset value, only the point with the maximum intensity is retained; candidate points with an intensity value lower than 1.5 times the global mean are excluded; check whether the signals before and after the peak point satisfy the "rising, stable to falling" pattern. Through the above steps, the final peak points are screened out to exclude false detections caused by noise interference.

[0052] Step 124: Based on the final peak points, perform metadata extraction operations within the corresponding time window to generate a metadata set containing peak intensity values, peak timestamps, and peak durations.

[0053] Among them, the metadata extraction operation refers to the process of extracting structured information such as intensity, timestamp, and duration from peak points. The peak intensity value refers to the maximum radiation intensity value corresponding to the peak point. The peak timestamp refers to the precise moment mark of the peak point within the time window. The metadata set of the peak duration refers to the structured data set containing peak intensity, timestamp, and duration.

[0054] In the embodiments of the present application, the peak intensity value is extracted from the maximum radiation intensity corresponding to the peak point; the peak timestamp is extracted from the specific moment of the peak point within the time window; the peak duration is the time length covered by the range that extends from the peak point to both sides until the intensity value drops to 80% of the peak. Finally, a structured metadata set is generated to characterize the foot stay characteristics within each window.

[0055] Step 125: Insert the metadata sets of all time windows into a preset sliding cache queue in chronological order. When the number of time windows in the sliding cache queue is equal to N, generate a user stay event based on all the metadata sets in the sliding cache queue.

[0056] Among them, the preset sliding cache queue refers to a dynamic storage structure that stores the metadata of N windows in chronological order.

[0057] In the embodiments of the present application, the capacity of the preset sliding cache queue is N time windows. Whenever the metadata of a new window is inserted into the queue, the data of the earliest window is automatically eliminated to keep the queue length fixed. When the queue is full, generate a user stay event based on the following rules: calculate the weighted sum of all peak intensity values in the queue, calculate the variance of the peak timestamp distribution to measure the concentration degree of the stay time; check whether the number of peaks in the queue exceeds the threshold. If the weight, dispersion, and density conditions are met simultaneously, it is determined as a valid user stay event.

[0058] The following is a specific example: In the intelligent clothing area of a large shopping mall, an infrared thermal imaging sensor array is evenly deployed on the top of the shelves to collect the foot thermal radiation signals of customers in real time. When a customer stops to browse in the men's summer clothing display area, the sensor captures the foot thermal radiation intensity at a sampling frequency of 10 times per second and synchronously transmits it to the data analysis module. The system performs a stability analysis on the signal intensity peaks within three consecutive 5 - second time windows, detects that the peak mean continuously exceeds the threshold and the fluctuation range is lower than the preset range, determines that a user stay event occurs at this location and records the start timestamp. Subsequently, the signal is decomposed in the frequency domain, and the low - frequency component of the pedestrian flow density indicates that the number of customers in the area rises smoothly, while the high - frequency component of the stay duration shows that the single stay lasts for 12 seconds. The coefficient of variation method is used to calculate the discrete difference degree between the two as 0.65, which is higher than the system - set threshold of 0.5, triggering the dynamic advertising strategy: the electronic screen hanging above the shelf immediately switches to a time - limited discount advertisement for men's summer clothing, synchronously plays a video introducing the function of breathable fabrics, and at the same time reduces the advertisement playback frequency in the adjacent women's clothing area. The whole process from data collection to advertisement content update takes less than 2 seconds, realizing a seamless connection between the customer's stopping intention and product promotion.

[0059] By executing steps 121 to 125, the embodiments of the present application enhance the ability to capture short-term signal changes through time slicing and differential calculation; the hierarchical filtering mechanism effectively suppresses environmental noise interference and improves the peak detection accuracy; the metadata extraction and sliding cache queue design support the dynamic correlation analysis of multi-window data and enhance the spatio-temporal consistency of the determination of stay events; ultimately, high-sensitivity and low false-alarm-rate user stay detection is achieved, providing reliable behavior data for precision marketing.

[0060] In a possible embodiment, step 125, when the number of time windows in the sliding cache queue is equal to N, generating a user stay event based on all the metadata sets in the sliding cache queue includes: Step a1, calculating the spatial cumulative weight and the time-domain dispersion based on the metadata sets of N time windows in the sliding cache queue.

[0061] Among them, the spatial cumulative weight refers to the weighted sum based on the peak intensity values and their position weight coefficients of multiple time windows, reflecting the spatial distribution and time continuity of the signal intensity. The time-domain dispersion refers to the variance of the peak timestamp distribution, used to quantify the concentration degree of the stay time, and the smaller the value, the more concentrated the time distribution. Calculation steps: Assign dynamic weight coefficients to each time window, and the weights decay according to the distance of the window from the current time. Weightedly fuse the intensity values and durations of all peak points within each window to generate a window comprehensive intensity value, and then accumulate the comprehensive intensity values of each window after multiplying them by their time weight coefficients. For the calculation of the time-domain dispersion, convert all peak timestamps into offsets relative to the start time of the queue, calculate the mean of all peak timestamps, and calculate the variance of the timestamp distribution, which is the time-domain dispersion.

[0062] In the embodiments of the present application, the spatial cumulative weight can be calculated by the weighted summation algorithm. Specifically: Multiply the peak intensity value of each window by the corresponding position weight coefficient, and then accumulate the weighted values of all windows. The time-domain dispersion is realized by calculating the variance of all peak timestamps. The formula is the square of the mean of the timestamps minus the mean of the squares of the timestamps, used to measure the concentration degree of the stay time.

[0063] Step a2, identifying peak clusters based on all the peak timestamps in the sliding cache queue.

[0064] Among them, a peak cluster refers to a set of multiple peak points identified by a clustering algorithm with closely related time intervals, which characterizes high-frequency staying behaviors. Identifying peak clusters is to arrange all peak timestamps in the sliding cache queue in chronological order, and use the sliding window algorithm to traverse the sequence. If the time interval between adjacent peaks is less than a preset separation threshold, they are merged into the same peak cluster; at the same time, the number of peaks and the time span within the cluster are recorded, and finally a cluster set that meets the minimum peak number constraint is output, which characterizes the high-frequency staying behaviors of users. In this embodiment, the size of the preset separation threshold is not specifically limited, and it can be 1 second, 2 seconds, 10 seconds, etc.

[0065] In the embodiment of the present application, a density clustering algorithm is used to analyze all peak timestamps in the sliding cache queue, and adjacent peak points with a time interval less than the preset threshold are merged into the same peak cluster. For example, if three peak timestamps are 10:00:03, 10:00:03.5, and 10:00:04.2 respectively, and the preset interval threshold is 1 second, they are merged into one peak cluster, and the number of peaks and the time span within the cluster are recorded.

[0066] Step a3, when the spatial cumulative weight value exceeds the preset weight threshold, the time domain dispersion is lower than the preset dispersion threshold, and the peak cluster meets the time density constraint, a user staying event is generated.

[0067] Among them, the preset dispersion threshold refers to the determination critical value of the time domain dispersion. If it is higher than this value, it is determined that the staying time is dispersed. The time density constraint refers to the minimum number of peaks required within the peak cluster, which is used to exclude occasional interferences.

[0068] In the embodiment of the present application, a spatial cumulative weight threshold, a time domain dispersion threshold, and a time density constraint are set. If the following conditions are met simultaneously: the spatial cumulative weight value > the preset weight threshold, the time domain dispersion < the preset dispersion threshold, and the number of peaks within the peak cluster ≥ the time density constraint, it is determined as a valid user staying event, and an event trigger instruction is generated.

[0069] The following is a specific example: In the smart clothing area of a large shopping mall, when a customer stops at the exhibition area of new smart temperature-controlled clothing, the infrared thermal imaging sensor array at the top of the shelf continuously captures the foot thermal radiation signal. The sliding buffer queue accumulates metadata for nearly 3 time windows, a total of 15 seconds, including 4 peak point intensity values of 1350, 1400, 1380, and 1420 units respectively. The system calculates that the spatial cumulative weight is 1600 units, exceeding the preset threshold of 1200, and the time-domain dispersion is 0.2, lower than the critical value of 0.5. Through density clustering, a dense cluster with a time span of 3 seconds and containing 3 peak points is identified, meeting the time density constraint. After all three conditions are met, it is determined as a valid user stay event, and immediately triggers the circular advertising screen above the exhibition area to switch to a holographic demonstration of the working principle of smart temperature-controlled fabrics, and synchronously pushes personalized customization services to the interactive terminal. In the classic business men's clothing area, due to the dispersion of the peak clusters and the excessive dispersion, the system maintains the original brand historical feature film playback, accurately distinguishing the marketing strategies of the technology experience and traditional shopping scenarios.

[0070] By executing steps a1 to a3, the embodiment of the present application quantifies the spatio-temporal correlation of the signal intensity distribution through the spatial cumulative weight, combines the time-domain dispersion and the peak cluster density constraint, realizes the detection of stay events with multi-dimensional joint criteria, and reduces the false triggering caused by environmental noise. At the same time, based on the dynamic sliding window and clustering analysis, it enhances the ability to capture short-term and high-frequency stay behaviors and improves the accuracy of user intention judgment.

[0071] In a possible embodiment, step 123, performing hierarchical filtering on the candidate peak points within each time window to obtain the final peak points, includes: Step b1, within each time window, calculate the intensity difference between the candidate peak point and each sampling point within the sampling point range. The sampling point range is the range composed of the first K sampling points and the last K sampling points before and after the candidate peak point.

[0072] Among them, the intensity difference of each sampling point refers to the difference value between the intensity of the candidate peak point and the intensities of its first and last K sampling points, which is used to verify the local maximum value.

[0073] In the embodiment of the present application, for each candidate peak point, expand K sampling points forward and backward centered on it, and sequentially calculate the difference value between the intensity value of the candidate peak point and the intensity value of each sampling point within the range. For example, if the intensity of the candidate peak point is 1200 units, the intensities of its first two sampling points are 900 and 1000 units, and the last two are 1100 and 1050 units, then the differences are 300, 200, 100, and 150 units respectively, which are used to verify whether the candidate point is a local maximum value.

[0074] Step b2, according to the intensity difference, retain the candidate peak points whose intensity values are greater than all sampling points within the sampling point range, and generate the preliminary screening peak points.

[0075] Among them, the candidate peak points of the sampling points refer to the positions of the sampling points that are initially detected as possibly being peaks and need to be further screened. The initially screened peak points refer to the candidate peak points retained through the local maximum condition.

[0076] In the embodiment of the present application, if the intensity value of a candidate peak point is greater than the intensity values of all sampling points within the range of K sampling points before and after it, then it is determined that this candidate point is an effective local maximum and is retained as an initially screened peak point; otherwise, it is excluded. For example, if the intensity of a certain candidate point is 1150 units, but the intensity of the sampling point immediately following it is 1200 units, then it does not meet the condition and is excluded.

[0077] Step b3: Detect the time interval between adjacent initially screened peak points, and merge adjacent initially screened peak points with a time interval less than a preset separation threshold to generate a candidate peak sequence.

[0078] Among them, the candidate peak sequence is a sequence formed after merging initially screened peak points with too close time intervals.

[0079] In the embodiment of the present application, traverse all initially screened peak points in chronological order and calculate the time interval between adjacent points. If the interval is less than the preset separation threshold, then they are merged into the same candidate peak sequence, and only the point with the maximum intensity is retained as a representative. For example, three peak points with an interval of 0.5 seconds are merged into one sequence, and the point with the highest intensity is retained.

[0080] Step b4: Perform stability verification on the candidate peak sequence to eliminate instantaneous fluctuation peak points with an intensity duration less than a preset duration threshold to obtain the final peak points.

[0081] Among them, stability verification refers to judging whether a peak is generated by a stable staying behavior through a duration threshold. An instantaneous fluctuation peak point refers to an invalid peak with an overly short duration caused by a short-term interference.

[0082] In the embodiment of the present application, count the duration of each peak point in the candidate peak sequence: centered on the peak point, expand forward and backward to the boundary points where the intensity value drops to 80% of the peak, and calculate the time span. If the duration is less than the preset threshold, then it is determined to be instantaneous noise interference and is excluded; otherwise, it is retained as the final peak point.

[0083] The following is a specific example: In the light-sensitive color-changing fabric exhibition area of the intelligent clothing area in a large shopping mall, the infrared sensor captures that the intensity of the customer's foot thermal radiation signal reaches 1450 units. The differences between the candidate peak point and the two sampling points before and the two sampling points after are calculated to be 320 units, 280 units, 210 units, and 190 units in the positive direction in sequence, all of which meet the local maximum conditions; it is retained as a preliminary screening peak point. Subsequently, it is detected that the time interval between two adjacent preliminary peak points is 0.5 seconds, which is lower than the preset separation threshold of 0.8 seconds. The two are merged into a candidate sequence and the peak point with a higher intensity of 1500 units is selected; it is verified that its duration reaches 0.8 seconds. After exceeding the preset threshold of 0.6 seconds, it is determined to be a valid peak point, which is incorporated into the sliding cache queue to participate in event analysis. Finally, it triggers the playback of the daylight response experiment video of the light-sensitive color-changing fabric on the circular screen in the exhibition area, and pushes the fabric maintenance guide and the reservation entrance for the customized dyeing scheme to the customer's portable smart device.

[0084] By performing step b1 to step b4, the embodiment of the present application excludes isolated noise interference through local maximum screening and time interval merging; the stability verification further filters out instantaneous fluctuations to ensure that the peak points reflect the real stay behavior; finally, high-robustness peak detection is achieved, the false detection rate is reduced, and a reliable data basis is provided for subsequent stay event analysis.

[0085] In a possible embodiment, S13: Perform frequency-domain decomposition on the user's foot thermal radiation signals at multiple positions to extract the crowd density component and the stay duration component synchronized with the time stamp of the user's stay event, including: Step 131: Perform frequency-domain decomposition on the user's foot thermal radiation signals at multiple positions to generate the steady-state amplitude-frequency component and the transient amplitude-frequency component at each position.

[0086] Among them, the steady-state amplitude-frequency component refers to the low-frequency band signal energy extracted through frequency-domain decomposition, reflecting the long-term trend of the crowd density. The transient amplitude-frequency component refers to the high-frequency band signal energy extracted through frequency-domain decomposition, characterizing the instantaneous intensity of short-term foot movements.

[0087] Step 132: Based on the time stamp of the user's stay event, perform spatial superposition on the steady-state amplitude-frequency components at multiple positions within the same time window to generate an aggregation component, and perform time accumulation on the transient amplitude-frequency components at a single position within the same time window to generate an integration component.

[0088] Among them, the aggregation component refers to the spatial weighted sum of the steady-state components at multiple positions, used to quantify the overall crowd density in the area. The integration component refers to the accumulated value of the transient components at a single position within the time window, characterizing the total intensity of the stay behavior.

[0089] Step 133: According to the aggregation component, intercept the amplitude extreme value interval aligned with the time stamp of the user's stay event to obtain the crowd density component.

[0090] Step 134: Calculate the integral slope that matches the time stamp of the user's stay event based on the integral component to obtain the stay duration component. Here, the integral slope refers to the change rate of the integral component curve, reflecting the duration characteristics of the stay behavior.

[0091] The following is a specific example: In the intelligent clothing area of a large shopping mall, an infrared sensor captures the foot thermal radiation signals of customers in real time. The steady-state component with an integral of 280 units in the low-frequency band energy and the transient component with a sudden change in the high-frequency band amplitude are extracted through frequency-domain decomposition. The steady-state components of two adjacent display cabinets are superimposed according to spatial weights to generate an aggregated component of 360 units. At the same time, the transient component in the current exhibition area is cumulated over time to obtain an integral component of 650 units. The amplitude extreme value interval of the aggregated component is intercepted according to the stay event time stamp, and the pedestrian flow density component is calculated to be 320 units. The integral curve is fitted to obtain a slope of 280 units per second, which is determined as an intention of a long stay. Based on this, the system controls the holographic projection screen in the exhibition area to switch to an animation demonstrating the morphological changes of memory shaping fibers when heated, synchronously pushes the fabric wrinkle resistance test data and the download link of the intelligent ironing program to the customer's mobile terminal, and circulates the reservation QR code for the customized tailoring service on the electronic price tag screen.

[0092] By executing Steps 131 to 134, the embodiments of the present application separate the steady-state and transient characteristics through frequency-domain decomposition, accurately distinguish the pedestrian flow density trend and short-term stay behavior; the spatial superposition enhances the robustness of the regional pedestrian flow analysis, and the time accumulation quantifies the single-point stay intensity; the extreme value interval interception and integral slope calculation respectively extract highly discriminative behavior indicators from the aggregated component and the integral component, improve the accuracy of stay intention judgment, and provide multi-dimensional data support for the dynamic advertising strategy.

[0093] In a possible embodiment, S15: Adjust the content playback queue of the advertising screens in the offline commercial premises according to the discrete difference degree, including: Step 151: Generate an advertising push priority index based on the discrete difference degree.

[0094] Exemplarily, the calculation formula of the advertising push priority index is: Advertising push priority index = Discrete difference degree × Weight scaling factor + Basic priority value. The specific form of the calculation formula of the advertising push priority index in this embodiment is not specifically limited.

[0095] Step 152: Adjust the content playback queue of the advertising screens in the offline commercial premises according to the advertising push priority index and the commodity category corresponding to the top of the shelf.

[0096] Among them, the dynamic priority queue algorithm is adopted, which combines the advertisement push priority index with the product category matching rule. The advertisements of the same product category are arranged in descending order of the priority index. If the priority index of a certain type of advertisement exceeds the threshold, for example, the advertisement priority index is 70 points, it will be inserted into the front end of the current playback queue. For every 10-point increase in the priority index, the corresponding advertisement playback duration is extended by 20%. Finally, an updated playback instruction is generated and sent to the advertisement screen controller for execution through the Internet of Things protocol.

[0097] The following is a specific example: In the light-sensitive color-changing fabric exhibition area of the intelligent clothing area in a large shopping mall, the system detects that the discrete difference degree of a certain customer stay event is 0.8. In step 151, an advertisement push priority index of 76 points is generated; in step 152, according to the corresponding product category "intelligent functional fabric" in this exhibition area, the advertisement priority of "sunlight color-changing T-shirt" in the retrieved and matched advertisement library is 76 points, and that of "temperature-sensitive color-changing sweatshirt" is 65 points. The former is inserted into the first place of the current playback queue and assigned 1.6 times the benchmark duration, interrupting the advertisement of "basic shirt" that is being played. At the same time, the latter is added to the second place in the pending playback queue, triggering the electrode screen to play the ultraviolet irradiation color-changing experiment video of the T-shirt, and scrolling the interactive entrance of "scan the code to experience virtual sun exposure color change" at the bottom of the screen, completing the closed-loop response from data analysis to advertisement placement.

[0098] By executing steps 151 to 152, the embodiment of the present application realizes the direct association between the dynamic change of user behavior and the advertisement push strategy through the linear mapping from the discrete difference degree to the priority index; combines the product category matching rule with the dynamic queue algorithm to ensure the instant insertion and precise exposure of high-priority advertisements, improves the spatio-temporal correlation between the advertisement content and the shelf products, and enhances the marketing conversion efficiency.

[0099] In a possible embodiment, step 151, generating an advertisement push priority index based on the discrete difference degree, includes: Step c1, generating a dynamic fusion weight coefficient based on the discrete difference degree.

[0100] Among them, the dynamic fusion weight coefficient refers to a dynamic adjustment parameter generated based on the discrete difference degree, reflecting the real-time association intensity between the change of user behavior and the advertisement push requirement. Exemplarily, the dynamic fusion weight coefficient = discrete difference degree / (1 + discrete difference degree).

[0101] Step c2, combining the preset scene three-dimensional model, determining the spatial topological association value between the spatial coordinates of the user stay event and the shelf vertex.

[0102] Among them, the preset scenario three-dimensional model includes a digital three-dimensional model of the shelf coordinates, path topology, and spatial structure of a commercial venue, which is used to calculate the position correlation. The straight-line distance between the user's staying position and the shelf vertex is calculated through the Euclidean distance, and then through normalization processing, a spatial topology correlation value is generated.

[0103] Step c3: Multiply the dynamic fusion weight coefficient by the spatial topology correlation value to obtain the advertisement push priority index.

[0104] Among them, the spatial topology correlation value refers to the quantization value of the spatial relationship between the user's staying position and the shelf vertex. The larger the value, the stronger the position correlation.

[0105] In the embodiment of the present application, a multiplication operation is performed on the generated dynamic fusion weight coefficient and the calculated spatial topology correlation value. The formula is: advertisement push priority index = dynamic fusion weight coefficient × spatial topology correlation value.

[0106] The following is a specific example: In the intelligent temperature control clothing exhibition area in the intelligent clothing area, the system detects that the discrete difference degree of a certain customer's staying event is 0.75, and generates a dynamic fusion weight coefficient of 0.75 / (1 + 0.75) = 0.43; according to the three-dimensional model, the distance between the staying position and the adjacent shelf vertex is calculated to be 2 meters, and the spatial topology correlation value is 0.8; calculate the advertisement push priority index of 0.43 × 0.8 = 0.344, and normalize it to 68 points. Based on this, the system inserts the advertisement of the temperature control fabric corresponding to this shelf into the first place of the playback queue, interrupts the current low-priority advertisement, and superimposes the try-on interface on the electronic screen. At the same time, a time-limited full reduction coupon is pushed to the customers within 5 meters, realizing precise marketing driven by both space and behavior.

[0107] By performing steps c1 to c3, in the embodiment of the present application, the weight coefficient is dynamically adjusted through the discrete difference degree, enhancing the response sensitivity to sudden changes in user behavior; combined with the spatial topology calculation of the three-dimensional model, the physical position correlation is accurately quantified; finally, the priority index is generated through product fusion, realizing an advertisement push strategy driven by both behavior and spatial position, and improving the spatio-temporal matching accuracy of content and scenario.

[0108] Figure 2 It is a schematic structural diagram of an advertisement push system based on multi-modal data fusion provided by the embodiment of the present application. As Figure 2 shown, the system includes: An acquisition module 21, configured to synchronously acquire the user's foot thermal radiation signals at multiple positions through an infrared thermal imaging sensor array deployed at the shelf vertices in an offline commercial venue.

[0109] A determination module 22, configured to determine the user's staying events at each position according to the intensity peak distribution of the user's foot thermal radiation signals at each position within N consecutive time windows.

[0110] An extraction module 23 is configured to perform frequency-domain decomposition on the user foot thermal radiation signals at multiple locations to extract a pedestrian flow density component and a stay duration component synchronized with the time stamps of user stay events.

[0111] A calculation module 24 is configured to calculate the discrete difference degree between the pedestrian flow density component and the stay duration component by using the coefficient of variation method.

[0112] A matching module 25 is configured to adjust the content playback queue of the advertising screens in the offline commercial venue according to the discrete difference degree, so that the played content matches the product category corresponding to the top of the shelf.

[0113] Figure 2 The described advertising push system based on multi-modal data fusion can execute Figure 1 The described advertising push method based on multi-modal data fusion in the illustrated embodiment, the implementation principle and technical effects will not be elaborated. For the advertising push system based on multi-modal data fusion in the above embodiment, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0114] In a possible design, Figure 2 The advertising push system based on multi-modal data fusion in the illustrated embodiment can be implemented as a computing device, such as Figure 3 shown, the computing device may include a storage component 31 and a processing component 32.

[0115] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32.

[0116] The processing component 32 is configured to: synchronously obtain user foot thermal radiation signals at multiple locations through an infrared thermal imaging sensor array deployed at the top of the shelves in the offline commercial venue; determine user stay events at each location according to the intensity peak distribution of the user foot thermal radiation signals at each location within N consecutive time windows; perform frequency-domain decomposition on the user foot thermal radiation signals at multiple locations to extract a pedestrian flow density component and a stay duration component synchronized with the time stamps of user stay events; calculate the discrete difference degree between the pedestrian flow density component and the stay duration component by using the coefficient of variation method; adjust the content playback queue of the advertising screens in the offline commercial venue according to the discrete difference degree, so that the played content matches the product category corresponding to the top of the shelf.

[0117] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above method.

[0118] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0119] Of course, the computing device may also necessarily include other components, such as input / output interfaces, display components, communication components, etc.

[0120] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module can be an output device, an input device, etc.

[0121] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0122] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server. The above-mentioned processing components, storage components, etc. can be basic server resources leased or purchased from a cloud computing platform.

[0123] An embodiment of the present application also provides a computer storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the above-mentioned Figure 1 advertising push method based on multi-modal data fusion shown in the embodiment.

[0124] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0126] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An advertisement pushing method based on multi-modal data fusion, characterized in that, Including: Synchronously acquiring the thermal radiation signals of users' feet at multiple positions through an infrared thermal imaging sensor array deployed at the top of the shelves in an offline commercial venue; Determining the user stay events at each position according to the intensity peak distribution of the thermal radiation signals of users' feet at each position within consecutive N time windows; Performing frequency-domain decomposition on the thermal radiation signals of users' feet at the multiple positions to extract the crowd density component and the stay duration component synchronized with the timestamps of the user stay events; Calculating the discrete difference degree between the crowd density component and the stay duration component by using the coefficient of variation method; Adjusting the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree so that the played content matches the commodity category corresponding to the top of the shelf; 2. The method according to claim 1, wherein The determining the user stay events at each position according to the intensity peak distribution of the thermal radiation signals of users' feet at each position within consecutive N time windows includes: Performing time slicing on the thermal radiation signals of users' feet at each position according to a fixed time window length to obtain the radiation intensity sequence of each time window corresponding to each position; For the radiation intensity sequence of each time window, performing adjacent sampling point difference calculation, generating an intensity change rate sequence according to the difference result, and screening out candidate peak points with an intensity change rate exceeding a preset intensity change rate threshold from the sampling point sequence corresponding to the radiation intensity sequence; Performing hierarchical filtering on the candidate peak points within each time window to obtain the final peak points; Based on the final peak points, performing metadata extraction operations within the corresponding time windows to generate a metadata set including peak intensity values, peak timestamps, and peak duration times; Inserting the metadata sets of all time windows into a preset sliding cache queue in chronological order. When the number of time windows in the sliding cache queue is equal to N, generating user stay events based on all the metadata sets in the sliding cache queue; 3. The method according to claim 2, wherein The generating user stay events based on all the metadata sets in the sliding cache queue when the number of time windows in the sliding cache queue is equal to N includes: Calculating the spatial cumulative weight and the time-domain discreteness based on the metadata sets of N time windows in the sliding cache queue; Identifying peak clusters based on all the peak timestamps in the sliding cache queue; When the spatial cumulative weight value exceeds a preset weight threshold, the time-domain discreteness is lower than a preset discreteness threshold, and the peak cluster satisfies the time density constraint, generating user stay events; 4. The method according to claim 2, wherein The performing hierarchical filtering on the candidate peak points within each time window to obtain the final peak points includes: Within each time window, calculating the intensity difference between the candidate peak points and each sampling point within the sampling point range, where the sampling point range is the range composed of the first K sampling points and the last K sampling points of the candidate peak point; According to the intensity difference, retaining the candidate peak points with intensity values greater than all sampling points within the sampling point range to generate preliminary screened peak points; Detecting the interval time between adjacent preliminary screened peak points, and merging adjacent preliminary screened peak points with an interval time less than a preset separation threshold to generate a candidate peak sequence; Perform stability verification on the candidate peak sequence to eliminate instantaneous fluctuation peak points with an intensity duration less than a preset duration threshold, and obtain the final peak points.

5. The method according to claim 1, wherein The frequency-domain decomposition of the user's foot thermal radiation signals at the multiple positions to extract the crowd density component and the stay duration component synchronized with the time stamp of the user's stay event includes: Perform frequency-domain decomposition on the user's foot thermal radiation signals at the multiple positions to generate a steady-state amplitude-frequency component and a transient amplitude-frequency component for each position; Based on the time stamp of the user's stay event, perform spatial superposition on the steady-state amplitude-frequency components at multiple positions within the same time window to generate an aggregated component, and perform time integration on the transient amplitude-frequency component at a single position within the same time window to generate an integrated component; According to the aggregated component, intercept the amplitude extreme value interval aligned with the time stamp of the user's stay event to obtain the crowd density component; According to the integrated component, calculate the integration slope matching the time stamp of the user's stay event to obtain the stay duration component.

6. The method according to claim 1, characterized in that The adjustment of the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree includes: Generate an advertisement push priority index based on the discrete difference degree; According to the advertisement push priority index and the commodity category corresponding to the shelf vertex, adjust the content playback queue of the advertising screen in the offline commercial venue.

7. The method according to claim 6, characterized in that, The generation of the advertisement push priority index based on the discrete difference degree includes: Generate a dynamic fusion weight coefficient based on the discrete difference degree; In combination with a preset scene three-dimensional model, determine the spatial topological association value between the spatial coordinates of the user's stay event and the shelf vertex; Perform a multiplication operation on the dynamic fusion weight coefficient and the spatial topological association value to obtain the advertisement push priority index.

8. An advertisement push system based on multi-modal data fusion, characterized in that, Includes: An acquisition module for synchronously acquiring the user's foot thermal radiation signals at multiple positions through an infrared thermal imaging sensor array deployed at the shelf vertices in the offline commercial venue; A determination module for determining the user's stay event at each position according to the intensity peak distribution of the user's foot thermal radiation signals at each position within N consecutive time windows; An extraction module for performing frequency-domain decomposition on the user's foot thermal radiation signals at the multiple positions to extract the crowd density component and the stay duration component synchronized with the time stamp of the user's stay event; A calculation module for calculating the discrete difference degree between the crowd density component and the stay duration component by using the coefficient of variation method; A matching module for adjusting the content playback queue of the advertising screen in the offline commercial venue according to the discrete difference degree so that the playback content matches the commodity category corresponding to the shelf vertex.

9. A computing device, characterized in that, Includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement an advertisement push method based on multi-modal data fusion as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, Stores a computer program, and when the computer program is executed by a computer, it implements an advertisement push method based on multi-modal data fusion as described in any one of claims 1 to 7.

Images