System for playing AR interactive advertisement based on multimedia charging pile

Abstract

The invention discloses an AR interactive advertisement playing system based on a multimedia charging pile, and relates to the technical field of augmented reality, and the system comprises a multi-source collection module which continuously monitors a vehicle charging event, triggers multi-focus synchronous imaging, and collects a multi-source data set in real time; the feature extraction module is used for performing space-time alignment on the multi-source data set and extracting multi-source sensing features to form a multi-source sensing feature set; the dynamic coupling module is used for collecting time-space difference data in real time, carrying out dynamic coupling by combining a combined judgment result and generating an audience portrait and a strategy constraint set; and the chain type putting module is used for carrying out material priority adjustment on the structured interaction feedback data and triggering chain type advertisement logic when detecting that the same vehicle appears at different charging piles. According to the invention, the dynamic coupling module collects the space-time difference data in real time and combines the joint judgment result to carry out dynamic coupling, so that the deep fusion of the user state and the environment context is realized, and a multi-dimensional constraint basis is provided for an advertisement strategy.

Description

Technical Field

[0001] This invention relates to the field of augmented reality technology, and in particular to an AR interactive advertising system based on multimedia charging piles. Background Technology

[0002] Against the backdrop of rapid development in intelligent transportation and new energy vehicles, multimedia advertising systems based on charging piles are gradually becoming an important carrier for urban digital marketing. Existing technologies typically rely on integrated displays or audio equipment within charging piles to push static or pre-set advertising content while the vehicle is charging. Some systems incorporate basic sensors to identify the presence of vehicles or users, thereby triggering ad playback. These methods, to a certain extent, achieve contextualized advertising, providing new pathways for commercial information delivery.

[0003] However, conventional methods have limitations in terms of audience identification accuracy and dynamic adaptation of advertising content. On the one hand, they often rely on a single data source for user judgment, making it difficult to accurately construct high-confidence vehicle-person relationships; on the other hand, advertising creatives and delivery strategies lack deep coupling with the real-time environment, user behavior, and historical interactions, resulting in low content matching and limited interactivity. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a multimedia charging pile-based AR interactive advertising system to solve the collaborative optimization problem of AR interactive advertising in multi-source perception and dynamic coupling.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides an AR interactive advertising system based on multimedia charging piles, comprising: a multi-source acquisition module that continuously monitors vehicle charging events and triggers multi-focus synchronous imaging to acquire multi-source datasets in real time; a feature extraction module that performs spatiotemporal alignment on the multi-source datasets and extracts multi-source perception features to form a multi-source perception feature set; a recognition confidence analysis module that analyzes the confidence of vehicle and human image recognition based on the multi-source perception feature set to obtain a joint judgment result; a dynamic coupling module that collects spatiotemporal difference data in real time and performs dynamic coupling with the joint judgment result to generate an audience profile and a strategy constraint set; a semantic matching module that matches the audience profile and strategy constraint set according to the three-layer semantics of vehicle, crowd, and scene to obtain a material instruction set and display control parameters; an instruction generation module that performs brightness and color adaptation and projection posture correction on the material instruction set and display control parameters to obtain multimodal interaction and behavior instructions; an interaction recording module that executes multimodal interaction and behavior instructions and records structured interaction feedback data; and a chained delivery module that adjusts the material priority on the structured interaction feedback data and triggers chained advertising logic when the same vehicle is detected to appear on different charging piles.

[0007] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the vehicle charging event refers to the vehicle plugging in, unplugging, or approaching the charging area. The multi-source dataset includes vehicle information, vehicle and personnel information, and environmental information.

[0008] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the specific steps for spatiotemporal alignment of multi-source datasets and extraction of multi-source perceptual features to form a multi-source perceptual feature set are as follows. Timestamp calibration and lens field of view registration are performed on the multi-source dataset to obtain the calibrated multi-source dataset; The calibrated multi-source dataset is divided into regions and targets are extracted to form a primary feature set; Spatiotemporal correlation is performed on the primary feature set to obtain a multi-source sensing feature set.

[0009] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the specific steps for obtaining a joint judgment result by analyzing the confidence levels of vehicle and human facial recognition based on multi-source perception feature sets are as follows. Analyze the temporal and locational relationships of multi-source sensing feature sets to form a candidate set of associated records; Based on the candidate associated record set, the current vehicle is compared with the historical records item by item to obtain the preliminary evaluation results of vehicle recognition confidence and facial recognition confidence. The initial assessment results are corrected in context and the current vehicle is confirmed to have a record. The joint judgment result is then output.

[0010] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the steps for real-time collection of spatiotemporal difference data, combined with joint judgment results for dynamic coupling, to generate audience profiles and strategy constraint sets are as follows. The spatiotemporal difference data includes the current location and time clues, parking space number and surrounding crowd density, on-site lighting and weather conditions, vehicle dwell time and distance of people approaching; The spatiotemporal difference data are matched with the joint judgment results item by item, and interactive opportunity points and potential interference factors are extracted to obtain a key information set; Multi-dimensional profiling constraints are applied to the key information set to generate audience profiles and a set of strategy constraints.

[0011] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the steps for matching the audience profile with the strategy constraint set according to the three-layer semantics of vehicle, crowd, and scene to obtain the material instruction set and display control parameters are as follows. The audience profile and strategy constraint set are classified in parallel, and semantic cross-matching is performed based on the parallel classification results to obtain the advertising theme combination; The advertising theme combination is modified and parameterized for vehicle, target audience and scene adaptability, forming material instruction set and display control parameters.

[0012] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the specific steps for performing brightness and color adaptive adjustment and projection posture correction on the material instruction set and display control parameters to obtain multimodal interaction and behavior instructions are as follows. Based on the display control parameters, brightness is mapped to illumination and color temperature data to obtain an adaptive brightness value; The projection unit's attitude is corrected using adaptive brightness values ​​to determine the projection angle and position parameters. By combining projection angle and position parameters, the material instruction set is rendered synchronously to generate multimodal interaction and behavior instructions.

[0013] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the specific steps for executing multimodal interaction and behavioral commands and recording structured interaction feedback data are as follows: Perform multimodal interaction and behavioral commands to play AR ads, and switch the ads to interactive modes based on the user's location; Real-time collection of interaction records between people and AR advertisements in interactive mode yields structured interaction feedback data.

[0014] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the material priority adjustment of structured interactive feedback data refers to the weight rearrangement based on structured interactive feedback combined with audience profiles and strategy constraint sets to form a priority list.

[0015] As a preferred embodiment of the AR interactive advertising system based on multimedia charging piles described in this invention, the specific steps for triggering chained advertising logic when the same vehicle is detected appearing at different charging piles are as follows: Based on the joint determination results, the occurrence of the same vehicle at different charging piles is correlated in time sequence, cross-site link context is generated, and vehicle and vehicle personnel information is saved to the historical record database. Based on the cross-site link context and combined with the current audience profile and strategy constraints, determine the delivery stage and content for this site, generate updated strategy parameters, and simultaneously distribute them to each terminal.

[0016] The beneficial effects of this invention are as follows: By dynamically coupling the spatiotemporal difference data in real time through the dynamic coupling module and combining it with the joint judgment results for dynamic coupling, an accurate audience profile and strategy constraint set are generated, realizing a deep integration of user state and environmental context, and providing multi-dimensional constraint basis for advertising strategies; by triggering the chain advertising logic when the same vehicle appears across charging piles through the chain delivery module, continuous content delivery based on user travel trajectory is realized, effectively improving the coherence of advertising narrative and the depth of user participation; AR interactive advertising not only has scene adaptive capabilities, but also forms a personalized dissemination closed loop across sites, enhancing the immersion, timeliness and conversion efficiency of advertising. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an AR interactive advertising system based on a multimedia charging pile.

[0019] Figure 2 A flowchart for generating multi-source sensing feature sets.

[0020] Figure 3 This is a flowchart for the joint determination process.

[0021] Figure 4 This is a flowchart of the chain-based delivery process. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides an AR interactive advertising system based on a multimedia charging pile, including the following steps: The multi-source acquisition module continuously monitors vehicle charging events and triggers multi-focus synchronous imaging to acquire multi-source datasets in real time. A vehicle charging event refers to a vehicle plugging in or unplugging the charging gun, or being near a charging area. Furthermore, the vehicle plugging-in action is obtained by detecting the physical connection status between the charging gun and the vehicle charging port through the charging gun status sensor, the unplugging action is obtained by detecting the change in the state of the charging gun detaching from the vehicle charging port through the charging gun status sensor, and the vehicle approaching the charging area is obtained by detecting the spatial position of the vehicle's outline entering the boundary of the charging area through video monitoring equipment combined with infrared sensing device. Multi-source datasets include vehicle information, vehicle and personnel information, and environmental information; Furthermore, multiple imaging units deployed at different angles and focal lengths are activated simultaneously to capture concurrent images of the same spatiotemporal region through a multi-focus synchronous imaging mechanism. Among them, vehicle information is obtained by acquiring images of vehicle appearance, license plate and model through imaging units at the front and side views; vehicle and personnel information is obtained by acquiring images of facial features, posture and behavior of people around the vehicle through imaging units close to the vehicle; and environmental information is obtained by acquiring images of lighting conditions, weather conditions, distribution of people in the surrounding area and parking space numbers through wide-angle imaging units.

[0026] The feature extraction module performs spatiotemporal alignment on the multi-source dataset and extracts multi-source sensing features to form a multi-source sensing feature set. Timestamp calibration and lens field of view registration are performed on the multi-source dataset to obtain the calibrated multi-source dataset; Furthermore, by applying a time synchronization protocol to align the timestamps of images acquired by each imaging unit in the multi-source dataset, time offset is eliminated, and a time-consistent multi-source dataset is obtained. Lens field-of-view registration identifies common spatial features (such as fixed objects in the charging area, parking space boundaries, and charging equipment outlines) in images from different imaging units in the multi-source dataset through feature extraction algorithms, and maps the fields of view of images from different imaging units in the multi-source dataset to a unified spatial coordinate system, thereby obtaining a spatially consistent multi-source dataset. Combining timestamp calibration and lens field-of-view registration, a calibrated multi-source dataset is obtained. It should be noted that the principle of the feature extraction algorithm is to detect and match common spatial features in images of different imaging units (such as fixed objects in the charging area, parking space boundaries and charging equipment outlines), calculate the geometric transformation relationship between images, and thus map the multi-view field of view to a unified spatial coordinate system to achieve registration. The calibrated multi-source dataset is divided into regions and targets are extracted to form a primary feature set; Furthermore, target extraction is performed on the segmented regions of interest. Target detection algorithms are applied to locate and separate independent targets from the regions of interest, including vehicle appearance, license plate, vehicle type, facial features, posture, behavior, lighting conditions, weather conditions, surrounding crowd distribution, and parking space numbers, obtaining target image patches. Feature extraction is then performed on these target image patches. Scale-invariant feature transform algorithms are applied to calculate feature vectors describing texture, shape, and edge information from the target image patches. Finally, feature concatenation operations are used to integrate all feature vectors calculated by traditional image feature descriptor algorithms, forming a primary feature set. It should be noted that the principle of the target detection algorithm is to generate candidate regions through a sliding window, extract multi-scale gradient features (such as histogram of oriented gradients, HOG) of each region to capture edge and texture information, calculate the class probability of multi-scale gradient features (such as the confidence of class such as vehicles and people), output the localized target bounding box and crop it to generate independent target image patches. Spatiotemporal correlation is performed on the primary feature set to obtain a multi-source sensing feature set; Furthermore, based on the timestamps and spatial coordinates of each target image patch in the calibrated multi-source dataset, the feature vectors from vehicle appearance, license plate, vehicle type, facial features of people, posture of people, behavior of people, lighting conditions, weather conditions, distribution of surrounding crowds and parking space numbers in the primary feature set are bound according to the acquisition time and spatial location (e.g., vehicle appearance - acquisition time - spatial location). A spatiotemporal correspondence is established for the feature vectors in the primary feature set, forming an associated feature structure that includes vehicles, people and environment in a unified spatiotemporal context, thus obtaining a multi-source perception feature set.

[0027] The confidence analysis module analyzes the confidence levels of vehicle and human face recognition based on multi-source perception feature sets to obtain joint judgment results; Analyze the temporal and locational relationships of multi-source sensing feature sets to form a candidate set of associated records; Furthermore, the timestamps and spatial coordinates associated with each set of feature vectors are extracted from the multi-source sensing feature set. Using a time series alignment method, feature vectors with similar timestamps are grouped into the same time window to obtain time-related groups. A spatial clustering method is then used to perform proximity analysis on the spatial coordinates of each feature vector within the same time window, grouping vehicle features, personnel features, and environmental features located within the same charging area into the same spatial cluster. Finally, the time-related groups and spatial clusters are cross-matched to form a spatiotemporal data set defined by both time window identifiers and spatial location identifiers, expressed as: ; in, For spatiotemporal data sets, For feature vectors, The timestamp of the feature vector. As the base timestamp, For time window, For the coordinates in the feature vector space, The center coordinates of the charging area The geographical area of ​​the charging zone; It should be noted that the reference timestamp refers to the timestamp when the system first detected the vehicle, and the center coordinates of the charging area are determined by the wide-angle imaging unit; The features of vehicle appearance, license plate number, vehicle type, facial features of people, posture of people, behavior of people, lighting conditions, weather conditions, distribution of surrounding people and parking space number in each spatiotemporal data set are combined and encapsulated to generate a record that contains the co-occurrence of vehicles and people at a specific time and place; the records corresponding to all spatiotemporal data sets are summarized to form a candidate association record set; Based on the candidate associated record set, the current vehicle is compared with the historical records item by item to obtain the preliminary evaluation results of vehicle recognition confidence and facial recognition confidence. Furthermore, the license plate number (in string form) corresponding to the current vehicle is extracted from the candidate associated record set; a search is conducted in the historical record database to determine if a completely matching license plate record exists. If a matching record exists, the initial assessment result of the vehicle recognition confidence is set to 1.0; otherwise, it is set to 0.0. The facial features corresponding to the current person are extracted from the candidate associated record set, and facial features (such as the relative distance between the eyes and nose and the local binary pattern of skin texture) from historical portrait records are retrieved from the historical record database. The similarity between the current facial features and the facial features in historical portrait records is calculated using a face comparison algorithm to obtain the initial assessment result of the face recognition confidence. It should be noted that the principle of the face comparison algorithm is to extract facial feature embedding vectors and calculate the cosine similarity between them and historical portrait records to quantify the confidence of identity matching.

[0028] Furthermore, based on the initial evaluation results of vehicle recognition confidence and facial recognition confidence, and combined with the corresponding time window identifier, spatial location identifier, vehicle dwell time, personnel approach distance, surrounding crowd density, and on-site lighting and weather conditions in the candidate associated record set, the context consistency verification method is used to determine whether the current vehicle and personnel are logically consistent in their physical behavior. If the vehicle dwell time and personnel approach distance meet the typical interaction pattern in the charging scenario, the vehicle recognition confidence and facial recognition confidence are increased. If the surrounding crowd density is too high, causing the personnel's facial features to be blurred, or the lighting conditions are too dark, affecting license plate recognition, the corresponding confidence is reduced. The current vehicle appearance record, the corresponding personnel information, spatiotemporal information, and the corrected confidence are encapsulated, and the joint judgment result is output. It should be noted that the typical interaction mode refers to calling the corresponding brand's customized materials based on the vehicle brand / model (such as prioritizing matching the same brand's peripheral products with electric vehicle brand advertisements), and filtering the material type based on the charging time—if the charging time is ≥30 minutes, long-term immersive AR content (such as 3D car disassembly animation) is pushed; if it is <30 minutes, short-term promotional information is pushed (such as "charging for 20 yuan or more to get a car wash coupon, car insurance advertisement, car cover advertisement, etc."); the matching process takes ≤50ms.

[0029] The dynamic coupling module collects spatiotemporal difference data in real time, combines the joint judgment results to perform dynamic coupling, and generates audience profiles and strategy constraint sets. The spatiotemporal difference data includes the current location and time clues, parking space number and surrounding crowd density, on-site lighting and weather conditions, vehicle dwell time and the distance people approach; Furthermore, the geographical coordinates of the vehicle are obtained through positioning devices deployed in the charging area, and the current time is obtained through network protocols (such as Network Time Protocol) to form a current location and time clue; images of the charging area are acquired through a wide-angle imaging unit to obtain parking space numbers, and a crowd density estimation algorithm is applied to statistically analyze the distribution area of ​​people in the image to obtain the surrounding crowd density; ambient light sensors collect the on-site light intensity, and combined with weather recognition algorithms in video surveillance equipment, the sky area image is analyzed to obtain the on-site lighting and weather conditions; the charging pile charging timer is used to calculate the vehicle's dwell time, and based on the spatial coordinates of the vehicle and people in the multi-source perception feature set, the Euclidean distance calculation method is used to obtain the distance of people approaching; It should be noted that the principle of the crowd density estimation algorithm is to capture real-time images of the charging area through a wide-angle camera, generate pedestrian bounding boxes and count the total number; the actual physical area of ​​the charging area (unit: square meters) is obtained through area boundary mapping (for example, using known parking space dimensions or fixed landmarks to convert pixels to physical coordinates); the ratio of the total number of detected pedestrians to the actual physical area of ​​the charging area is calculated to obtain the crowd density value; The spatiotemporal difference data are matched with the joint judgment results item by item, and interactive opportunity points and potential interference factors are extracted to obtain a key information set; Furthermore, the current vehicle appearance record, personnel information, and corrected confidence level contained in the joint judgment result are aligned at the field level with the current location and time clues, parking space number and surrounding crowd density, on-site lighting and weather conditions, vehicle dwell time and personnel approach distance in the spatiotemporal difference data. The time window identifier of the joint judgment result is associated with the collection time of the spatiotemporal difference data through timestamp matching, and the spatial location identifier of the joint judgment result is bound to the parking space number through spatial coordinate matching. On this basis, interactive opportunity points are marked according to vehicle dwell time and personnel approach distance (e.g., vehicle dwell time ≥ 30 minutes, personnel approach distance ≤ 5m). Potential interference factors (e.g., obstruction, low light, or severe weather) are judged according to surrounding crowd density, on-site lighting intensity, and weather conditions. All interactive opportunity points and potential interference factors, together with the corresponding spatiotemporal difference data and joint judgment result fields, are structurally integrated to obtain a key information set. Multi-dimensional profiling constraints are applied to the key information set to generate audience profiles and a set of strategy constraints. Furthermore, based on the key information set, the system obtains the consumption preference tags of the user group to which the vehicle belongs through brand knowledge base mapping based on vehicle brand and model. Based on facial features, it obtains gender and age range tags through facial attribute analysis algorithms. Based on charging time, it divides the user into commuting time, nighttime, or weekend scenarios through time semantic classification methods and integrates them in a structured manner to form an audience profile that includes gender and age range tags, consumption preference tags, and scenario tags. The system then triggers corresponding delivery strategy constraints according to potential interference factors, such as privacy protection constraints triggered by high crowd density, brightness enhancement constraints triggered by low light, and content duration limits triggered by short dwell time, to obtain a set of strategy constraints. Finally, the audience profile and the set of strategy constraints are jointly output. It should be noted that the brand knowledge base was obtained by integrating publicly available databases of vehicle brands and models, official product information from manufacturers, data from third-party automotive information platforms, and historical advertising records.

[0030] The semantic matching module matches the audience profile and strategy constraint set according to the three-layer semantics of vehicle, crowd and scene to obtain the material instruction set and display control parameters; The audience profile and strategy constraint set are classified in parallel, and semantic cross-matching is performed based on the parallel classification results to obtain the advertising theme combination; Furthermore, the consumer preference tags are classified into vehicle semantic categories using the consumer preference mapping relationship in the brand knowledge base, and the scene tags are classified into scene semantic categories using the time semantic classification method, to obtain the audience semantic category, vehicle semantic category, and scene semantic category. The audience semantic category, vehicle semantic category, and scene semantic category are jointly matched using a multi-dimensional semantic association rule base to filter out advertising theme items that simultaneously satisfy the vehicle semantic category, audience semantic category, and scene semantic category. All successfully matched advertising theme items are combined and encapsulated to obtain the advertising theme combination. It should be noted that the multidimensional semantic association rule base is obtained by integrating the vehicle brand and model database in the brand knowledge base, official product information from manufacturers, data from third-party automotive information platforms, and historical advertising records. The advertising theme combination is modified and parameterized for vehicle, target audience and scene adaptability, forming material instruction set and display control parameters; Furthermore, each ad theme item in the ad theme combination is checked for consistency with the vehicle brand / model in the joint judgment result, the scene tag in the candidate association record set, and the gender and age range tag in the audience profile. Ad theme items that pass the consistency check are retained, and those that fail are removed, thus completing the vehicle, audience, and scene adaptability correction. The corrected ad theme items are then structured according to material ID, playback sequence, and interaction trigger conditions to obtain material instruction set, and the corresponding illumination compensation parameters, projection area coordinates, and color correction coefficients are extracted simultaneously as display control parameters. It should be noted that the illumination compensation parameters are obtained by collecting the ambient light intensity from the ambient light sensor; the projection area coordinates are obtained by identifying the projection target area through the spatial coordinates of vehicles and personnel in the multi-source sensing feature set and the wide-angle imaging unit camera; the color correction coefficient is obtained by judging the current weather conditions from the ambient environmental images collected by the video monitoring equipment and combining the color temperature data output by the ambient light sensor, and then calculating it through the color balance correction algorithm.

[0031] The instruction generation module performs brightness and color adaptation and projection posture correction on the material instruction set and display control parameters to obtain multimodal interaction and behavior instructions; Based on the display control parameters, brightness is mapped to illumination and color temperature data to obtain an adaptive brightness value; Furthermore, illumination compensation parameters are extracted from the display control parameters. Combined with ambient light intensity and color temperature data collected by the ambient light sensor, the ambient light intensity and color temperature data are converted into brightness adjustment values ​​using a brightness-color adaptive method. These brightness adjustment values ​​are then weighted and fused with the illumination compensation parameters to calculate the adaptive brightness value, expressed as: ; in, For adaptive brightness values, These are the illumination compensation parameters. This is the brightness adjustment amount. The intensity of the ambient light. This is color temperature data; The projection unit's attitude is corrected using adaptive brightness values ​​to determine the projection angle and position parameters. Furthermore, based on the adaptive brightness value, the angle deviation between the optical axis of the projection unit and the normal vector of the target projection area is calculated using the projection attitude correction method. The pitch angle and yaw angle of the advertising projection device are adjusted according to the angle deviation to obtain the projection angle. Based on the boundary range in the projection area coordinates and the field of view parameters of the projection unit, the projection position of the projection unit in three-dimensional space is inferred through the inverse projection mapping method to obtain the position parameters. By combining projection angle and position parameters, the material instruction set is rendered synchronously to generate multimodal interaction and behavior instructions; Furthermore, the material instruction set is input into the charging pile rendering engine to ensure the visual consistency of AR content on the target projection area; on this basis, the corresponding voice prompts, gesture recognition areas or touch response logic are activated according to the interaction trigger conditions, and the corrected visual content and interaction logic are encapsulated into multimodal interaction and behavior instructions containing image frame sequences, spatial anchors and interaction event binding relationships. It should be noted that the charging pile rendering engine refers to the graphics processing component deployed in the edge computing device or advertising terminal that is linked to the charging pile.

[0032] The interaction recording module executes multimodal interactions and behavioral commands, and records structured interaction feedback data; Perform multimodal interaction and behavioral commands to play AR ads, and switch the ads to interactive modes based on the user's location; Furthermore, the multimodal interaction and behavioral commands are input into the charging pile rendering engine, which drives the projection unit to project AR advertising content onto the target projection area in real time. The distance between the current location of the person and the AR advertising interaction area is calculated using the spatial proximity judgment method based on the continuously updated spatial coordinates of the person in the multi-source perception feature set. When the person's location enters the interaction trigger range (distance < 5m), the corresponding gesture recognition area or voice prompt logic is activated according to the interaction trigger conditions defined in the multimodal interaction and behavioral commands, and the AR advertisement is switched from playback mode to interaction mode. Real-time collection of interaction records between people and AR advertisements in interactive mode to obtain structured interaction feedback data; Furthermore, after the interaction mode is activated, the imaging unit deployed in the charging area continuously captures the visual interaction behavior between people and AR advertisements. The gesture recognition algorithm is used to identify the gestures of people and obtain the gesture type and action sequence. The voice input content is acquired through the voice acquisition device, and the voice signal is converted into text semantics using the voice recognition algorithm. The gesture type, action sequence, text semantics and the binding relationship between the multimodal interaction and the interaction events defined in the behavior instructions are matched to determine the specific interaction intent. Combined with the timestamp of the interaction, the spatial coordinates of the person and the corresponding material ID, the data is organized into structured interaction feedback data containing interaction type, interaction content, spatiotemporal location and material identifier through field structuring methods. It should be noted that the principle of gesture recognition algorithm is to match human gestures with a predefined gesture template library to obtain the gesture type and action sequence; the principle of speech recognition algorithm is to align speech signals with the feature sequences of pre-stored speech templates and output the matching text semantics; the gesture template library is obtained by manually demonstrating predefined interactive gestures (such as waving and clicking) and storing them as templates; the speech templates are obtained by manually pre-recording standard command speech samples (such as start and confirm) as reference templates.

[0033] The chained delivery module adjusts the priority of materials based on structured interactive feedback data and triggers chained advertising logic when the same vehicle is detected to appear at different charging stations. Based on structured interactive feedback, combined with audience profiles and policy constraint sets, weights are rearranged to form a priority list; Furthermore, interaction type, interaction content, spatiotemporal location, and material identifiers are extracted from structured interaction feedback data, and the material identifiers are mapped to the corresponding advertising theme items. Based on the interaction type and content, user response scores are obtained for each advertising theme item (e.g., confirmed interaction = 1.0 points, clicked interaction = 0.7 points, ignored interaction = 0.2 points). The user response scores are correlated with gender and age range tags, consumption preference tags, and scene tags in the audience profile, and a content fit score is obtained using a tag matching calculation method. Combining the content duration upper limit constraint, brightness enhancement constraint, and privacy protection constraint in the strategy constraint set, it is checked whether any advertising theme item violates any constraint (e.g., content duration ≤ upper limit, brightness enhancement enabled when light intensity < daytime, privacy protection activated when crowd density > threshold). If no violation occurs, the feasibility score is 1.0; otherwise, the feasibility score is 0.0. The user response score, content fit score, and feasibility score are weighted and integrated to obtain the comprehensive priority weight for each advertising theme. All advertising theme items are sorted in descending order based on the comprehensive priority weight to form a priority list. Based on the joint determination results, the occurrence of the same vehicle at different charging piles is correlated in time sequence, cross-site link context is generated, and vehicle and vehicle personnel information is saved to the historical record database. Furthermore, vehicle identification confidence, license plate features, vehicle model features, vehicle appearance features, facial features of personnel, time window identifiers, and spatial location identifiers are extracted from the joint judgment results generated by each charging pile. License plate features of different charging piles are matched using a license plate recognition comparison algorithm to confirm whether they belong to the same vehicle. Multiple joint judgment results confirming the same vehicle are sorted chronologically according to the time window identifier to form a sequence of vehicle occurrences across charging piles. The facial features of personnel, spatial location identifiers, and time window identifiers corresponding to each occurrence record in the sequence are bound together to construct a cross-station link context that includes vehicle trajectory, personnel companion relationships, and temporal evolution relationships. The cross-station link context, along with the corresponding vehicle and personnel information, is written to a historical record database using a data persistence method. It should be noted that the historical record database refers to a database that stores cross-site link context along with the corresponding vehicle and vehicle personnel information; Based on the cross-site link context and combined with the current audience profile and strategy constraints, determine the delivery stage and content for this site, generate updated strategy parameters, and simultaneously distribute them to each terminal. Furthermore, the cross-site link context is retrieved from the historical record database, and the sequence of vehicle appearances across charging piles, the accompanying relationships of personnel, and the time-series evolution relationships are analyzed to identify the current stage of the vehicle in the link. This is then matched with typical interaction patterns to determine the appropriate placement stage for this site. Combining the gender and age range tags, consumption preference tags, and scenario tags in the current audience profile, as well as the content duration limit constraints, brightness enhancement constraints, and privacy protection constraints in the strategy constraint set, a multi-constraint joint decision-making method is used to select advertising themes suitable for the placement stage of this site. The selection results are cross-compared with the priority list, and the advertising theme with the highest comprehensive priority weight is selected as the content to be placed on this site. The updated strategy parameters are then synchronously distributed to each terminal via network communication protocols. In summary, this invention achieves deep integration of user state and environmental context by: a dynamic coupling module that collects spatiotemporal difference data in real time and combines it with joint judgment results to generate accurate audience profiles and strategy constraint sets, providing multi-dimensional constraint basis for advertising strategies; a chain-based delivery module that triggers chain-based advertising logic when the same vehicle is detected crossing charging piles, achieving continuous content delivery based on user travel trajectories, effectively improving the coherence of advertising narratives and the depth of user engagement; and enabling AR interactive advertising to not only have scene adaptability but also form a personalized dissemination loop across sites, enhancing the immersion, timeliness, and conversion efficiency of advertising.

[0034] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multimedia charging pile-based AR interactive advertising system, characterized in that: include, The multi-source acquisition module continuously monitors vehicle charging events and triggers multi-focus synchronous imaging to acquire multi-source datasets in real time. The feature extraction module performs spatiotemporal alignment on the multi-source dataset and extracts multi-source sensing features to form a multi-source sensing feature set. The confidence analysis module analyzes the confidence levels of vehicle and human face recognition based on multi-source perception feature sets to obtain joint judgment results; The dynamic coupling module collects spatiotemporal difference data in real time, combines the joint judgment results to perform dynamic coupling, and generates audience profiles and strategy constraint sets. The semantic matching module matches the audience profile and strategy constraint set according to the three-layer semantics of vehicle, crowd and scene to obtain the material instruction set and display control parameters; The instruction generation module performs brightness and color adaptation and projection posture correction on the material instruction set and display control parameters to obtain multimodal interaction and behavior instructions; The interaction recording module executes multimodal interactions and behavioral commands, and records structured interaction feedback data; The chained delivery module adjusts the priority of materials based on structured interactive feedback data and triggers chained advertising logic when the same vehicle is detected to appear at different charging stations.

2. The AR interactive advertising system based on multimedia charging piles as described in claim 1, characterized in that: The vehicle charging event refers to the vehicle plugging in or unplugging the charging gun, or approaching the charging area. The multi-source dataset includes vehicle information, vehicle and personnel information, and environmental information.

3. The AR interactive advertising system based on multimedia charging piles as described in claim 2, characterized in that: The specific steps for spatiotemporal alignment of the multi-source dataset and extraction of multi-source sensing features to form a multi-source sensing feature set are as follows. Timestamp calibration and lens field of view registration are performed on the multi-source dataset to obtain the calibrated multi-source dataset; The calibrated multi-source dataset is divided into regions and targets are extracted to form a primary feature set; Spatiotemporal correlation is performed on the primary feature set to obtain a multi-source sensing feature set.

4. The AR interactive advertising system based on multimedia charging piles as described in claim 3, characterized in that: The method for analyzing the confidence levels of vehicle and human facial recognition based on multi-source perception feature sets to obtain a joint judgment result involves the following specific steps. Analyze the temporal and locational relationships of multi-source sensing feature sets to form a candidate set of associated records; Based on the candidate associated record set, the current vehicle is compared with the historical records item by item to obtain the preliminary evaluation results of vehicle recognition confidence and facial recognition confidence. The initial assessment results are corrected in context and the current vehicle is confirmed to have a record. The joint judgment result is then output.

5. The AR interactive advertising system based on multimedia charging piles as described in claim 4, characterized in that: The real-time collection of spatiotemporal difference data, combined with the joint judgment results, is dynamically coupled to generate audience profiles and strategy constraint sets. The specific steps are as follows: The spatiotemporal difference data includes the current location and time clues, parking space number and surrounding crowd density, on-site lighting and weather conditions, vehicle dwell time and distance of people approaching; The spatiotemporal difference data are matched with the joint judgment results item by item, and interactive opportunity points and potential interference factors are extracted to obtain a key information set; Multi-dimensional profiling constraints are applied to the key information set to generate audience profiles and a set of strategy constraints.

6. The AR interactive advertising system based on multimedia charging piles as described in claim 5, characterized in that: The process of matching the audience profile with the strategy constraint set based on the three-layer semantics of vehicle, crowd, and scene to obtain the material instruction set and display control parameters is as follows: The audience profile and strategy constraint set are classified in parallel, and semantic cross-matching is performed based on the parallel classification results to obtain the advertising theme combination; The advertising theme combination is modified and parameterized for vehicle, target audience and scene adaptability, forming material instruction set and display control parameters.

7. The AR interactive advertising system based on multimedia charging piles as described in claim 6, characterized in that: The process of adapting the material instruction set and display control parameters to brightness and color, and correcting the projection posture to obtain multimodal interaction and behavior commands, involves the following specific steps. Based on the display control parameters, brightness is mapped to illumination and color temperature data to obtain an adaptive brightness value; The projection unit's attitude is corrected using adaptive brightness values ​​to determine the projection angle and position parameters. By combining projection angle and position parameters, the material instruction set is rendered synchronously to generate multimodal interaction and behavior instructions.

8. The AR interactive advertising system based on multimedia charging piles as described in claim 7, characterized in that: The specific steps for executing multimodal interaction and behavioral commands, and recording structured interaction feedback data are as follows. Execute multimodal interaction and behavioral commands to play AR ads, and switch the ads to interactive modes based on the location of the users; Real-time collection of interaction records between people and AR advertisements in interactive mode yields structured interaction feedback data.

9. The AR interactive advertising system based on multimedia charging piles as described in claim 8, characterized in that: The aforementioned adjustment of material priorities for structured interactive feedback data refers to re-ranking the weights based on structured interactive feedback, audience profiles, and strategy constraint sets to form a priority list.

10. The AR interactive advertising system based on multimedia charging piles as described in claim 9, characterized in that: The logic for triggering a chain of advertisements when the same vehicle is detected appearing at different charging stations is as follows: Based on the joint determination results, the occurrence of the same vehicle at different charging piles is correlated in time sequence, cross-site link context is generated, and vehicle and vehicle personnel information is saved to the historical record database. Based on the cross-site link context and combined with the current audience profile and strategy constraints, determine the delivery stage and content for this site, generate updated strategy parameters, and simultaneously distribute them to each terminal.

Images