System and method for retail advertising placement and billing based on physical behavior triggers and real-time bidding

Abstract

The application discloses a retail advertisement putting and charging system and method based on physical behavior triggering and real-time bidding, and belongs to the technical field of intelligent retail advertisement and edge compliance audit. The system is deployed on the edge computing device of a retail place, relies on three multi-modal sensing channels (including shelf wide-angle lens and microphone matrix) of shelves, shopping carriers and cash desks, performs cross verification of visual, weight and structured audio events in a unified synchronous time window, and accurately identifies customer commodity physical taking behavior as a high-priority advertisement triggering source. The edge device performs real-time bidding based on advertisement bidding, estimated conversion rate and terminal weight, and superimposes inventory hard constraints such as forced top placement of near-expiration commodities and forced removal of low-inventory commodities to push preferentially. The system constructs a three-level difference-making charging system of display, purchase and transaction, distinguishes between two types of triggering scenarios of customer self-taking and store clerk manual recommendation, and realizes no repeated deduction. The original audio and video data of the store are immediately destroyed after local analysis and desensitization at the edge, and are not uploaded to the cloud, meeting the personal information compliance requirements; charging structured logs are stored through SHA-256 hash chain and periodic Merkle tree root hash chain, and data isolation is realized in combination with a tenant independent HMAC key. The application solves the pain points of traditional offline retail advertisement triggering inaccuracy, rough charging, high privacy risk and un-auditable data.

Description

Technical Field

[0001] This invention relates to the fields of smart offline retail, edge IoT multimodal sensing, programmatic real-time advertising bidding, conversion effect billing, blockchain cryptographic evidence storage and personal information privacy protection, and in particular to an offline retail advertising placement and billing system and method that integrates physical behavior triggering, hard inventory constraint intervention, edge local decision-making and tenant isolation auditing. Background Technology

[0002] Traditional offline brick-and-mortar retail advertising primarily relies on static posters, fixed electronic price tags, and large screen carousels above the checkout counter. This approach suffers from several critical technical shortcomings: First, ad triggering is severely disconnected from customers' actual purchasing intentions. Existing solutions often use fixed carousels or simple human body sensors, failing to accurately push relevant ads when customers exhibit a strong purchase intention ("picking up a product"), resulting in high invalid exposure rates. Second, the billing model is crude and lacks fairness. The industry generally uses a uniform CPM (cost per mille) deduction, charging the same fee regardless of whether the ad leads to add-to-cart or a sale. Advertiser ROI (return on investment) cannot be quantified and guaranteed, and there is a lack of differentiated pricing capabilities for high-value scenarios such as manual recommendations from store staff. Third, data trust is lacking, and auditing is difficult. Ad exposure, conversion, and billing data are all stored unilaterally by the advertising platform without an immutable third-party verification mechanism. Brands cannot independently verify the authenticity of billing, easily leading to settlement disputes. Furthermore, in a multi-tenant SaaS model, logs from different brands are stored together, posing a serious risk of data leakage. Furthermore, privacy compliance risks are prominent. Existing sensing solutions typically continuously upload raw audio and video streams from store cameras and microphones to the cloud for analysis, violating the Personal Information Protection Law's requirements for minimum collection, localized processing, and prohibition of excessive collection of biometric information. Finally, existing technologies cannot achieve multi-intent signal fusion and inventory linkage. They are incompatible with supplementary intent signals such as staff recommendations, customer AR gaze, and handheld terminal scanning, and cannot dynamically adjust bidding rankings based on product expiration status or inventory levels, leading to wasted recommendation resources or unsold near-expiration products. While existing technologies include scattered solutions such as shelf weighing, visual recognition of customer pickup behavior, real-time bidding for online advertising, Merkle log storage, and edge local data processing, there is still no solution that integrates six core technologies: "dual-mode synchronous time window verification of physical pickup events with multiple sensors for both shelf and cashier," "forced placement of near-expiry products / direct removal of low-inventory products with dual inventory hard constraints," "three-tier supplementary billing system of display-add-to-purchase," and "edge-original audio and video local destruction + tenant-isolated cryptographic storage." This fails to systematically address the technical pain points in offline retail advertising scenarios, such as high false trigger rates, unfair billing, privacy leaks, lack of audit documentation, and insecure multi-tenant data. Summary of the Invention

[0003] This invention relates to the fields of smart offline retail, edge IoT sensing, programmatic real-time advertising bidding, data cryptography storage and personal information privacy protection, and in particular to an offline retail advertising placement and billing system and method that integrates multimodal physical behavior sensing, edge local real-time decision-making, conversion tiered billing, and tenant isolation audit storage.

[0004] Traditional offline brick-and-mortar retail advertising primarily relies on static posters, fixed electronic price tags, and large-screen carousel ads, which suffer from the following unresolved technical problems: ad triggering is disconnected from customers' actual purchasing intentions, lacking real-time physical behavior perception capabilities; the billing model is crude, generally using CPM (cost per mille) for uniform deduction based on exposure, making it impossible to quantify and guarantee advertiser ROI; data trust is lacking, as ad exposure, conversion, and billing data are all stored unilaterally by the advertising platform without an immutable evidence preservation mechanism; centralized cloud processing poses serious privacy and compliance risks, with continuous uploading of raw audio and video to the cloud violating minimum personal information collection requirements; and in multi-tenant scenarios, mixed storage of logs from different brands greatly increases the risk of data leakage. Currently, there is no complete solution that integrates and coordinates six core technologies: cross-verification triggering of dual-mode physical behavior, inventory constraint intervention in bidding ranking, tiered billing for display-add-to-cart-sale differentials, multi-intent signal fusion, edge raw data destruction, and tenant-isolated cryptographic evidence preservation.

[0005] The purpose of this invention is to overcome the above-mentioned defects and provide a retail advertising delivery and billing system and method based on physical behavior triggering and real-time bidding. It relies on edge computing local closed-loop processing to achieve precise intent advertising push, tiered conversion billing, local destruction of privacy data, and cryptographic log storage and auditing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a retail advertising placement and billing system based on physical behavior triggering and real-time bidding, including a cloud management platform, a local edge computing device in the retail store, a shelf area sensing channel, a shopping vehicle sensing channel, a cashier sensing channel, and a recommendation output terminal; the edge computing device integrates a physical verification triggering unit, a bidding allocation unit, a behavior tracking unit, a billing settlement unit, and a billing data storage unit. The cloud management platform is responsible for ad configuration, model distribution, tenant key distribution, Merkle root hash chain storage, and tenant audit verification. Multiple sensing channels collect weight, images, and POS structured data from shelves, shopping carts, and checkout counters. All raw audio and video data is directly destroyed after local parsing on edge devices, without being uploaded to the cloud; only structured numerical logs are output. The physical verification trigger unit sets up two independent dual-sensor cross-verification logics for shelves and checkout counters, limiting a unified synchronization time window and weight matching error threshold. The bidding allocation unit calculates a comprehensive score based on bid, estimated conversion rate, and terminal weight, overlaid with dual forced inventory correction rules: near-expiry placement and low-inventory removal. The behavior tracking unit performs dual verification of product entry into the cart and the entire conversion process at checkout. The billing and settlement unit adopts a differential tiered billing system, differentiating between self-service pickup and manual guidance exposure prices. The billing data storage unit constructs a SHA-256 hash chain and a periodic Merkle tree, with the root hash on the chain, and achieves data isolation based on tenant-specific HMAC keys.

[0007] The technical effects brought about by the core distinguishing technical features of this invention can be reflected in the following aspects (the following quantitative values ​​are exemplary test results in a typical convenience store / supermarket environment, and may vary in different stores due to sensor resolution and customer flow density): 1) Dual-mode synchronous time window for physical retrieval events involving both shelves and checkout counters: This solves the problem of high false trigger rates for single-shelf visual / weighing recognition. Technical results: The false trigger rate for single-shelf recognition is approximately 15%~20%, which can be reduced to approximately 1%~2% after dual-mode cross-verification (example record approximately 1.2%), resulting in a corresponding reduction of approximately 85%~90% in invalid advertising exposure. 2) The dual hard constraint of forced placement of near-expiry products in bidding and direct removal of low-inventory products solves the problem that conventional bidding relies solely on bidding competition, resulting in near-expiry products not receiving exposure; Technical effects: Ad exposure for near-expiry products can be increased by approximately 110%~130% (approximately 127% in the example record), and the turnover days of near-expiry products in corresponding categories in supermarkets can be shortened by approximately 35%~45% (approximately 42% in the example record); 3) A three-tiered billing mechanism that differentiates between trigger sources and conversion levels: This solves the problems of traditional uniform exposure billing and the lack of premium differentiation in manual recommendations by sales associates; Technical effects: The overall ROI of advertisers (attributed to conversion rates) can be increased by an average of about 30%~40% (approximately 37% in the example record), the premium mechanism for exposure of manual recommendations by sales associates can incentivize store sales associates to actively promote brand products, the differential billing avoids double billing, and the number of disputed work orders can be reduced by about 90%~95% (approximately 95% in the example record); 4) Multi-source client low-priority intent fusion to supplement bidding candidates: solves the problem of limited intent coverage scenarios when relying solely on customer take-up behavior to trigger ads; technical effect: the ad trigger scenario coverage can be expanded from "physical take-up only" to about 1.5~1.6 times (the example record is about 160%), and the low-weight downgrade strategy will not interfere with the ranking of ads triggered by high-credibility physical take-up.

[0008] Distinguishing the Boundaries from Prior Applications: The applicant's prior application (application number 2026108930143) only disclosed the hardware sensing and acquisition architecture for shelf vision, shelf weighing, and shopping cart weighing, and only addressed the hardware setup for collecting data on product retrieval and placement. The improvements of this invention are entirely focused on the software linkage logic after sensor data acquisition. The prior application did not disclose all of the following core technical solutions: ① Quantitative judgment rules for cross-verification of synchronous time windows; ② Geometric verification logic for checkout vision + weighed items; ③ Real-time bidding scoring mechanism integrating hard inventory constraints; ④ Tiered billing system for display / add-to-cart / sale compensation; ⑤ Multi-intent fusion strategy for staff recommendations, AR eye tracking, and handheld terminals; ⑥ Privacy scheme for local destruction of edge raw data; ⑦ SHA-256 hash chain + Merkle tree tenant isolation evidence storage and auditing system. The protected objects of the two applications are completely distinct. The hardware is merely used as a data input carrier for this invention and is not considered an innovative improvement point of this invention. There is no overlap in the scope of protection between the two applications, and there is no risk of duplicate authorization.

[0009] Privacy and Security Statement: All raw image and audio data streams generated by the shelves, cashier cameras, and microphone matrix of this system are only temporarily cached in the memory of the store's edge computing device. They are permanently deleted immediately after structured behavior parsing is completed and are not persistently stored or uploaded to the cloud server. All billing log structured fields only record product hashes, time, tenant IDs, and conversion levels. No sensitive personal information such as customer faces, voiceprints, identity phone numbers, or biometrics is collected, stored, or uploaded. Each brand tenant relies on an independently derived HMAC key to isolate the log subtree, preventing them from accessing each other's billing data, thus fully meeting the data compliance and regulatory requirements for offline retail stores.

[0010] The present invention will now be described in detail with reference to the accompanying drawings.

[0011] Figure 1This is a system architecture diagram of an embodiment of the present invention. The system includes an edge computing device (10), a shelf area sensing channel (20), a shopping vehicle sensing channel (30), a checkout sensing channel (40), a recommendation output terminal (50), and a cloud management platform (60). The shelf area sensing channel (20) includes a shelf wide-angle camera (21), a shelf weighing sensor (23), and a shelf microphone matrix (24). The shopping vehicle sensing channel (30) includes a vehicle weighing sensor (22) and a positioning tag (32). The checkout sensing channel (40) includes a checkout camera (43) and a POS system (41). The recommendation output terminal (50) includes a third-party advertising display area (51).

[0012] Figure 2 This is a complete flowchart of the billing, behavior tracking, and cryptographic evidence storage process of this invention. The steps include: physical retrieval event triggering (S201) → bidding allocation and advertising display (S202) → behavior tracking (add to cart S203 / complete transaction S204) → tiered supplementary billing (S205) → generating billing event log (S206) → organizing into hash chain and Merkle tree (S207) → writing to immutable storage (S208) → tenant isolated access (S209).

[0013] Figure 3 This is a flowchart of the real-time bidding and inventory adjustment logic of the present invention. The steps include: receiving physical retrieval events (S301) → querying the candidate ad set (S302) → checking inventory constraints (S303): if near expiration (T_rem / T_total≤0.15), then forcibly placing it at the top (S304); if out of stock (DOS<3), then forcibly excluding it (S305); otherwise, calculating the score (S306) → taking the Top-K (S307) → pushing and displaying (S308).

[0014] Figure 4 This is a schematic diagram of the multi-type intent signal fusion and multi-trigger source differentiated billing unit price process of the present invention. The steps include: detecting client intent signals (S401): near-eye gaze ray (S402), handheld terminal operation (S403), shelf microphone voice (S404) → intent signal as downgrade candidate to participate in bidding (S405) → physical take-up event trigger (S406) → personnel recommendation event trigger (S407) → apply different display unit prices according to trigger source (S408) → add to purchase confirmation upgrade billing (S409) → transaction confirmation upgrade billing (S410).

[0015] Example 1: Shelf Mode Physical Behavior Trigger + Inventory Bidding Intervention. In a typical convenience store test environment (shelf wide-angle camera 1080P@30fps, strain gauge weighing resolution ±5g, main test SKUs were 330ml canned carbonated beverages and 500ml...),... (PET bottled water) A customer enters the beverage shelf area and reaches for a can of cola. The shelf's wide-angle camera captures the product's outline as it moves out of the shelf, and the shelf's weighing sensor simultaneously detects a decrease in weight in the corresponding shelf area. The difference in the timestamps of the two signals is 32ms, falling within the 200-500ms synchronization time window. The weight reduction is within 6% of the cola's single-unit weight, meeting all the shelf mode's judgment conditions, and generating the first physical take-away event. Simultaneously, the shelf's microphone matrix detects the customer saying "Take a bottle of cola" with the sound source pointing towards the cola display area. The system increases the confidence level of this physical take-away event to 1.15 times the enhancement value for the bidding allocation unit's reference. The edge device searches all advertising campaigns linked to the cola scenario. Among them, the potato chip product has only 10% of its total shelf life remaining, meeting the near-expiry top placement constraint, and is directly ranked first and pushed to the electronic shelf label terminal to display the near-expiry special offer advertisement. After the advertisement is displayed, the system deducts the basic exposure fee based on the self-take-away display unit price.

[0016] Example 2: Staff Recommendation Trigger + Three-Tier Billing. Staff wearing smart bracelets with integrated microphones and motion sensors proactively recommend potato chips to hesitant customers and read out preset recommendation keywords. The bracelet detects wrist movement towards the customer within a preset range of 10cm to 60cm, and simultaneously, if there is no matching weight change in the shelf or shopping cart, a valid staff recommendation event is determined. The system marks the recommendation source and applies a higher staff recommendation display price when advertising is shown. When the customer places the potato chips into the shopping basket, the weighing device simultaneously detects the corresponding weight increase, the shelf visually recognizes the ROI of placing the potato chips into the shopping basket, generates an add-to-cart confirmation, and the system only deducts the add-to-cart tier price difference. When the customer completes checkout and scans the code, a transaction confirmation is generated, and the system deducts the transaction tier price difference. There is no full double billing throughout the entire process.

[0017] Example 3: Low-weight supplementary bidding for multiple types of client intent. When a customer wearing a near-eye AR display terminal keeps their gaze on the beer shelf area for 6 seconds, reaching the T_gaze threshold, a low-priority gaze intent candidate is generated; when a customer uses a handheld barcode scanner to scan the beer barcode, a barcode scanning intent candidate is generated; when a customer says "How about this?" in front of the shelf, the shelf microphone matrix detects the keyword and generates a voice intent candidate; these three types of intent ads only participate in bidding when there is no physical event of a customer taking the product, and their overall score is lower than that of ads triggered by physical behavior, so they will not take over the display position of high-priority ads.

[0018] Example 4: Billing Log Hash Chain + Merkle Tree Tenant Isolation Audit. For each billing record generated by a store edge device, an independent SHA-256 hash is generated and concatenated to form a hash chain. Every 100 accumulated logs automatically construct a Merkle tree, generating a unique root hash that is automatically uploaded to the cloud consortium blockchain for solidification. A food brand inputs its own tenant key-derived HMAC credentials via the Java SDK, retrieving only all billing log subtrees and Merkle verification paths for its own brand. Offline verification confirms the logs are complete and unaltered, and the exported audit package complies with SOC2 and ISO27001 enterprise audit standards.

[0019] Example 5: Physical Behavior Triggering at the Checkout + Inventory Bidding Intervention. A customer picks up a bag of potato chips near the checkout counter, but the POS system does not immediately scan it (it remains silent). The checkout's local vision device performs geometric verification on the potato chip outline, outputting a visual confidence score Conf_v = 0.92, exceeding the preset visual confidence score threshold of 0.85. Simultaneously, the customer places the potato chips into a shopping bag (vehicle), and the vehicle's weight sensor detects the weight increase matching the weight of a single bag of potato chips, outputting a weight confidence score Conf_w = 0.94, exceeding the preset weight confidence score threshold of 0.8. Within the same synchronous time window, both Conf_v and Conf_w exceed their corresponding thresholds, and the system determines this as a second physical retrieval event, triggering the bidding process for potato chip-related advertisements (such as beverages).

[0020] Instructions for setting key thresholds: The parameters mentioned in this manual, such as "preset visual confidence threshold," "preset weight confidence threshold," and "symmetry ≥ 0.85," are exemplary settings based on product packaging characteristics. In practical applications, these thresholds can be pre-stored in the SKU database through offline calibration, based on the shape, material, weight, and other characteristics of different SKUs, to adapt to the detection needs of different products.

[0021] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. All equivalent substitutions, simple modifications, and improvements made within the inventive concept and protection scope of the present invention are covered within the protection scope of the present invention.

Claims

1. A retail advertising placement and billing system based on physical behavior triggering and real-time bidding, characterized in that, include: The edge computing device is deployed in the retail space, and the shelf area sensing channel, shopping vehicle sensing channel, cashier sensing channel and recommendation output terminal are respectively connected to the edge computing device. The shelf area sensing channel includes a visual acquisition device (21) deployed in the merchandise display area, a shelf load-bearing sensing device (23) and a fixedly installed microphone matrix (24). It is configured to output a shelf area image frame sequence, a load change sequence and an audio signal sequence. After the audio signal sequence is processed by edge beamforming and trigger keyword detection, only structured audio events (including the direction of arrival of the sound source DOA and trigger keyword markers) are output. The original audio stream is terminated locally at the edge and is not uploaded to the outside. The shopping vehicle sensing channel includes a vehicle weight sensing device (22) integrated into the shopping vehicle and a positioning tag (32). It is configured to output a vehicle weight change sequence and location information. The cashier sensing channel includes a cashier local vision device (43) and a POS interface (41). It is configured to output a cashier area image frame sequence and a stream of scanned merchandise items. The edge computing device is configured with: The physical verification trigger unit, configured with a unified synchronization time window ΔT∈[200ms,500ms], is configured to output physical retrieval events and lock the corresponding product SKUs through two independent modes: Shelf mode: Within the same synchronization time window, when the visual acquisition device detects that the outline of the product has been completely removed from the shelf display surface, and the shelf load-bearing sequence detects a weight reduction ΔW in the corresponding load-bearing area, and the error between ΔW and the weight of a single product m_item does not exceed ±8%, it is determined as a product being taken and the first physical take event is output; furthermore, when the microphone matrix outputs a trigger keyword tag (such as "take a bottle") bound to the product within the same synchronization time window and the DOA falls within the normal sector of the product display surface, the confidence of the physical take event is increased to an enhancement value of 1.1 to 1.2 times for subsequent bidding weighting. Cashier mode: If the POS interface returns a valid scanned item containing the product SKU within the heartbeat timeout threshold, the second physical retrieval event is output directly. If the POS channel is silent and has no valid data, then within the same synchronization time window, the local vision device of the cashier performs item geometry verification on the target outline of the cashier ROI and outputs visual confidence Conf_v. At the same time, the vehicle weight sensor detects the weight increment of the goods and outputs Conf_w. Only when Conf_v exceeds the preset visual confidence threshold and Conf_w exceeds the preset weight confidence threshold, the second physical take-up event is output. The bidding allocation unit, in response to the physical take-up event, queries the set of brand advertising campaigns that set the triggering product as the trigger condition to obtain a candidate ad set, and calculates a comprehensive bidding score for each candidate ad according to the following formula: Score = Bid ​​× (pCVR)^α × ω_t If the physical takeaway event carries the aforementioned enhancement value, then the overall bidding score is multiplied by that enhancement value; Where Bid is the single recommendation bid, pCVR is the estimated add-to-cart conversion rate output by the edge-side lightweight model, α∈[0.1,2.0] is the adjustable weight coefficient, and ω_t∈[0.5,1.5] is the terminal weight factor of the recommendation output terminal; the bidding allocation unit also has an inventory constraint correction module, which is configured to: read the remaining shelf life T_rem, total shelf life T_total, and inventory available days DOS of the corresponding products of the candidate advertisement in real time; when T_rem ≤ 0.15×T_total, the candidate advertisement is set to the highest near-expiration priority and forcibly displayed at the top; when DOS < 3, the score of the candidate advertisement is directly set to zero and excluded from the current round of sorting; otherwise, the score is calculated according to the formula and sorted in descending order, and the Top-K is taken to generate rendering instructions and push them to the recommendation output terminal; The behavior tracking unit is configured to generate a purchase confirmation when the advertised product is placed in the shopping cart after the advertisement is displayed, through dual verification by the visual acquisition device in the shelf area and the weight sensor device in the shopping cart, and to generate a transaction confirmation by matching the session SKU through the POS interface; if the product is detected to be taken out of the cart within the synchronization time window and there is no settlement record, the purchase confirmation is cancelled. The billing and settlement unit is configured to perform tiered supplementary billing based on the trigger source and tracking result: the display level is billed based on the self-taken display unit price or the personnel-recommended display unit price, and the personnel-recommended display unit price is greater than the self-taken display unit price; the add-on level only deducts the difference between the add-on level and the display level after the add-on is confirmed; the transaction level only deducts the difference between the transaction level and the add-on level after the transaction is confirmed.

2. The system according to claim 1, characterized in that, After the behavior tracking unit confirms that the product has been placed into the vehicle, the billing and settlement unit immediately locks the settlement unit price of the advertised product and generates a temporary pending payment order. When the behavior tracking unit detects that the product has been taken out of the vehicle within the synchronization time window and no POS settlement record has been generated before the end of the session, it automatically unlocks the price, cancels the pending payment order, and cancels the corresponding level billing difference.

3. The system according to claim 1, characterized in that, The item geometry verification performed by the local vision device at the checkout counter includes an adaptive parallel verification pipeline based on packaging type. Multiple shape verifiers are called in parallel for the target circumscribed contour within the checkout counter's ROI. Each verifier corresponds to a type of product packaging and has its own unique geometric constraints. A contour is considered passed if it satisfies all constraints of any one type of verifier. The shape verifiers include at least bottle-shaped, can-shaped, and box-shaped verifiers, each with the following constraints: Bottle-shaped verifier: The ratio of the long side to the short side of the circumscribed rectangle ∈ [3.5, 7.0]; The width sequence contraction point is identified as the neckline by scanning from top to bottom along the contour's principal axis; the length of the contour below the neckline accounts for the proportion of the overall contour ∈ [0.20, 0.40]; The vertical projection symmetry of the left and right edges of the contour is ≥0.85; Can-shaped verifier: The ratio of the long side to the short side of the circumscribed rectangle ∈ [1.0, ... 2.5]; The difference in horizontal width between the top and bottom of the contour is ≤ 10% of the maximum width of the contour; The overall symmetry of the contour is ≥ 0.85; Box-shaped verifier: The ratio of the area of ​​the minimum bounding rectangle of the target contour to the actual area of ​​the contour is ∈ [0.85, 1.0]; The included angle of the four vertex corners of the contour is ∈ [80°, 100°]; The parallelism of the two pairs of opposite sides of the contour is ≥ 0.

85.

4. The system according to claim 3, characterized in that, The verification pipeline has a built-in packaging classification subunit. When the target outline enters the checkout ROI, it first uses the outline aspect ratio to coarsely classify and match the packaging category, and then only calls the corresponding category's dedicated shape verifier to perform fine verification, reducing redundant visual calculations and lowering edge inference latency.

5. The system according to claim 1, characterized in that, The edge computing device is also equipped with a personnel recommendation event recognition module, configured to recognize valid personnel recommendation events: Store clerk wearable smart device: microphone audio signal matches pre-recorded brand recommendation words, while built-in motion sensors detect wrist displacement vectors pointing towards the customer area with displacement amplitude within a preset range of 10cm to 60cm, and no corresponding product weight change occurs in the shelf and carrier weight sequence within the same synchronous time window; all three conditions must be met for validity. Shopping guide robot: onboard vision continuously monitors customer dwell time exceeding a preset threshold, and robot voice broadcasts and screen-displayed advertising content are bound and matched with the same advertised product SKU. When a valid personnel recommendation event is recognized, the system marks an independent recommendation source identifier in the billing log, and the billing and settlement unit automatically applies the personnel recommendation display unit price described in claim 1.

6. The system according to claim 1, characterized in that, The edge computing device is also configured with a client intent fusion module, which is configured to receive low-priority intent candidate signals: near-eye display terminal signals: the terminal outputs eye-tracking sampling gaze ray, the ray coordinate system is uniformly registered with the visual coordinate system of the shelf area perception channel, and the cumulative gaze dwell time in the same shelf target area is ≥T_gaze, generating gaze intent candidates; Handheld smart terminal signals: The terminal reports product barcode decoding records, NFC product touch records, and product picking direction data inferred from the IMU sensor to generate information collection intent candidates; Shelf microphone matrix signal: The microphone matrix output is bound to the trigger keyword tag of the triggering product and the DOA falls in the display sector of the product, generating voice intent candidates; The client intent signal is only expanded to the bidding candidate ad set when the physical verification triggering unit does not identify a valid physical take event. The candidate ads generated by the client intent have a built-in fixed attenuation weight of 0.3 to 0.5, and their overall bidding score is lower than that of ads triggered by physical take events.

7. The system according to claim 1, characterized in that, The raw images and audio data generated by the shelf area sensing channel, the local vision device at the checkout counter, and the microphone matrix are processed within the edge computing device. The raw audio and video data are not uploaded externally; only structured trigger event logs and rendering receipts are output. The pCVR lightweight model equipped in the bidding allocation unit has less than 1M parameters, and the total time for a single forward inference is ≤50ms. The model only receives structured data such as store inventory, historical transactions, and days until expiration as input features, and does not introduce customer facial, identity, or voiceprint biometric data throughout the process, thus meeting personal information compliance requirements.

8. The system according to claim 1, characterized in that, The edge computing device is also equipped with a billing data storage unit, configured to: collect a complete structured field group for each billing event, with fields containing only tenant ID, timestamp, product SKU hash, trigger type, billing level, and referral source identifier, without any customer facial, voiceprint, or biometric information; encapsulate each billing event into a SHA-256 hash chain entry, automatically construct a Merkle tree and generate a unique root hash every 100 accumulated logs or every 10 minutes, and synchronously upload the root hash to the cloud consortium blockchain to complete tamper-proof storage; allocate an HMAC signature field derived from an independent key to each tenant, and any tenant can only access its own exclusive log subtree and integrity verification path through the HMAC credentials generated by its own key.

9. The system according to claim 1, characterized in that, It also includes a cloud management platform, configured to send advertising campaign configurations, bidding parameter thresholds, tenant-specific HMAC keys, and pCVR lightweight inference models with less than 1M parameters to edge computing devices, and to receive Merkle tree root hashes periodically uploaded by edge computing devices to synchronize to the consortium blockchain for evidence storage. The cloud-based open audit and verification interface allows each tenant to independently verify the integrity of its own billing logs. The cloud management platform exposes standardized local log reporting endpoints and is equipped with SDKs for multiple languages, including TypeScript, Python, Go, and Java. The system front-end provides an embedded audit query component. When the component is loaded, it carries a tenant-specific token to achieve contextual data isolation, allowing tenants to only load and view their own billing audit data.

10. A method for retail advertising placement and billing based on physical behavior triggering and real-time bidding, characterized in that, Applied to the system of any one of claims 1 to 9, comprising the following steps: S1: Local edge-side collection of multi-channel sensor data from shelves and checkout counters, cross-validation within a unified synchronization time window, differentiation between shelf mode and checkout counter mode to output standardized physical retrieval events, and locking the corresponding triggering product SKU; original image and audio data are destroyed locally and not uploaded to the cloud. S2. Edge devices respond to physical retrieval events to retrieve corresponding brand advertisements and generate a candidate advertisement set. The base bid score for each advertisement is calculated using the Score formula. The inventory constraint is forcibly corrected: near-expiry products are placed at the top, low-inventory products are removed, and the remaining advertisements are sorted in descending order to generate the Top-K advertisement rendering command and send it to the recommendation terminal. S3. The behavior tracking unit continuously monitors the flow of advertised products, double-verifies the placement of products into the shopping cart to generate a confirmation of adding to cart; matches the POS settlement record to generate a transaction confirmation; if the product is removed but not settled, the addition to cart mark is canceled. S4. The billing and settlement unit performs differential tiered billing based on the trigger source (self-pickup / personal recommendation) and product conversion level (display / add to cart / complete transaction), generating a structured attribution log with product hash and recommendation source identifier; the log does not contain customer biometric privacy information; S5, the billing data storage unit encapsulates a single log entry into a SHA-256 hash chain, constructs a Merkle tree at fixed intervals, and synchronously uploads the root hash to the cloud consortium blockchain to complete tamper-proof storage; data isolation is achieved based on the tenant's HMAC key, and each tenant independently obtains its own log verification path to complete compliance audit.

Images