An intelligent cargo container commodity change detection method, system and device based on normalized state consistency constraint and asymmetric judgment mechanism

Abstract

This invention discloses a method, system, and device for detecting changes in goods in an intelligent vending machine based on normalized state consistency constraints and an asymmetric decision mechanism. This invention uses a standard 1080P camera to capture image sequences before and after the vending machine door opens and closes, constructing a system containing global appearance features. G Sealing structure characteristics C and contents liquid level characteristics L 3D normalized state vector S = {G, C, L} The invention utilizes a weighted adaptive asymmetric difference function to calculate state change values ​​and combines multi-frame median fusion technology to eliminate environmental noise interference. When the total change value or the change in key feature components exceeds a set threshold, it is determined that a genuine change has occurred in the product. This invention effectively solves the problems of confusion between highly similar products and covert cheating behaviors such as "returning the drink after use," significantly reducing hardware costs while greatly improving the accuracy and security of unmanned retail checkout.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, pattern recognition, and smart retail technology, specifically to a method, system, and device for detecting changes in goods by constructing a multi-dimensional state vector of a product instance and performing differential calculations in a unified normalized space, while introducing asymmetric decision constraints. Background Technology

[0002] Current smart vending machines primarily rely on gravity sensing, RFID, or purely visual object detection solutions. Gravity sensing solutions are costly and cannot detect items that have been adulterated with water and then returned to their original position. RFID is also costly and prone to damage. Existing purely visual solutions often rely solely on changes in the spatial position of objects. When a product is removed, consumed, and then returned to its original location, the system often misses detection because the appearance and position remain almost unchanged, leading to checkout errors and food safety risks. Furthermore, misjudging highly similar SKUs from the same brand under complex lighting conditions is another pain point in the industry. Summary of the Invention

[0003] The purpose of this invention is to propose a change detection scheme based on the consistency constraint of product instance state. The core innovation lies in: 1. Definition of State Vector: The recognition of objects is upgraded from a single "classification" to "state monitoring", and a three-dimensional vector containing appearance (G), structure (C) and content (L) is constructed. 2. Asymmetric judgment mechanism: Instead of relying solely on the total score, it sets mandatory constraints on the integrity of the seal and changes in liquid level. Even if the overall appearance is extremely similar, once the structural or content features change, it is judged as a change. 3. Multi-frame robust fusion: Through temporal median filtering, dynamic lighting and occlusion noise are effectively filtered. Attached Figure Description

[0004] Figure 1 This is a schematic diagram of the overall process of a smart vending machine product change detection method based on normalized state consistency constraints and asymmetric judgment mechanism according to the present invention.

[0005] Figure 2 This is a flowchart illustrating the process of extracting appearance features, sealing features, and liquid level features from a product instance and constructing a normalized state vector in this invention.

[0006] Figure 3 This is a schematic diagram of the process for outputting product change results based on state difference values, multi-frame fusion, and asymmetric judgment conditions in this invention. Detailed Implementation

[0007] The present invention will be further described below with reference to an embodiment. This embodiment is based on a training set containing 20,000 high-dimensional SKU data items and 100,000 real-world product images to build a model. Example 1: Anti-cheating detection When a user takes out a bottle of high-end liquor, dilutes it with water, and puts it back, in step S3, the system detects the seal integrity feature of this instance. C The value dropped from 0.98 to 0.12 (anti-counterfeiting ring broke), indicating a change in liquid level. L A slight offset of ±5% occurs. In step S7, even with similar appearance... d'(G) Extreme 3, high score but total score not up to standard, but due to ΔC>Tc The asymmetric judgment threshold was triggered, and the system accurately output the product as "Taken", effectively preventing cheating. Comparison of experimental data:

Claims

1. A method for detecting changes in goods in an intelligent vending machine based on normalized state consistency constraints and an asymmetric decision mechanism, characterized in that, Includes the following steps: S1. Use a regular 1080P camera deployed inside the container to capture images before the container door is opened and a series of multiple consecutive images after the door is closed. S2. Perform partitioned spatial alignment and brightness uniformity preprocessing on the acquired images to eliminate interference caused by container mechanical displacement and drastic changes in lighting. S3. For each product instance in the image, extract its global appearance feature vector using a hierarchical convolutional neural network. G High-resolution feature branching was used to extract the probability features of the bottle cap seal integrity. C Normalized liquid level height features were extracted using edge energy detection. L ; S4. Combine the features to construct a normalized state vector for the product instance. S={G,C,L} ; S5. Match weight coefficients according to the SKU category to which the product instance belongs. W={w 1 ,w 2 ,w 3 } And calculate the asymmetric state difference before and after the door is opened. D : D=w 1 ·d'(G)+w 2 ·ΔC+w 3 ·ΔL in, d'(G) The normalized distance between appearance features. ΔC This represents the change in seal integrity. ΔL This represents the absolute value of the change in liquid level. S6. Perform median fusion on the difference value sequence calculated from multiple frames of images to obtain the final state difference value. Dfinal ; S7. List of output changes based on asymmetric decision logic: When the condition is met... Dfinal>T or the change in a single key feature ΔC> Tc or ΔL>Tl When a change occurs, the product instance is determined to have changed and is added to the change list.

2. The method according to claim 1, characterized in that, The liquid level height feature described in step S3 L The extraction method includes: correcting the posture of the bottle instance through principal component analysis (PCA), extracting the normalized bottle height, and locating the strongest horizontal edge within the search window as the liquid level height.

3. The method according to claim 1, characterized in that, In the normalized state vector described in step S4, each component is normalized to the [0, 1] interval by Min-Max normalization to eliminate the dimensional differences between different physical dimensions.

4. The method according to claim 1, characterized in that, The weighting coefficients mentioned in step S5 w 2 and w 3 Dynamic adjustments will be made based on the food safety sensitivity of the products; for high-value products such as liquor, the level of sensitivity will be increased. w 2 and w 3 The percentage.

5. A smart vending machine merchandise change detection system that performs the method according to any one of claims 1-4, characterized in that, include: The system includes an edge image acquisition module, an image preprocessing module, a hierarchical feature extraction module, a state vector construction module, an asymmetric difference calculation module, a multi-frame fusion module, and a change determination module.

Images