A fake cigarette identification method and system based on AR glasses

Abstract

The application relates to a fake cigarette identification method and system based on AR glasses, and relates to the field of fake cigarette identification. The method comprises the following steps: acquiring a real-time image of a to-be-identified cigarette case through a camera of the AR glasses in response to an identification starting signal; comparing the real-time image with a real cigarette standard image to obtain a real-time detection surface; screening real cigarette standard features in the real cigarette standard features that are consistent with the real-time detection surface and key region positioning, and defining the real cigarette standard features as real-time real cigarette standard features; extracting image features of the cigarette case in the real-time image according to the key region positioning, and comparing the image features with the real-time real cigarette standard features to obtain abnormal features; if there are no abnormal features, outputting a preset surface changing operation signal; if there are abnormal features, generating an abnormal correction operation signal based on the abnormal features and outputting the abnormal correction operation signal; and if all the real-time detection surfaces do not have abnormal features, outputting a real cigarette identification signal. The application has the effect of solving the problem that the use scene is limited and on-site verification cannot be flexibly performed.

Description

Technical Field

[0001] This invention relates to the field of counterfeit cigarette identification, and in particular to a method and system for counterfeit cigarette identification based on AR glasses. Background Technology

[0002] In recent years, counterfeit and substandard tobacco products have become widespread in the tobacco industry, with large quantities of counterfeit and substandard tobacco products entering the market, seriously affecting the national economy, market order, and public health.

[0003] Among related technologies, machine vision technology already exists to address the low efficiency of manual experience in identifying counterfeit cigarettes. This technology uses high-definition industrial cameras to capture multi-angle images of cigarette cartons, individual packs, and individual cigarettes, performing preprocessing such as grayscale conversion, noise reduction, and distortion correction. Next, the system extracts key visual features such as printing texture, color difference, hot stamping patterns, micro-characters, glue marks, and embossed features. These extracted features are then compared and matched against a database of genuine product images, using algorithms to identify anomalies such as texture deviations, color errors, and manufacturing defects. Finally, a threshold model is used to classify the product as genuine or counterfeit, quickly outputting the identification results. This allows for batch automatic screening, replacing manual identification of counterfeit packaging features.

[0004] Regarding the aforementioned technologies, traditional fixed machine vision requires a fixed camera position, multi-angle image capture, and neatly arranged samples. Due to limitations in the usage scenarios, it cannot flexibly verify on-site, and it cannot promptly handle situations such as creases in cigarette boxes, making it difficult to combine practical details for judgment. Summary of the Invention

[0005] To achieve human-computer interaction, this invention provides a method and system for identifying fake cigarettes based on AR glasses.

[0006] In a first aspect, the present invention provides a method for identifying fake cigarettes based on AR glasses, employing the following technical solution: A method for identifying fake cigarettes based on AR glasses includes: Step 1: In response to the recognition activation signal, acquire a real-time image of the cigarette box to be recognized through the camera of the AR glasses; Step 2: Compare the real-time image with all standard images of genuine cigarettes to obtain the real-time detection surface of the cigarette box to be identified; Step 3: Based on the real-time detection surface, filter all preset real cigarette standard features that match the real-time detection surface and locate key areas, and define the real cigarette standard feature as the real-time real cigarette standard feature. Step 4: Extract image features of the cigarette box from the real-time image according to the key area location, and compare the image features with the real-time standard features of real cigarettes to obtain abnormal features; Step 5: If no abnormal features are found, output the preset face-changing operation signal; Step 6: If abnormal features exist, generate and output an abnormality correction operation signal based on the abnormal features; The abnormal features include appearance abnormal features and placement abnormal features, and also include a method for determining whether the abnormal feature is a preset appearance abnormal feature or a preset placement abnormal feature, the method including: Step 60: Filter the regular standard features in the real-time real cigarette standard features according to the preset rule size, and define the image features corresponding to the regular standard features as regular image features; Step 61: Compare the regular image features with the regular standard features to determine other features besides the regular standard features; Step 62: When other features exist, remove them from the regular image features to obtain the actual regular image features; Step 63: Compare and calculate the placement information of the real-time image by comparing the contour lines of the standard features of the rules with the contour lines of the actual regular image features; Step 64: When other features are not present, output the preset orientation information as the placement information; Step 65: When the placement information is not the correct placement information, output the placement information as a placement anomaly feature; Step 66: When the placement information is upright, compare it with the image features one by one according to the preset appearance problem features to obtain the image features that match the appearance problem features. Define the image features as problem image features and output the matching appearance problem features as appearance anomaly features. The appearance problem features include wrinkle features, missing features and occlusion features. Step 7: Upon receiving the preset abnormal maintenance completion signal, reacquire the real-time image and re-execute steps 2 to 66; Step 8: If no abnormal features are found on any of the real-time detection surfaces, then output a genuine smoke identification signal.

[0007] By adopting the above technical solution, the real-time detection surface is determined by comparing the real-time image of the cigarette box with the standard image of genuine cigarettes. Image features are extracted according to key areas and compared with standard features to obtain abnormal features. Furthermore, regular standard features are filtered using regular dimensions, and the regular image features are compared with the regular standard features to determine whether there are other features. This distinguishes between appearance abnormalities and placement abnormalities, solving the problem that traditional fixed machine vision cannot flexibly verify on-site. It effectively distinguishes between the authenticity of the cigarette box itself and the recognition interference caused by improper placement, achieving accurate recognition of cigarette box features and realizing the effect of human-computer interactive counterfeit cigarette recognition, thus improving the accuracy and refinement of counterfeit cigarette recognition.

[0008] Optionally, it also includes a method for generating and outputting an anomaly correction operation signal based on the anomaly feature if the anomaly feature is an appearance anomaly feature, the method comprising: Step 660: When the abnormal feature is a wrinkle feature, the abnormal feature is parsed to obtain the missing region and the wrinkle stacking region. Step 661: Form a paving direction based on the folded stacked area and the missing area, wherein the paving direction is from the center of the folded stacked area toward the center of the missing area; Step 662: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed by assigning a preset wrinkle mark to the contour of the area where the abnormal feature is located and a mark arrow corresponding to the flattening direction. Step 663: When the abnormal feature is an occlusion feature, identify the occlusion region where the occlusion feature is located in the real-time image, and extract the contour of the occlusion region to obtain the occlusion boundary. Step 664: Based on the location of the key area corresponding to the obstruction boundary and the real-time real smoke standard features, generate the movement direction to remove the obstruction; Step 665: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed after assigning a preset occlusion mark to the occlusion area and a marker arrow corresponding to the movement direction.

[0009] By adopting the above technical solution, missing areas and fold stacking areas are obtained by analyzing the fold features, occlusion areas are identified and boundaries are extracted by occlusion features, and specific processing operation methods are displayed in AR glasses. This solves the problem that users do not know how to properly arrange or remove obstructions when faced with folds or partial occlusions on the surface of cigarette boxes, thus improving the efficiency of successful recognition.

[0010] Optionally, it also includes a method for generating and outputting an anomaly correction operation signal based on the anomaly feature if the anomaly feature is a placement anomaly feature, the method comprising: Step 650: Adjust the size of the preset real cigarette model according to the placement information to obtain the adjusted real cigarette model; Step 651: Deconstruct the placement information to obtain the rotation angle deviation and the cigarette box area; Step 652: Analyze the real-time image to obtain the user's gripping hand; Step 653: Simulate the rotation angle deviation according to the user's grip to obtain the corrected rotation direction; Step 654: Generate a dynamic adjustment illustration video based on the corrected rotation direction and the adjusted real cigarette model. The dynamic adjustment illustration video demonstrates the adjustment of the real cigarette model by rotating it according to the corrected rotation direction. Step 655: Form a display area for a dynamically adjusted schematic video based on the cigarette box area, wherein the display area and the cigarette box area do not interfere with each other; Step 656: Generate a dynamically adjusted schematic video based on the real-time image and display it in the display area of ​​the AR glasses.

[0011] By adopting the above technical solution, the size of the real cigarette model is adjusted according to the placement information to obtain the adjusted real cigarette model. The placement information is disassembled to obtain the rotation angle deviation and the cigarette box area. The real-time image is analyzed to obtain the user's holding hand, and a dynamic adjustment illustration video is generated and played in the AR glasses display area. This solves the problem of recognition failure when the cigarette box is not placed at the correct angle. It provides a personalized solution based on the user's actual holding method, reduces the difficulty of operation, and improves the recognition efficiency.

[0012] Optionally, before generating a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real smoke model, the following steps are also included: Step 6540: Compare the cigarette box area with the preset standard cigarette box area to obtain the cigarette box size multiple; Step 6541: Calculate the distance between the cigarette box and the eye based on the multiple of the cigarette box size; Step 6542: If the distance between the cigarette box and the eyes is greater than the preset comfortable distance, then generate a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real cigarette model; Step 6543: If the distance between the cigarette pack and the eyes is less than the comfortable distance, output the preset cigarette pack moving away signal and receive the moving away completion signal; Step 6544: Upon receiving the signal indicating that the distance has been cleared, a dynamic adjustment illustration video is generated based on the correction of the rotation direction and the adjustment of the real smoke model.

[0013] By adopting the above technical solution, the size ratio of the cigarette box area is obtained by comparing it with the standard area of ​​the cigarette box, and the distance between the cigarette box and the eyes is calculated. Based on whether it is within the comfortable distance range, it is determined whether a dynamic adjustment illustration video needs to be generated and adjusted. This solves the problem that the content displayed by AR glasses is not coordinated with the user's field of vision and is difficult to see when the cigarette box is too close to the eyes, thus achieving the effect of improving the comfort of human-computer interaction and the accuracy of operation.

[0014] Optionally, methods for outputting a preset cigarette box distance from the signal include: Step 65430: If the distance between the cigarette box and the eye is less than the comfortable distance, adjust the angle of the real cigarette model according to the rotation angle deviation to obtain a model that is far away from the real cigarette model; Step 65431: Develop the model's movement trajectory based on the distance between the cigarette box and the eye; Step 65432: Based on the changes in the moving away from the real cigarette model and the adjustment of the real cigarette model according to the preset change duration, a change process from adjusting the real cigarette model to moving away from the real cigarette model is formed. The change process is placed on the model movement trajectory to form a dynamic video of the cigarette box moving away, and the dynamic video of the cigarette box moving away is generated in the display interface of the AR glasses. Step 65433: Update the distance of the cigarette box from the dynamic video in real time based on the placement information in the real-time image and continuously obtain the distance of the cigarette box from the eye, and define the distance of the cigarette box from the eye as the distance away; Step 65434: When the distance away is equal to the comfort distance, generate a static image of the real smoke model and display it on the AR glasses' display interface. In the static image, the real smoke model is in the upright position.

[0015] By adopting the above technical solution, when the distance between the cigarette pack and the eye is less than the comfortable distance, the real cigarette model is adjusted according to the rotation angle deviation to obtain a model that is further away from the real cigarette model. The model's movement trajectory is formed according to the distance from the eye, and the dynamic video of the process of moving away from the real cigarette model is displayed in the AR glasses. When the distance away is equal to the comfortable distance, a static image of the real cigarette model is generated. This solves the problem that users have difficulty judging how far is appropriate, and achieves the effect of guiding users to correctly adjust the cigarette pack to the optimal recognition distance through dynamic video demonstration.

[0016] Optionally, it also includes an optimization method for adjusting the position of the cigarette box to be identified based on its distance from the real cigarette model, the method comprising: Step 65435: Develop the predicted trajectory of the cigarette box moving away from the moving video. Step 65436: Generate the actual cigarette box's trajectory based on the real-time image; Step 65437: When the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide, a preset stop movement command is issued, and the distance between the cigarette box and the eye is determined at this time. This distance between the cigarette box and the eye is defined as the obstacle avoidance distance. Step 65438: Update the comfort distance according to the obstacle avoidance distance and regenerate the still image.

[0017] By adopting the above technical solution, the expected moving trajectory is formed based on the dynamic video of the cigarette box moving away, and the actual moving distance is formed by the real-time image. When the two do not coincide, a stop movement command is issued, and the obstacle avoidance distance is determined at this time. After updating the comfortable distance according to the obstacle avoidance distance, a static image is regenerated, which solves the problem that users cannot complete the distance adjustment according to the preset path when they encounter obstacles during the movement of the cigarette box.

[0018] Optionally, an optimization method is also included when the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide, the method including: Step 654370: Determine the background image other than the real-time image based on the real-time image corresponding to the obstacle avoidance distance; Step 654371: Perform object feature analysis based on the background image to obtain obstacle categories other than the preset human arm features; Step 654372: Determine the occlusion range of obstacles in the background image based on the obstacle category; Step 654373: Calculate the optimal obstacle avoidance path based on the real-time image and the obstacle occlusion range; Step 654374: Update the model's movement trajectory based on the optimal obstacle avoidance path. The starting point for the update is the point on the model's movement trajectory corresponding to the obstacle avoidance distance. Step 654375: Perform steps 65432 to 65434 according to the updated model movement trajectory.

[0019] By adopting the above technical solution, the background image is determined based on the real-time image corresponding to the obstacle avoidance distance, the types and occlusion ranges of obstacles other than the user's arms are analyzed, and the optimal obstacle avoidance path is calculated. This solves the problem that simple stop commands cannot help users bypass obstacles and complete the correct position adjustment, and achieves the effect of intelligent obstacle avoidance path planning.

[0020] Optionally, it also includes a method for generating obstacle cues, the method comprising: Step 654376: Determine the corresponding display area of ​​the obstacle in the AR glasses display interface based on the occlusion range of the obstacle in the background image; Step 654377: Overlay a preset obstacle mark on the corresponding display area, the obstacle mark being a highlighted border; Step 654378: Generate text prompts corresponding to the obstacle category and display them on the edge area of ​​the obstacle occlusion range on the AR glasses display interface; Step 654379: When the occlusion range of an obstacle changes, update the position of the obstacle marker and the text prompt information in real time.

[0021] By adopting the above technical solution, the area corresponding to the obstacle in the AR glasses display interface is determined based on the occlusion range of the obstacle in the background image, and marked with a bright border. Text prompts corresponding to the obstacle category are also generated, which solves the problem that users cannot accurately identify specific obstacles, their categories and locations based solely on dynamic videos, thus improving the intuitiveness of operation.

[0022] Secondly, the present invention provides a fake cigarette identification system based on AR glasses, which adopts the following technical solution: A fake cigarette detection system based on AR glasses includes: The acquisition module is used to acquire real-time images and the distance between the cigarette box and the eye; A memory for storing the program of the fake smoke identification method based on AR glasses as described above; The processor loads and executes programs from memory.

[0023] By adopting the above technical solution, AR glasses can automatically acquire real-time images, compare detection surfaces, and perform a process from anomaly feature identification to anomaly correction. This realizes the interactive and standardized verification of counterfeit cigarettes on-site, solves the problem of difficulty in combining practical details for identification, and effectively improves detection efficiency and ease of operation.

[0024] In summary, the present invention has at least one of the following beneficial technical effects: 1. By comparing the real-time image of the cigarette box with the standard image of genuine cigarettes, the real-time detection surface is determined. Image features are extracted according to key areas and compared with standard features to obtain abnormal features. This solves the problem that traditional fixed machine vision cannot flexibly verify on-site, achieves accurate identification of cigarette box features, realizes the effect of human-computer interactive counterfeit cigarette identification, and improves the accuracy of counterfeit cigarette identification. 2. By using regular size to filter standard features, the regular image features are compared with the standard features to determine whether there are other features, thereby distinguishing between appearance abnormalities and placement abnormalities. This effectively distinguishes between the authenticity of the cigarette box itself and the recognition interference caused by improper placement, improving the precision of anomaly detection. Attached Figure Description

[0025] Figure 1 This is a flowchart of a fake cigarette identification method based on AR glasses in an embodiment of this application; Figure 2 This is a flowchart of the method for determining whether an abnormal feature is an appearance abnormal feature or a placement abnormal feature in the embodiments of this application; Figure 3 This is a flowchart of an optimized method for adjusting the position of the cigarette box to be identified based on a model that is far from a real cigarette, as described in this application. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0027] This invention discloses a method for identifying fake cigarettes based on AR glasses. (Refer to...) Figure 1 A method for identifying fake cigarettes based on AR glasses includes: Step 1: In response to the recognition start signal, acquire a real-time image of the cigarette box to be recognized.

[0028] The recognition activation signal refers to the initiation command triggered by the user through the AR glasses' interactive interface, such as a voice command, button press, or gesture. The cigarette box to be recognized refers to the tobacco packaging box that needs to be verified for authenticity, such as a soft pack, hard pack, or carton. The real-time image refers to the image of the cigarette box captured by the AR glasses at the current moment. This image is acquired through the AR glasses' camera.

[0029] Step 2: Compare the real-time image with all standard images of genuine cigarettes to obtain the real-time detection surface of the cigarette box to be identified.

[0030] The standard image of a genuine cigarette pack refers to the standard images of each side of a genuine cigarette pack that are pre-stored in the system. The real-time detected side refers to the side of the cigarette pack that is currently displayed in the real-time image. The comparison method is to perform feature matching between the real-time image and each standard image of a genuine cigarette pack to determine which side of the cigarette pack is currently being displayed.

[0031] Step 3: Based on the real-time detection surface, filter all preset real cigarette standard features that conform to the real-time detection surface and locate key areas, and define the real cigarette standard feature as the real-time real cigarette standard feature.

[0032] Genuine cigarette standard features refer to pre-stored features found on genuine cigarette boxes, such as patterns, text, and stamped lettering. Key area positioning refers to the specific location of a genuine cigarette standard feature on a particular face of the cigarette box. Real-time genuine cigarette standard features refer to the portion of the genuine cigarette standard features that belongs to the currently detected face. The filtering method involves extracting features belonging only to a given real-time detected face (e.g., the front) from the feature database. The feature database is a database generated based on all features on each face of a genuine cigarette box, creating a one-to-one correspondence between features and faces.

[0033] Step 4: Extract image features of the cigarette box from the real-time image according to the key area location, and compare the image features with the real-time standard features of real cigarettes to obtain abnormal features.

[0034] Image features refer to visual attributes extracted from real-time images, such as edge contours and pattern shapes. Anomaly features refer to the parts of the image features that differ from the standard features of a genuine cigarette, including appearance anomalies and placement anomalies. The comparison method refers to comparing the extracted image features (edge ​​contours and pattern shapes) with the standard features to see if they are consistent.

[0035] Step 5: If no abnormal features are found, output the preset face-changing operation signal.

[0036] The "face-swapping" signal is a command prompting the user to rotate the cigarette box to detect the next face. The output is a visual prompt generated on the AR glasses' display interface, such as text, arrows, or animations.

[0037] Step 6: If abnormal features exist, generate and output an abnormal correction operation signal based on the abnormal features.

[0038] Anomaly correction operation signals refer to correction guidance information generated for specific anomaly characteristics. They are generated by invoking the corresponding correction strategy module based on the anomaly type, producing visual operation instructions.

[0039] refer to Figure 2 The abnormal features include appearance abnormal features and placement abnormal features, and also include a method for determining whether an abnormal feature is a preset appearance abnormal feature or a preset placement abnormal feature, the method including: Step 60: Filter the regular standard features in the real-time real cigarette standard features according to the preset rule size, and define the image features corresponding to the regular standard features as regular image features.

[0040] Regular dimensions are standard dimensions and allowable error ranges with fixed geometric features on genuine cigarette boxes, calibrated offline and stored in the system. Regular dimensions are obtained as follows: Select genuine cigarette boxes of the same brand, capture standard images at a standard shooting distance (35–50 cm) in an upright position, and determine the standard width, height, and diameter of regular shape features on the cigarette box (such as warning box frames, anti-counterfeiting label outlines, logo borders, and embossed area outlines), forming a regular dimension library. Regular standard features refer to features with clear geometric rules. Regular image features refer to the image content in real-time images corresponding to the regular standard features.

[0041] Step 61: Compare the regular image features with the regular standard features to determine other features besides the regular standard features.

[0042] Other features refer to additional image content within the regular image feature region that does not belong to the regular standard features themselves. The method for determining these features involves removing the content of the regular standard features from the regular image features using image differencing or logical subtraction, leaving only the remaining portion.

[0043] Step 62: When other features exist, remove the other features from the regular image features to obtain the actual regular image features.

[0044] Actual regular image features refer to the images that truly belong to the regular features of the cigarette box itself, after removing other features from the regular image features.

[0045] Step 63: Compare and calculate the placement information of the real-time image by comparing the contour lines of the standard features of the rules with the contour lines of the actual regular image features.

[0046] The contour line refers to the closed or open curve formed by connecting the edge pixels of an image region. Placement information refers to the current placement data of the cigarette box, such as upright, tilted, or rotated. The calculation method involves shape matching between the contour lines of the standard features and the contour lines of the actual image features, and by comparing the translation, rotation, or scaling between the two, the actual placement posture of the cigarette box is deduced.

[0047] Step 64: When other features are not present, output the preset orientation information as the placement information.

[0048] "Upright orientation" means that the cigarette pack is facing the user with the front side facing it, without any rotation or tilt.

[0049] Step 65: When the placement information is not the correct placement information, output the placement information as a placement anomaly feature.

[0050] Abnormal placement characteristics refer to cigarette boxes that are not placed upright, such as being tilted or upside down.

[0051] Step 66: When the placement information is upright, compare it with the image features one by one according to the preset appearance problem features to obtain the image features that match the appearance problem features. Define the image features as problem image features and output the matching appearance problem features as appearance anomaly features. Appearance problem features include wrinkle features, missing features and occlusion features.

[0052] Appearance problems refer to abnormal physical characteristics on cigarette boxes.

[0053] Step 7: Upon receiving the preset abnormal maintenance completion signal, reacquire the real-time image and re-execute steps 2 to 66.

[0054] The abnormal maintenance completion signal is a trigger signal that the user confirms "processing completed" through the AR glasses display interface or voice after following the correction instructions.

[0055] Step 8: If no abnormal features are found on any of the real-time detection surfaces, then output a genuine smoke identification signal.

[0056] The genuine cigarette recognition signal refers to the final result output by the system after determining that all detection surfaces of the cigarette box to be identified meet the standard characteristics of genuine cigarettes.

[0057] This also includes a method for generating and outputting an anomaly correction signal based on the anomaly feature if the anomaly feature is an appearance anomaly feature. This method includes: Step 660: When the abnormal feature is a wrinkle feature, the abnormal feature is parsed to obtain the missing region and the wrinkle stacking region.

[0058] Wrinkle features refer to uneven folds, bulges, or compressed areas on the surface of a cigarette pack. Missing areas refer to areas on the cigarette pack surface where information that should be displayed is obscured due to wrinkles. Overlapping wrinkle areas refer to areas where material overlaps within the wrinkle features. The analysis method involves examining the brightness variations or texture direction of the image regions corresponding to the wrinkle features to distinguish between overlapping and missing areas.

[0059] Step 661: Form a paving direction based on the folded and stacked areas and the missing areas, with the paving direction from the center of the folded and stacked areas toward the center of the missing areas.

[0060] The method involves calculating the line connecting the center points of the two regions and determining the direction vector.

[0061] Step 662: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed by assigning a preset wrinkle mark to the contour of the area where the abnormal feature is located, and a mark arrow corresponding to the flattening direction.

[0062] Manipulated images refer to AR displays generated by overlaying markers and guide arrows onto raw real-time images. Wrinkle markers are special color markings, such as red borders, applied to the outlines of areas containing anomalous features.

[0063] Step 663: When the abnormal feature is an occlusion feature, identify the occlusion region where the occlusion feature is located in the real-time image, and extract the contour of the occlusion region to obtain the occlusion boundary.

[0064] Occlusion features refer to key areas of a cigarette pack being covered by other objects, such as fingers or foreign objects. The occluded region refers to the area in a real-time image that is obscured by an occluded object. The occlusion boundary is the boundary between the occluded region and the visible region. The identification method involves analyzing the mismatch between image features and standard features to determine if occlusion exists. The extraction method uses edge detection algorithms (such as the Canny operator) to extract the closed boundary of the occluded region.

[0065] Step 664: Based on the location of the key area corresponding to the obstruction boundary and the real-time true smoke standard features, generate the movement direction to remove the obstruction.

[0066] The generation method involves comparing the relative positions of the occlusion boundary and the key region, and calculating the direction of the minimum displacement vector that moves the occlusion region away from the key region.

[0067] Step 665: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed after assigning a preset occlusion mark to the occluded area and a marker arrow corresponding to the movement direction.

[0068] Occlusion markings are special identifiers assigned to occluded areas.

[0069] This also includes a method for generating and outputting an anomaly correction operation signal based on the anomaly feature if the anomaly feature is a placement anomaly feature. This method includes: Step 650: Adjust the size of the preset real cigarette model according to the placement information to obtain the adjusted real cigarette model.

[0070] A genuine cigarette box model is a model that accurately reproduces the appearance features and size proportions of each surface of an authentic cigarette box. Adjusting the genuine cigarette box model refers to scaling and rotating the model to obtain a new model based on the actual size and orientation of the cigarette box in the current real-time image.

[0071] Step 651: Deconstruct the placement information to obtain the rotation angle deviation and the cigarette box area.

[0072] Rotation angle deviation refers to the angular deviation between the current actual orientation of the cigarette box and its upright position. The cigarette box area refers to the area occupied by the cigarette box in the real-time image.

[0073] Step 652: Analyze the real-time image to obtain the user's gripping hand.

[0074] The user's gripping hand refers to whether the hand currently holding the cigarette pack is the left or right hand. The method involves analyzing features such as the relative position of the arm and cigarette pack, and the direction of finger bending in real-time images, using a gesture recognition algorithm to determine the gripping hand. Here, the gesture recognition algorithm employs the MediaPipe Hands solution. In the AR glasses, this solution is integrated by the processor calling the glasses' built-in camera SDK interface to acquire real-time image frames at a frame rate of 30fps. These image frames are then input into the MediaPipe Hands palm detection model pre-deployed in the processor's NPU (Neural Processing Unit). This model outputs a bounding box representing the presence and approximate position of the hand. Based on this bounding box, the system then activates a pose estimation model to extract the 2D coordinates of 21 key points on the hand. Finally, by analyzing the relative angles between the index, middle, and thumb fingers, as well as the spatial relationship between the wrist and the cigarette pack area, the gripping hand is determined.

[0075] Step 653: Simulate the rotation angle deviation according to the user's grip to obtain the corrected rotation direction.

[0076] Correcting the rotation direction refers to calculating the actual direction the user should rotate the cigarette box based on the deviation between the user's grip and the rotation angle, such as rotating clockwise or flipping it to the right.

[0077] Step 654: Generate a dynamic adjustment illustration video based on the corrected rotation direction and the adjusted real cigarette model. The dynamic adjustment illustration video demonstrates how the real cigarette model is rotated according to the corrected rotation direction.

[0078] The dynamic adjustment illustration video refers to the dynamic process of adjusting the real cigarette model from its current posture to its upright posture according to the corrected rotation direction.

[0079] Step 655: Based on the cigarette box area, dynamically adjust the display area of ​​the schematic video, ensuring that the display area and the cigarette box area do not interfere with each other.

[0080] The display area refers to the area in AR glasses used to demonstrate how to operate the device.

[0081] Step 656: Generate a dynamically adjusted schematic video based on the real-time image and display it in the display area of ​​the AR glasses.

[0082] The process of generating a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real smoke model also includes: Step 6540: Compare the cigarette box area with the preset standard cigarette box area to obtain the cigarette box size multiple.

[0083] The standard area of ​​a cigarette pack refers to the area of ​​the image that the cigarette pack should occupy at the optimal recognition distance. The cigarette pack size multiple refers to the ratio of the cigarette pack area in the real-time image to the area of ​​the standard cigarette pack area.

[0084] Step 6541: Calculate the distance between the cigarette box and your eyes based on the multiple of the cigarette box size.

[0085] The distance from the cigarette box to the eye refers to the straight-line distance from the actual surface of the cigarette box to the user's eye. It is calculated using the pinhole imaging principle, employing triangulation based on a multiple of the cigarette box's size and its actual dimensions.

[0086] Step 6542: If the distance between the cigarette box and the eyes is greater than the preset comfortable distance, then generate a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real cigarette model.

[0087] The comfortable distance refers to the range within which users can easily observe AR prompts while ensuring clear image recognition. Here, the comfortable distance is set to 35 cm to 50 cm.

[0088] Step 6543: If the distance between the cigarette box and the eye is less than the comfortable distance, output the preset cigarette box moving away signal and receive the moving away completion signal.

[0089] The "Cigarette Pack Away" signal instructs the user to move the cigarette pack backward, increasing the distance between it and their glasses. The "Away Complete" signal confirms that the cigarette pack has been moved to a comfortable distance.

[0090] Step 6544: Upon receiving the signal indicating that the distance has been cleared, a dynamic adjustment illustration video is generated based on the correction of the rotation direction and the adjustment of the real smoke model.

[0091] The methods for outputting a preset signal to move the cigarette box away include: Step 65430: If the distance between the cigarette box and the eye is less than the comfortable distance, adjust the angle of the real cigarette model according to the rotation angle deviation to obtain a model that is far away from the real cigarette model.

[0092] The model that is far removed from the real smoke model refers to the model that is regenerated after adjusting the real smoke model.

[0093] Step 65431: Develop the model's movement trajectory based on the distance between the cigarette box and the eye.

[0094] The model's movement trajectory refers to the virtual path from the current position of the cigarette box to a comfortable distance position.

[0095] Step 65432: Based on the changes in the moving-away real cigarette model and the adjusted real cigarette model according to the preset change duration, a change process from the adjusted real cigarette model to the moving-away real cigarette model is formed. The change process is placed on the model movement trajectory to form a dynamic video of the cigarette box moving away, and the dynamic video of the cigarette box moving away is generated in the display interface of the AR glasses.

[0096] The "cigarette box moving away from the background" dynamic video refers to a complete animation showing a cigarette box model moving and rotating along a trajectory from near to far until it is upright.

[0097] Step 65433: Update the distance of the cigarette box from the dynamic video in real time based on the placement information in the real-time image and continuously obtain the distance of the cigarette box from the eye, and define the distance of the cigarette box from the eye as the distance away.

[0098] "Distance away" refers to the distance between the cigarette pack and the user's eyes, calculated in real time as the user actually moves the cigarette pack. "Continuous acquisition" refers to calculating the distance away at a fixed frequency, such as 30Hz.

[0099] Step 65434: When the distance away is equal to the comfort distance, generate a static image of the real smoke model and display it on the AR glasses' screen. In the static image, the real smoke model is in the upright position.

[0100] A static image refers to a fixed image of a real cigarette model in an upright position at a comfortable distance.

[0101] refer to Figure 3 It also includes an optimization method for adjusting the position of the cigarette box to be identified based on the distance from the real cigarette model, the method including: Step 65435: Generate the predicted trajectory of the cigarette box moving away from the dynamic video.

[0102] The predicted cigarette box departure trajectory refers to the physical movement route that the user should take, calculated based on the model's movement path in the dynamic video of the cigarette box moving away from the device. The method involves using the AR glasses' SLAM system to obtain the current state of the cigarette box, determining the target state based on a comfortable distance, and then calculating a collision-free spatial trajectory as the predicted cigarette box departure trajectory using a path planning algorithm based on environmental obstacles identified by a depth sensor.

[0103] Step 65436: Generate the actual cigarette box's trajectory based on the real-time image.

[0104] The actual cigarette box moving away trajectory refers to the actual movement path fitted based on the displacement of the cigarette box in real-time images. The fitting method involves tracking the cigarette box in continuous real-time images to obtain its position in each frame, converting the two-dimensional image position into three-dimensional spatial coordinates using the SLAM system of the AR glasses, and then performing curve fitting on the obtained series of three-dimensional spatial coordinate points to obtain the actual cigarette box moving away trajectory.

[0105] Step 65437: When the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide, a preset stop movement command is issued, and the distance between the cigarette box and the eye is determined at this time. This distance between the cigarette box and the eye is defined as the obstacle avoidance distance.

[0106] The "Stop Moving" command is a signal prompting the user to stop moving the cigarette box. The obstacle avoidance distance refers to the actual distance from the cigarette box to the user's eyes when a deviation from the trajectory is detected.

[0107] Step 65438: Update the comfort distance according to the obstacle avoidance distance and regenerate the still image.

[0108] This also includes an optimization method when the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide. This method includes: Step 654370: Determine the background image other than the real-time image based on the real-time image corresponding to the obstacle avoidance distance.

[0109] Background image refers to all areas in the real-time image except for the cigarette box.

[0110] Step 654371: Perform object feature analysis based on the background image to obtain obstacle categories other than the preset human arm features.

[0111] The user's arm features refer to the characteristics of their own arm, such as the arm's outline. The obstacle category refers to the type of non-arm-obstructed object, such as a water glass or a document.

[0112] Step 654372: Determine the occlusion range of obstacles in the background image based on the obstacle category.

[0113] Obstacle occlusion refers to the area occupied by an obstacle in an image.

[0114] Step 654373: Calculate the optimal obstacle avoidance path based on the real-time image and the obstacle occlusion range.

[0115] The optimal obstacle avoidance path is the path that moves the cigarette box from its current position to a comfortable distance without being obstructed by any obstacles.

[0116] Step 654374: Update the model's movement trajectory based on the optimal obstacle avoidance path. The starting point for the update is a point on the model's movement trajectory corresponding to the obstacle avoidance distance.

[0117] The starting point for the update refers to replanning the subsequent path from the position corresponding to the obstacle avoidance distance on the model's movement trajectory.

[0118] Step 654375: Perform steps 65432 to 65434 according to the updated model movement trajectory.

[0119] This also includes a method for generating obstacle warning information, the method comprising: Step 654376: Determine the corresponding display area of ​​the obstacle in the AR glasses display interface based on the occlusion range of the obstacle in the background image.

[0120] Step 654377: Overlay preset obstacle markers on the corresponding display area, with the obstacle markers having a highlighted border.

[0121] Highlighted borders refer to borders that are superimposed around the outline of an obstacle.

[0122] Step 654378: Generate text prompts corresponding to the obstacle category and display them on the edge area of ​​the obstacle occlusion range on the AR glasses display interface.

[0123] Textual prompts refer to text that describes the type of obstacle and provides guidance on how to avoid it.

[0124] Step 654379: When the occlusion range of an obstacle changes, update the position of the obstacle marker and the text prompt information in real time.

[0125] Based on the same inventive concept, embodiments of the present invention provide a fake cigarette identification system based on AR glasses.

[0126] A fake cigarette detection system based on AR glasses includes: The acquisition module is used to acquire real-time images and the distance between the cigarette box and the eye; A memory for storing a program for a fake cigarette detection method based on AR glasses; The processor loads and executes programs from memory.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0128] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying fake cigarettes based on AR glasses, characterized in that, include: Step 1: In response to the recognition activation signal, acquire a real-time image of the cigarette box to be recognized through the camera of the AR glasses; Step 2: Compare the real-time image with all standard images of genuine cigarettes to obtain the real-time detection surface of the cigarette box to be identified; Step 3: Based on the real-time detection surface, filter all preset real cigarette standard features that match the real-time detection surface and locate key areas, and define the real cigarette standard feature as the real-time real cigarette standard feature. Step 4: Extract image features of the cigarette box from the real-time image according to the key area location, and compare the image features with the real-time standard features of real cigarettes to obtain abnormal features; Step 5: If no abnormal features are found, output the preset face-changing operation signal; Step 6: If abnormal features exist, generate and output an abnormality correction operation signal based on the abnormal features; The abnormal features include appearance abnormal features and placement abnormal features, and also include a method for determining whether the abnormal feature is a preset appearance abnormal feature or a preset placement abnormal feature, the method including: Step 60: Filter the regular standard features in the real-time real cigarette standard features according to the preset rule size, and define the image features corresponding to the regular standard features as regular image features; Step 61: Compare the regular image features with the regular standard features to determine other features besides the regular standard features; Step 62: When other features exist, remove them from the regular image features to obtain the actual regular image features; Step 63: Compare and calculate the placement information of the real-time image by comparing the contour lines of the standard features of the rules with the contour lines of the actual regular image features; Step 64: When other features are not present, output the preset orientation information as the placement information; Step 65: When the placement information is not the correct placement information, output the placement information as a placement anomaly feature; Step 66: When the placement information is upright, compare it with the image features one by one according to the preset appearance problem features to obtain the image features that match the appearance problem features. Define the image features as problem image features and output the matching appearance problem features as appearance anomaly features. The appearance problem features include wrinkle features, missing features and occlusion features. Step 7: Upon receiving the preset abnormal maintenance completion signal, reacquire the real-time image and re-execute steps 2 to 66; Step 8: If no abnormal features are found on any of the real-time detection surfaces, then output a genuine smoke identification signal.

2. The method for identifying fake cigarettes based on AR glasses according to claim 1, characterized in that, It also includes a method for generating and outputting an anomaly correction operation signal based on the anomaly feature if the anomaly feature is an appearance anomaly feature, the method including: Step 660: When the abnormal feature is a wrinkle feature, the abnormal feature is parsed to obtain the missing region and the wrinkle stacking region. Step 661: Form a paving direction based on the folded stacked area and the missing area, wherein the paving direction is from the center of the folded stacked area toward the center of the missing area; Step 662: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed by assigning a preset wrinkle mark to the contour of the area where the abnormal feature is located and a mark arrow corresponding to the flattening direction. Step 663: When the abnormal feature is an occlusion feature, identify the occlusion region where the occlusion feature is located in the real-time image, and extract the contour of the occlusion region to obtain the occlusion boundary. Step 664: Based on the location of the key area corresponding to the obstruction boundary and the real-time real smoke standard features, generate the movement direction to remove the obstruction; Step 665: Generate an operation image based on the real-time image and display it in the AR glasses. The operation image includes a real-time image formed after assigning a preset occlusion mark to the occlusion area and a marker arrow corresponding to the movement direction.

3. The method for identifying fake cigarettes based on AR glasses according to claim 1, characterized in that, It also includes a method for generating and outputting an anomaly correction operation signal based on the anomaly feature if the anomaly feature is a placement anomaly feature, the method including: Step 650: Adjust the size of the preset real cigarette model according to the placement information to obtain the adjusted real cigarette model; Step 651: Deconstruct the placement information to obtain the rotation angle deviation and the cigarette box area; Step 652: Analyze the real-time image to obtain the user's gripping hand; Step 653: Simulate the rotation angle deviation according to the user's grip to obtain the corrected rotation direction; Step 654: Generate a dynamic adjustment illustration video based on the corrected rotation direction and the adjusted real cigarette model. The dynamic adjustment illustration video demonstrates the adjustment of the real cigarette model by rotating it according to the corrected rotation direction. Step 655: Form a display area for a dynamically adjusted schematic video based on the cigarette box area, wherein the display area and the cigarette box area do not interfere with each other; Step 656: Generate a dynamically adjusted schematic video based on the real-time image and display it in the display area of ​​the AR glasses.

4. The method for identifying fake cigarettes based on AR glasses according to claim 3, characterized in that, Before generating a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real smoke model, the following also included: Step 6540: Compare the cigarette box area with the preset standard cigarette box area to obtain the cigarette box size multiple; Step 6541: Calculate the distance between the cigarette box and the eye based on the multiple of the cigarette box size; Step 6542: If the distance between the cigarette box and the eyes is greater than the preset comfortable distance, then generate a dynamic adjustment illustration video based on correcting the rotation direction and adjusting the real cigarette model; Step 6543: If the distance between the cigarette pack and the eyes is less than the comfortable distance, output the preset cigarette pack moving away signal and receive the moving away completion signal; Step 6544: Upon receiving the signal indicating that the distance has been cleared, a dynamic adjustment illustration video is generated based on the correction of the rotation direction and the adjustment of the real smoke model.

5. The method for identifying fake cigarettes based on AR glasses according to claim 4, characterized in that, The preset methods for outputting the cigarette box away from the signal include: Step 65430: If the distance between the cigarette box and the eye is less than the comfortable distance, adjust the angle of the real cigarette model according to the rotation angle deviation to obtain a model that is far away from the real cigarette model; Step 65431: Develop the model's movement trajectory based on the distance between the cigarette box and the eye; Step 65432: Based on the changes in the moving away from the real cigarette model and the adjustment of the real cigarette model according to the preset change duration, a change process from adjusting the real cigarette model to moving away from the real cigarette model is formed. The change process is placed on the model movement trajectory to form a dynamic video of the cigarette box moving away, and the dynamic video of the cigarette box moving away is generated in the display interface of the AR glasses. Step 65433: Update the distance of the cigarette box from the dynamic video in real time based on the placement information in the real-time image and continuously obtain the distance of the cigarette box from the eye, and define the distance of the cigarette box from the eye as the distance away; Step 65434: When the distance away is equal to the comfort distance, generate a static image of the real smoke model and display it on the AR glasses' display interface. In the static image, the real smoke model is in the upright position.

6. The method for identifying fake cigarettes based on AR glasses according to claim 5, characterized in that, It also includes an optimized method for adjusting the position of the cigarette box to be identified based on the model of distance from the real cigarette, the method comprising: Step 65435: Develop the predicted trajectory of the cigarette box moving away from the moving video. Step 65436: Generate the actual cigarette box's trajectory based on the real-time image; Step 65437: When the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide, issue a preset stop movement command, and determine the distance between the cigarette box and the eye at this time, and define the distance between the cigarette box and the eye as the obstacle avoidance distance; Step 65438: Update the comfort distance according to the obstacle avoidance distance and regenerate the still image.

7. The method for identifying fake cigarettes based on AR glasses according to claim 6, characterized in that, It also includes an optimization method when the actual cigarette box moves away from the trajectory and the expected cigarette box moves away from the trajectory do not coincide, the method includes: Step 654370: Determine the background image other than the real-time image based on the real-time image corresponding to the obstacle avoidance distance; Step 654371: Perform object feature analysis based on the background image to obtain obstacle categories other than the preset human arm features; Step 654372: Determine the occlusion range of obstacles in the background image based on the obstacle category; Step 654373: Calculate the optimal obstacle avoidance path based on the real-time image and the obstacle occlusion range; Step 654374: Update the model's movement trajectory based on the optimal obstacle avoidance path. The starting point for the update is the point on the model's movement trajectory corresponding to the obstacle avoidance distance. Step 654375: Perform steps 65432 to 65434 according to the updated model movement trajectory.

8. The method for identifying fake cigarettes based on AR glasses according to claim 7, characterized in that, It also includes a method for generating obstacle warning information, the method comprising: Step 654376: Determine the corresponding display area of ​​the obstacle in the AR glasses display interface based on the occlusion range of the obstacle in the background image; Step 654377: Overlay a preset obstacle mark on the corresponding display area, the obstacle mark being a highlighted border; Step 654378: Generate text prompts corresponding to the obstacle category and display them on the edge area of ​​the obstacle occlusion range on the AR glasses display interface; Step 654379: When the occlusion range of an obstacle changes, update the position of the obstacle marker and the text prompt information in real time.

9. A fake cigarette identification system based on AR glasses, characterized in that, include: The acquisition module is used to acquire real-time images and the distance between the cigarette box and the eye; A memory for storing a program for a fake smoke identification method based on AR glasses as described in any one of claims 1 to 8; The processor loads and executes programs from memory.

Images