Parcel seal identification method and device, equipment and storage medium
By using a DETR-based seal status recognition model and a multi-scale edge perception module MEM, combined with confidence threshold judgment logic, the problems of low efficiency and poor accuracy in package seal status recognition are solved, achieving high-precision and fully automated seal status detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI DONGPU INFORMATION TECH CO LTD
- Filing Date
- 2026-01-28
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies have low efficiency in identifying the status of package seals, cannot meet the requirements of high-speed throughput, lack refined analysis capabilities, have a high rate of missed detections and serious misjudgments, and are particularly difficult to identify subtle states such as the integrity of the seal, positional offset, and secondary pasting.
A seal status recognition model based on the DETR architecture is adopted, which combines a multi-scale edge perception module MEM and a Transformer encoder. A sample dataset is generated by labeling key information, and edge consistency regularization term and confidence threshold judgment logic are introduced to realize automatic recognition of seal status and anomaly detection.
It significantly reduces the false negative rate and false positive rate of seal identification, realizes high-precision automated identification of seal status and end-to-end fully automated processing, avoids human subjective bias, and ensures the reliability and security of identification.
Smart Images
Figure CN121999291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of logistics technology, and in particular to a method, apparatus, equipment and storage medium for identifying package seals. Background Technology
[0002] In current logistics systems, one-time seals serve as physical proof of a package's airtightness, and their integrity directly impacts product safety and liability. However, traditional manual inspections are inefficient and cannot meet the high-throughput demands of modern logistics sorting centers. Furthermore, inconsistent judgment standards, strong subjectivity, and susceptibility to disputes make them difficult to scale effectively with increasing business volume. Existing computer vision solutions are mostly based on general object detection models, focusing primarily on the overall appearance or single-sided information of the package. They lack the ability to perform refined and automated analysis of one-time seal status, particularly failing to effectively identify whether the seal is intact, whether its position is offset, or whether there are signs of re-application or minor tears. They are not optimized for special targets like seals—thin, low-contrast, and susceptible to lighting and occlusion—leading to high false negative rates and severe misjudgments when identifying package seal status. Summary of the Invention
[0003] This invention provides a method, device, and storage medium for identifying package seals, which solves the problems of inability to automatically identify package seal status, high missed detection rate, and serious misjudgment in the prior art.
[0004] According to one aspect of this application, a method for identifying package seals is disclosed, the method comprising: Acquire an initial seal state image of the target seal, and annotate the initial seal state image with key information to obtain a target seal sample dataset. The key information includes at least the boundary polygon, integrity status label, and key edge point coordinates. A seal status recognition model based on the DETR architecture is constructed, and a multi-scale edge perception module MEM is embedded between its CNN backbone network and Transformer encoder. The DETR-MEM seal state recognition model, which embeds a multi-scale edge perception module (MEM), is trained based on the target seal sample dataset to obtain a trained DETR-MEM seal state recognition model. The loss function used in the training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of the key edge points. The initial seal state image is input into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category and anomaly type of the target seal. Obtain the preset confidence threshold and the preset list of abnormal states; When the confidence score of the target seal is higher than the preset confidence threshold, the integrity status category output by the model is matched with the preset list of abnormal statuses. When the integrity status category exists in the preset list of abnormal statuses, the target seal is determined to be in an abnormal state, and an early warning is triggered.
[0005] According to another aspect of this application, a package seal identification device is also disclosed, the device comprising: The sample dataset acquisition module is used to acquire an initial seal state image of the target seal, and to annotate the initial seal state image with key information to obtain a target seal sample dataset. The key information includes at least the boundary polygon, integrity status label and key edge point coordinates. A multi-scale edge perception module (MEM) is embedded to build a seal state recognition model based on the DETR architecture and to embed the multi-scale edge perception module between its CNN backbone network and Transformer encoder. The seal status recognition model training module is used to train the seal status recognition model embedded with the multi-scale edge perception module MEM based on the target seal sample dataset to obtain the trained DETR-MEM seal status recognition model. The loss function used for training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of the key edge points. The result output module is used to input the initial seal state image into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category and anomaly type of the target seal. The preset information acquisition module is used to acquire the preset confidence threshold and the preset list of abnormal states; The target seal status determination module is used to match the integrity status category output by the model with the abnormal status preset list when the confidence score of the target seal is higher than the confidence preset threshold. When the integrity status category exists in the abnormal status preset list, the target seal status is determined to be abnormal and an early warning is triggered.
[0006] According to another aspect of this application, an electronic device is also disclosed, the electronic device including a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to perform the various steps of the package seal identification method as described in any of the preceding claims.
[0007] According to another aspect of this application, a computer-readable storage medium is also disclosed, wherein instructions are stored on the computer-readable storage medium, characterized in that, when executed by a processor, the instructions implement the various steps of the package seal identification method as described in any of the preceding claims.
[0008] The present invention includes, but is not limited to, the following beneficial effects: (1) This scheme introduces a multi-scale edge perception module MEM to generate edge enhancement features, so that the model focuses on the contour and continuity of the seal, overcoming the interference caused by low contrast; by integrating features of different levels of CNN, the model has both detail perception and semantic understanding, thereby accurately distinguishing normal wear and abnormal tear, natural texture and secondary pasting marks, significantly reducing missed detection and false judgment. (2) A dual judgment logic based on confidence threshold and abnormal state list is introduced. The confidence threshold effectively filters out the uncertain fuzzy prediction of the model and only takes action on the high confidence score result, controlling false alarms from the source. The abnormal state list directly maps the technical label to the business action, realizing end-to-end full automation from recognition to execution without human intervention, avoiding the deviation caused by human subjectivity. The processing of low confidence scores ensures the safety redundancy of the system, ensuring that there are no false alarms caused by the uncertainty of the model, and ensuring the reliability of seal recognition. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0010] Figure 1 This is a flowchart of a package seal identification method according to an embodiment of this application; Figure 2 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 3 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 4 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 5 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 6 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 7 This is another flowchart of the package seal identification method according to an embodiment of this application; Figure 8 This is a structural block diagram of the package seal identification device according to an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation
[0011] This invention provides a method, apparatus, device, and storage medium for package seal recognition. The method includes acquiring an initial seal state image of a target seal; annotating the initial seal state image with key information to obtain a target seal sample dataset; the key information includes at least boundary polygons, integrity status labels, and key edge point coordinates; constructing a seal state recognition model based on the DETR architecture, and embedding a multi-scale edge perception module (MEM) between its CNN backbone network and Transformer encoder; training the seal state recognition model embedded with the MEM based on the target seal sample dataset to obtain a trained DETR-MEM seal state recognition model, wherein the loss function used for training includes edge prediction based on the model. The algorithm calculates an edge consistency regularization term based on features and key edge point coordinates. It inputs the initial seal status image into a pre-trained DETR-MEM seal status recognition model to obtain the target seal detection result and its corresponding confidence score. The target seal detection result includes at least the target seal's bounding box coordinates, integrity status category, and anomaly type. It obtains a preset confidence threshold and a preset list of anomaly states. When the target seal's confidence score is higher than the preset threshold, it matches the model's output integrity status category with the preset list of anomaly states. If the integrity status category exists in the preset list of anomaly states, the target seal status is determined to be abnormal, and an alert is triggered. This solution addresses the problems of automatic identification of package seal status, high false negative rates, and severe misjudgments in existing technologies.
[0012] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0013] For ease of understanding, the specific process of the embodiments of the present invention will be described below. Figure 1 A flowchart of a package seal identification method, such as Figure 1 As shown, it includes the following steps: S100. Collect the initial seal state image of the target seal, and annotate the key information of the initial seal state image to obtain the target seal sample dataset.
[0014] The key information includes at least the boundary polygon, integrity status label, and coordinates of key edge points.
[0015] Specifically, such as Figure 2 The diagram shown is another flowchart of the package seal identification method according to an embodiment of this application. This flowchart is an exemplary illustration of step S100 in this method. (See attached diagram.) Figure 2 It includes the following steps: S200: Collect images of packages containing different seal statuses.
[0016] The sealing status includes, but is not limited to, normal sealing, slight offset, partial tearing, complete detachment, re-application, and soiling / obstruction.
[0017] Specifically, industrial cameras and light sources are deployed at key locations such as the receiving and loading ports of the sorting center to automatically capture images of passing packages. Alternatively, high-pixel mobile phones or tablets can be used for staff to manually capture images of suspicious or documented packages. The types of images captured include, but are not limited to, normal seals, slight offsets, partial tears, complete detachment, re-adhesion, and soiling / obstruction. In actual collection, efforts should be made to cover different subtypes of each state. For example, partial tears can be further subdivided into horizontal tears, vertical tears, and corner tears. Seals of the same state type should be captured under different lighting conditions, different package materials, different backgrounds, and different degrees of newness and age.
[0018] S202. Clarify the rules for labeling key information. The rules for labeling key information include the rules for labeling boundary polygons, the classification standards for integrity labels, and the selection specifications for key edge points.
[0019] Specifically, the boundary polygon annotation rules define how to draw annotation polygons along the outer contour of the seal; the integrity label classification standard defines the category system of seal integrity status and the visual judgment criteria for each status; and the key edge point selection specification defines the position and density of feature points selected on the seal edge for supervised model learning. By clarifying the key information annotation rules, a "process standard" for data production is defined, transforming what to annotate into precise and unambiguous written specifications and visual guidelines on how to annotate, ensuring consistency in annotation quality and clarity of subsequent model learning objectives.
[0020] S204. Based on the key information annotation rules, the initial seal status image is annotated with key information to obtain an annotated seal status image.
[0021] Specifically, in a professional annotation platform, all initial seal state images are imported, and the previously defined key information annotation rules are loaded as label configurations and operation guidelines within the platform. This completes the association configuration between the annotation tool and the sample images. The annotation tool then performs detailed annotation on the seal regions in each image. First, a boundary polygon is drawn along the outer contour of the target seal. Then, referring to the integrity label classification standard, the most accurate integrity state category label is selected from the preset integrity state category list and assigned to this polygon region. Finally, according to the key edge point selection specifications, a series of key points are precisely placed on the edge of the target seal, and the pixel coordinates of each key point are recorded. After completing the above three-step cycle of drawing the frame, classifying, and annotating, an initial seal state image is transformed into an annotated seal state image. By annotating the initial seal state image with key information, the original image is transformed into structured training data that the system can understand and learn from, making the subsequent model's judgment of normal states more accurate and greatly reducing the system's false alarm rate.
[0022] S206. Perform data cleaning on the labeled seal status images, remove samples with incorrect or incomplete labeling, and obtain the target seal sample dataset.
[0023] Specifically, after annotation, cross-validation is performed. Integrity verification rules are used to check whether the boundary polygons completely cover the seal area and whether the integrity status labels are accurate. Consistency verification rules are used to check for spatial logical conflicts between the boundary polygons and key point coordinates, and whether the key coordinate points deviate from the true edge of the seal. Incorrectly annotated or incomplete samples are removed, resulting in a target seal sample dataset with accurate annotation information. Data cleaning provides the model with clean data information, allowing the model to learn correct patterns, avoid being misled by erroneous samples, reduce the burden of "guessing" and "correcting" noise during training, improve training efficiency, and reduce model bias and misjudgment caused by data problems from the source, ensuring the stability and reliability of the system.
[0024] S102. Construct a seal status recognition model based on the DETR architecture, and embed a multi-scale edge perception module MEM between its CNN backbone network and Transformer encoder.
[0025] Specifically, such as Figure 3 The diagram shown is another flowchart of the package seal identification method according to an embodiment of this application. This flowchart is an exemplary illustration of step S102 in this method. (See attached diagram.) Figure 3 It includes the following steps: S300, Construct a seal status recognition model based on the DETR architecture.
[0026] The model includes a CNN backbone network, a Transformer encoder, a Transformer decoder, and a prediction output layer.
[0027] Specifically, the CNN backbone network acts as the model's visual feature extractor, extracting multi-level, multi-scale semantic feature maps from the input image and performing dimensionality transformation and flattening on the feature maps to obtain a one-dimensional feature sequence. The Transformer encoder consists of multiple identical stacked layers, each containing a self-attention mechanism and a feedforward neural network. It interacts with all positions in the feature sequence output by the CNN backbone network, establishing global dependencies within the feature sequence through multi-layer self-attention calculations to obtain the encoded global feature sequence. The Transformer decoder also consists of multiple stacked layers, each containing self-attention, cross-attention, and feedforward networks. It receives the global features output by the encoder and calculates the encoded global features with the target query vector initialized by the decoder through the cross-attention mechanism. This completes the decoder's accurate extraction and association of global features, decoding the target feature sequence for output. The prediction output layer contains two parallel feedforward networks: one is a bounding box prediction head for predicting the seal region, and the other is a category prediction head for predicting the seal's integrity category. Each decoded target feature is processed by the bounding box prediction head and the category prediction head, ultimately yielding the predicted bounding box and integrity category probability.
[0028] S302. Input the target seal sample dataset into the CNN backbone network to obtain multi-scale feature maps. The multi-scale feature maps have different spatial resolutions and semantic information.
[0029] Specifically, in the constructed DETR model, image data from the target seal sample dataset is input into the CNN backbone network. The image data flows through each layer of the CNN backbone, where downsampling and channel number increase are performed to extract semantic feature maps from different stages of the CNN backbone network. These feature maps from different stages constitute a multi-scale feature map set. By actively extracting multi-scale feature maps from the CNN backbone, the model is equipped with a "magnifying glass" (shallow details) and an "instruction manual" (deep semantics). These tools are then passed to the next step—the multi-scale edge perception module MEM—which performs targeted "edge enhancement" processing on the information in the "magnifying glass" and "instruction manual," thus laying a solid data foundation for achieving high-precision seal status recognition.
[0030] S304, Introducing the Multi-Scale Edge Sensing Module (MEM).
[0031] S306. Calculate the gradient magnitude of each scale on the multi-scale feature map based on the Sobel edge detection algorithm, and obtain the Sobel edge response map that reflects the overall outline and direction of the target seal at that scale. Perform Gaussian filtering for noise reduction, gradient calculation and double threshold filtering on the feature map at that scale based on the Canny edge detection algorithm, and obtain the Canny edge response map of the target seal for minor fractures, burrs and continuous edges.
[0032] Specifically, the Sobel edge detection algorithm is deployed to calculate the gradient values of each pixel in the multi-scale feature map at each scale in the horizontal and vertical directions. Based on the gradient values in the horizontal and vertical directions, the gradient magnitude of each pixel is calculated. The magnitude of the gradient magnitude represents the strength of the edge at that pixel. The gradient magnitudes of all pixels are combined to obtain the Sobel edge response map. The Sobel edge detection algorithm can quickly and efficiently capture the overall outline and direction of the target seal, laying the foundation for subsequent detailed analysis. The Canny edge detection algorithm is deployed by first performing Gaussian filtering to denoise the multi-scale feature map, calculating the gradient magnitude and direction of each pixel, then performing non-maximum suppression, and finally performing double-threshold hysteresis filtering to obtain the Canny edge response map. The double-threshold hysteresis filtering mechanism is as follows: a high gradient magnitude threshold and a low gradient magnitude threshold are set. When the calculated gradient magnitude is greater than the high gradient magnitude threshold, the current pixel is marked as a strong edge and the system recognizes it as an edge. When the calculated gradient magnitude is less than the low gradient magnitude threshold, the current pixel is recognized as a non-edge and directly discarded. When the calculated gradient magnitude is between the high gradient magnitude threshold and the low gradient magnitude threshold, the current pixel is marked as a weak edge. The Canny algorithm can accurately display the continuity of edges. For a complete seal, its Canny edge response map should be a continuous or nearly continuous curve. Once there are breaks, burrs, or irregular gaps, the Canny map will clearly show the interruption or distortion of the edge line, which is direct evidence of "local tearing" or "wear". At the same time, secondary pasting may cause the edge line to appear as a ghost or slightly non-overlapping, which can also be sensitively captured by the Canny edge response map.
[0033] S308. Perform channel stitching and fusion of the Slbel edge response map and Canny edge response map at the same scale with their corresponding initial seal state image to obtain the edge enhancement features at that scale.
[0034] S310. Fuse edge enhancement features from all scales to obtain comprehensive edge enhancement features. Input the comprehensive edge enhancement features into the Transformer encoder to complete the integration of the multi-scale edge perception module MEM and the DETR model.
[0035] Specifically, a scale matching the Transformer input requirements is selected as the target size. The edge enhancement features obtained in step S308 at different scales are scale-aligned. All scale-aligned edge enhancement feature maps are then concatenated along the channel dimension. The concatenated feature map has a high number of channels. The concatenated high-channel feature map is then subjected to cross-scale information fusion and channel dimensionality reduction using a 1×1 convolution kernel to obtain a comprehensive edge enhancement feature. The number of channels after dimensionality reduction matches the input dimension of the Transformer encoder. In the already constructed seal status recognition model based on the DETR architecture, the newly generated comprehensive edge enhancement feature replaces the original single-scale feature map output by the CNN backbone network. The comprehensive edge enhancement feature is then input into the Transformer encoder to complete the embedding of the multi-scale edge perception module MEM.
[0036] S104. Train the DETR-MEM seal state recognition model based on the target seal sample dataset to obtain the trained DETR-MEM seal state recognition model.
[0037] The loss function used in training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of key edge points.
[0038] Specifically, such as Figure 4 The diagram shown is another flowchart of the package seal identification method according to an embodiment of this application. This flowchart is an exemplary illustration of step S104 in this method. (See attached diagram.) Figure 4 It includes the following steps: S400. Input the images in the target seal sample dataset into the seal status recognition model, and obtain the target detection prediction results and intermediate edge feature maps output by the model. The intermediate edge feature maps are generated by the multi-scale edge perception module MEM.
[0039] Specifically, the target seal sample dataset undergoes batch enhancement processing. This involves simulating transport wear by randomly adding noise and blurring; altering edge contrast by adjusting image brightness, contrast, and color temperature to simulate different lighting conditions; changing edge geometry by randomly rotating the image to simulate seal tilt; simulating edge termination by randomly occluding local areas of the image; and creating complex, deceptive edge structures by extracting normal seal textures and repeatedly pasting them to generate digitally synthesized secondary textures. The resulting enhanced target seal sample set is then obtained. The enhanced target seal sample set image input is embedded in the seal state recognition model of the multi-scale edge perception module MEM. The model performs forward transmission and outputs the prediction result and the intermediate edge feature map. The prediction result includes the seal position and integrity status category of the target seal.
[0040] S402. Calculate the composite loss function value.
[0041] The composite loss function value is achieved by weighted summation of the first loss term and the second loss term.
[0042] The first loss term is the bipartite graph matching loss based on the object detection prediction result and the ground truth bounding box. The second loss term is the edge consistency regularization term, which is obtained by calculating the difference between the intermediate edge feature map and the ground truth edge map generated by the coordinates of the key edge points.
[0043] S404. Based on the composite loss function value, update the parameters of the seal status recognition model through the backpropagation algorithm.
[0044] Specifically, the model calculates the gradient of the composite loss value with respect to each trainable parameter in the model through the backpropagation algorithm, where the gradient is a vector pointing in the direction of the fastest growth of the loss function; the optimizer in the model accepts the gradients of all parameters and applies their update rules to update the parameters in the model, completing the training iteration of the model.
[0045] S106. Input the initial seal state image into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model.
[0046] The target seal detection results include at least the target seal's detection frame coordinates, integrity status category, and anomaly type.
[0047] Specifically, such as Figure 5 The diagram shown is another flowchart of the package seal identification method according to an embodiment of this application. This flowchart is an exemplary illustration of step S106 in this method. (See attached diagram.) Figure 5 It includes the following steps: S500. Input the initial seal state image into the trained DETR-MEM seal state recognition model so that the model can preprocess the initial seal state image and then recognize it to obtain the original prediction sequence output by the model.
[0048] Specifically, the initial seal status image is input into the seal status recognition model. The model performs preprocessing on the initial seal status image, such as size standardization, tensor transformation, and numerical normalization. The preprocessed data passes through the CNN backbone network, the multi-scale edge perception module MEM, and the Transformer encoder-decoder in sequence, and finally outputs the original prediction sequence through the prediction output layer.
[0049] S502. After parsing the original prediction sequence into target seal detection results including seal detection box coordinates, integrity status label category, and anomaly type, output the target seal detection results and their corresponding confidence scores.
[0050] S108. Obtain the confidence level preset threshold and the abnormal state preset list.
[0051] S110. When the confidence score of the target seal is higher than the preset confidence threshold, the integrity status category output by the model is matched with the preset list of abnormal statuses. When the integrity status category exists in the preset list of abnormal statuses, the target seal status is determined to be abnormal and an early warning is triggered.
[0052] Specifically, such as Figure 6 The diagram shown is another flowchart of the package seal identification method according to an embodiment of this application. This flowchart is an exemplary illustration of step S110 in this method. (See attached diagram.) Figure 6 It includes the following steps: S600. Obtain the confidence score and integrity status category of the target seal output by the DETR-MEM seal status recognition model.
[0053] S602. Compare the confidence score with the preset confidence threshold.
[0054] S604. When the confidence score is higher than the confidence preset threshold, the integrity status category is matched with the abnormal status preset list.
[0055] S606. When the integrity status category exists in the abnormal status preset list, generate a seal status abnormality judgment instruction.
[0056] S608, based on the abnormal seal status judgment instruction, triggers system warning.
[0057] In one instance, Figure 7 Another flowchart of the package seal identification method according to an embodiment of this application includes the following steps: S700 When the confidence score is higher than the confidence preset threshold, but the integrity status category does not exist in the abnormal status preset list, the target seal status is determined to be normal and the detection result is recorded.
[0058] Specifically, if the confidence score is higher than the preset confidence threshold, it means that the model is very confident in its judgment. At the same time, if the integrity status category does not exist in the preset list of abnormal statuses, it means that the current seal status type does not belong to the abnormal situation that requires an alarm. It may be a normal situation, or it may include some minor abnormal statuses that are defined as not requiring immediate alarms. The system allows the detection and records the complete detection event in a structured way. The recorded content usually includes: timestamp, image ID / snapshot, detected coordinates, status category, confidence score, judgment result, etc., forming a complete audit log and data traceability chain.
[0059] S702. When the confidence score is lower than the preset confidence threshold, the target seal status is marked as uncertain, triggering the manual review process and recording and archiving.
[0060] Specifically, if the confidence score is lower than the preset confidence threshold, it means that the model prediction is unreliable or uncertain. If an alert is triggered based on an unreliable result, it will cause a large number of false alarms and seriously disrupt the normal production process. In order to prevent the model from making erroneous alerts when it is "unsure" and to ensure the reliability of the system, this detection record is marked as uncertain and a manual review process is triggered to notify the operator to conduct a key inspection of the current package.
[0061] Furthermore, Figure 8 The following is a structural block diagram of the package seal identification device according to an embodiment of this application, as shown below. Figure 8 As shown, the device includes: The sample dataset acquisition module is used to collect the initial seal state image of the target seal, and to annotate the initial seal state image with key information to obtain the target seal sample dataset. The key information includes at least the boundary polygon, integrity status label and key edge point coordinates. A multi-scale edge perception module (MEM) is embedded to build a seal state recognition model based on the DETR architecture and to embed the multi-scale edge perception module between its CNN backbone network and Transformer encoder. The seal status recognition model training module is used to train the seal status recognition model embedded with the multi-scale edge perception module MEM based on the target seal sample dataset, so as to obtain the trained DETR-MEM seal status recognition model. The loss function used for training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of key edge points. The result output module is used to input the initial seal state image into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category and anomaly type of the target seal. The preset information acquisition module is used to acquire the preset confidence threshold and the preset list of abnormal states; The target seal status determination module is used to match the integrity status category output by the model with the abnormal status preset list when the confidence score of the target seal is higher than the confidence threshold. When the integrity status category exists in the abnormal status preset list, the target seal status is determined to be abnormal and an early warning is triggered.
[0062] Furthermore, the sample dataset acquisition module includes: The image acquisition unit is used to acquire images of packages with different sealing conditions, including but not limited to normal sealing, slight offset, partial tearing, complete detachment, re-adhesion, and damage / obstruction. The rule determination unit is used to clarify the rules for labeling key information. The rules for labeling key information include the rules for labeling boundary polygons, the classification criteria for integrity labels, and the selection specifications for key edge points. The information annotation unit is used to annotate the initial seal status image with key information based on key information annotation rules, so as to obtain an annotated seal status image. The data cleaning unit is used to perform intelligent cleaning and security desensitization processing on the initially standardized data in parallel to obtain the initial pre-processed data; The standardization transformation unit is used to clean the labeled seal status images, remove incorrectly labeled or incomplete samples, and obtain the target seal sample dataset.
[0063] Furthermore, the multi-scale edge perception module embedding module includes: The model building unit is used to build a seal status recognition model based on the DETR architecture. The model includes a CNN backbone network, a Transformer encoder, a Transformer decoder, and a prediction output layer. The multi-scale feature map acquisition unit is used to input the target seal sample dataset into the CNN backbone network to acquire multi-scale feature maps, which have different spatial resolutions and semantic information. A multi-scale edge sensing module introduction unit is used to introduce a multi-scale edge sensing module MEM. The edge response map acquisition unit is used to calculate the gradient magnitude of each scale on the multi-scale feature map based on the Sobel edge detection algorithm, and obtain the Sobel edge response map that reflects the overall outline and direction of the target seal at that scale. Based on the Canny edge detection algorithm, Gaussian filtering is performed on the feature map at this scale for noise reduction, gradient calculation and double threshold filtering to obtain the Canny edge response map of the target seal for minor fractures, burrs and continuous edges. The image fusion unit is used to perform channel stitching and fusion of the Slbel edge response map and the Canny edge response map at the same scale with their corresponding initial sealing state image to obtain the edge enhancement features at that scale. The module integration unit is used to fuse edge enhancement features at all scales to obtain comprehensive edge enhancement features. The comprehensive edge enhancement features are then input into the Transformer encoder to complete the integration of the multi-scale edge perception module MEM and the DETR model.
[0064] Furthermore, in some embodiments, the seal status recognition model training module includes: The target detection prediction result and intermediate edge feature map acquisition unit is used to input the images in the target seal sample dataset into the seal status recognition model and obtain the target detection prediction result and intermediate edge feature map output by the model. The intermediate edge feature map is generated by the multi-scale edge perception module MEM. The composite loss function value calculation unit is used to calculate the composite loss function value, which is achieved by weighted summation of the first loss term and the second loss term. The first loss term is the bipartite graph matching loss based on the object detection prediction result and the ground truth bounding box, and the second loss term is the edge consistency regularization term, which is obtained by calculating the difference between the intermediate edge feature map and the edge ground truth map generated by the coordinates of the key edge points. The model parameter update unit is used to update the parameters of the seal status recognition model based on the composite loss function value through the backpropagation algorithm. Furthermore, in some embodiments, the result output module includes: The original prediction sequence acquisition unit is used to input the initial seal state image into the trained DETR-MEM seal state recognition model so that the model can preprocess the initial seal state image and then recognize it to obtain the original prediction sequence output by the model. The prediction sequence parsing unit is used to parse the original prediction sequence into target seal detection results including seal detection box coordinates, integrity status label category, and anomaly type, and then output the target seal detection results and their corresponding confidence scores. Furthermore, in some embodiments, the target seal status determination module includes: The data comparison unit is used to compare the confidence score with a preset confidence threshold. The status matching unit is used to match the integrity status category with the abnormal status preset list when the confidence score is higher than the confidence preset threshold. The status anomaly determination instruction generation unit is used to generate a seal status anomaly determination instruction when the integrity status category exists in the anomaly status preset list. The early warning triggering unit is used to trigger system early warnings based on the abnormal seal status determination command.
[0065] Furthermore, in some embodiments, the apparatus further includes: The normal state determination and recording module is used to determine that the target seal is in a normal state and record the detection result when the confidence score is higher than the confidence preset threshold, but the integrity state category does not exist in the abnormal state preset list. The uncertainty status determination module is used to mark the target seal status as uncertain when the confidence score is lower than the preset confidence threshold, triggering the manual review process and recording and archiving the information.
[0066] This solution introduces a multi-scale edge perception module to generate edge enhancement features, enabling the model to focus on the seal's contour and continuity, overcoming interference from low contrast. By fusing features from different levels of CNN, the model possesses both detail perception and semantic understanding, accurately distinguishing between normal wear and abnormal tears, and between natural textures and re-pasting marks, significantly reducing missed detections and false positives. A dual-judgment logic based on confidence thresholds and an anomaly list is introduced. The confidence threshold effectively filters out uncertain and fuzzy predictions from the model, taking action only on high-confidence scores, controlling false alarms at the source. The anomaly list directly maps technical labels to business actions, achieving end-to-end automation from recognition to execution without manual intervention, avoiding bias caused by human subjectivity. The processing of confidence scores ensures system safety redundancy, preventing false alarms due to model uncertainty and guaranteeing the reliability of seal recognition.
[0067] The application of the relevant modules of the device in this example can be referred to the relevant introduction of the method principle above, and will not be repeated here.
[0068] above Figure 8 The package seal identification device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The electronic device in this embodiment of the invention will be described in detail from the perspective of hardware processing.
[0069] Figure 9 This is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of the present invention. The electronic device 900 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 933 or data 932. The memory 920 and storage media 930 may be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the electronic device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the electronic device 900.
[0070] Electronic device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, MacOSX, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 9The illustrated electronic device structure does not constitute a limitation on electronic devices and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.
[0071] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the package seal identification method.
[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0073] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying package seals, characterized in that, The method includes: Acquire an initial seal state image of the target seal, and annotate the initial seal state image with key information to obtain a target seal sample dataset. The key information includes at least the boundary polygon, integrity status label, and key edge point coordinates. A seal status recognition model based on the DETR architecture is constructed, and a multi-scale edge perception module MEM is embedded between its CNN backbone network and Transformer encoder. The DETR-MEM seal state recognition model, which embeds a multi-scale edge perception module (MEM), is trained based on the target seal sample dataset to obtain a trained DETR-MEM seal state recognition model. The loss function used in the training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of the key edge points. The initial seal state image is input into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category and anomaly type of the target seal. Obtain the preset confidence threshold and the preset list of abnormal states; When the confidence score of the target seal is higher than the preset confidence threshold, the integrity status category output by the model is matched with the preset list of abnormal statuses. When the integrity status category exists in the preset list of abnormal statuses, the target seal is determined to be in an abnormal state, and an early warning is triggered.
2. The package seal identification method according to claim 1, characterized in that, The initial seal state image of the target seal is collected, and key information is annotated on the initial seal state image to obtain a target seal sample dataset. The key information includes at least the boundary polygon, integrity status label, and key edge point coordinates, including: Collect images of packages with different seal states, including but not limited to normal sealing, slight offset, partial tearing, complete detachment, re-attachment, and soiling / obstruction; Define the rules for labeling key information, including the rules for labeling boundary polygons, the classification criteria for integrity labels, and the selection specifications for key edge points; The initial seal status image is annotated with key information based on key information annotation rules to obtain an annotated seal status image. The labeled seal status images are cleaned to remove incorrectly labeled or incomplete samples, thus obtaining the target seal sample dataset.
3. The package seal identification method according to claim 1, characterized in that, The construction of the seal status recognition model based on the DETR architecture, and the embedding of a multi-scale edge perception module (MEM) between its CNN backbone network and Transformer encoder, includes: A seal status recognition model based on the DETR architecture is constructed. The model includes a CNN backbone network, a Transformer encoder, a Transformer decoder, and a prediction output layer. The target seal sample dataset is input into the CNN backbone network to obtain multi-scale feature maps, which have different spatial resolutions and semantic information. A multi-scale edge sensing module (MEM) is introduced. The gradient magnitude of each scale on the multi-scale feature map is calculated based on the Sobel edge detection algorithm to obtain the Sobel edge response map that reflects the overall outline and direction of the target seal at that scale. The feature map at that scale is then subjected to Gaussian filtering for noise reduction, gradient calculation and double threshold filtering based on the Canny edge detection algorithm to obtain the Canny edge response map of the target seal for minor breaks, burrs and continuous edges. The Slbel edge response map and the Canny edge response map at the same scale are channel-stitched and fused with their corresponding initial seal state images to obtain the edge enhancement features at that scale. The edge enhancement features from all scales are fused to obtain a comprehensive edge enhancement feature. This comprehensive edge enhancement feature is then input into the Transformer encoder to complete the integration of the multi-scale edge perception module MEM with the DETR model.
4. The package seal identification method according to claim 1, characterized in that, The seal state recognition model embedded with a multi-scale edge perception module (MEM) is trained based on the target seal sample dataset to obtain a trained DETR-MEM seal state recognition model. The loss function used in training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of the key edge points, including: The images in the target seal sample dataset are input into the seal status recognition model to obtain the target detection prediction results and intermediate edge feature maps output by the model. The intermediate edge feature maps are generated by the multi-scale edge perception module MEM. The composite loss function value is calculated by weighted summation of a first loss term and a second loss term, wherein: the first loss term is a bipartite graph matching loss based on the target detection prediction result and the ground truth bounding box; the second loss term is an edge consistency regularization term, which is obtained by calculating the difference between the intermediate edge feature map and the ground truth edge map generated from the coordinates of the key edge points. Based on the composite loss function value, the parameters of the seal status recognition model are updated using the backpropagation algorithm.
5. The package seal identification method according to claim 1, characterized in that, The initial seal state image is input into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category, and anomaly type of the target seal. The initial seal state image is input into the trained DETR-MEM seal state recognition model so that the model can preprocess the initial seal state image and then recognize it to obtain the original prediction sequence output by the model. After parsing the original predicted sequence into target seal detection results including seal detection box coordinates, integrity status label category, and anomaly type, the target seal detection results and their corresponding confidence scores are output.
6. The package seal identification method according to claim 1, characterized in that, When the confidence score of the target seal is higher than the preset confidence threshold, the integrity status category output by the model is matched with the preset list of abnormal statuses. If the integrity status category exists in the preset list of abnormal statuses, the target seal is determined to be in an abnormal state, and an early warning is triggered, including: Obtain the confidence score and integrity status category of the target seal output by the DETR-MEM seal status recognition model; The confidence score is compared with a preset confidence threshold; When the confidence score is higher than the preset confidence threshold, the integrity status category is matched with the preset list of abnormal statuses; When the integrity status category exists in the abnormal status preset list, a seal status abnormality determination instruction is generated; The system alert is triggered based on the abnormal seal status determination instruction.
7. The package seal identification method according to claim 1, characterized in that, The method further includes: When the confidence score is higher than the confidence preset threshold, but the integrity status category does not exist in the abnormal status preset list, the target seal status is determined to be normal and the detection result is recorded; When the confidence score is lower than the preset confidence threshold, the target seal status is marked as uncertain, triggering a manual review process and recording and archiving.
8. A package seal identification device, characterized in that, The device includes: The sample dataset acquisition module is used to acquire an initial seal state image of the target seal, and to annotate the initial seal state image with key information to obtain a target seal sample dataset. The key information includes at least the boundary polygon, integrity status label and key edge point coordinates. A multi-scale edge perception module (MEM) is embedded to build a seal state recognition model based on the DETR architecture and to embed the multi-scale edge perception module between its CNN backbone network and Transformer encoder. The seal status recognition model training module is used to train the seal status recognition model embedded with the multi-scale edge perception module MEM based on the target seal sample dataset to obtain the trained DETR-MEM seal status recognition model. The loss function used for training includes an edge consistency regularization term calculated based on the edge features predicted by the model and the coordinates of the key edge points. The result output module is used to input the initial seal state image into the trained DETR-MEM seal state recognition model to obtain the target seal detection result and its corresponding confidence score output by the model. The target seal detection result includes at least the detection box coordinates, integrity state category and anomaly type of the target seal. The preset information acquisition module is used to acquire the preset confidence threshold and the preset list of abnormal states; The target seal status determination module is used to match the integrity status category output by the model with the abnormal status preset list when the confidence score of the target seal is higher than the confidence preset threshold. When the integrity status category exists in the abnormal status preset list, the target seal status is determined to be abnormal and an early warning is triggered.
9. An electronic device, characterized in that, The electronic device includes a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to perform the steps of the package seal identification method as described in any one of claims 1-7.
10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the package seal identification method as described in any one of claims 1-7.