Logistics anomaly detection method based on DINOv2

Abstract

The invention provides a DINOv2-based logistics anomaly detection method, and the method comprises the steps: obtaining an original image of a logistics scene, and carrying out the preprocessing of the original image, so as to generate a unified-format image suitable for model input; inputting the unified format image into a pre-trained teacher network and a student network trained by a normal sample at the same time, and synchronously extracting local block features and global category token features from a plurality of intermediate Transform layers of the teacher network and the student network; performing fusion based on the difference between the local block features output by the teacher network and the student network and the difference between the global category token features to calculate an abnormal score; and when the fusion anomaly score exceeds a preset threshold value, judging that the original image is abnormal. According to the method, only normal sample training is needed, multi-scale local and global features are fused through a teacher-student network, local subtle and global structure abnormity are detected at the same time, and the detection precision, deployment efficiency and system flexibility are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of logistics automation and intelligent detection technology, and in particular to a logistics anomaly detection method based on DINOv2. Background Technology

[0002] With the rapid development of e-commerce and smart logistics, the demand for automated and intelligent anomaly detection in logistics systems is becoming increasingly urgent. Traditional manual visual inspection methods suffer from problems such as low efficiency, fatigue, and inconsistent standards, making them difficult to meet the requirements of modern logistics' high-speed sorting, warehousing, and transportation.

[0003] In recent years, deep learning-based anomaly detection technologies have been gradually applied in industrial scenarios. One type of method relies on classification models trained on a large number of normal and abnormal samples; however, this is difficult to apply in scenarios where abnormal samples are scarce. Another type of method is based on reconstruction or generative models, which identify anomalies through reconstruction errors, but is often insensitive to subtle local defects. Furthermore, some methods attempt to utilize pre-trained deep features for anomaly measurement, but in complex and ever-changing logistics environments, their feature adaptability and detection accuracy still need improvement.

[0004] However, existing methods still have significant shortcomings in logistics scenarios: they lack image feature optimization for multiple logistics stages (such as warehousing, sorting, and packaging); they do not fully consider the problem of scarce and diverse abnormal samples in logistics scenarios; the models are highly complex and difficult to deploy on edge devices; at the same time, they are not capable of detecting local minor defects and global structural anomalies, and cannot flexibly adapt to the detection needs of different logistics stages. Summary of the Invention

[0005] In view of the above-mentioned deficiencies of the prior art, the present invention provides a logistics anomaly detection method based on DINOv2 to solve the technical problems of low efficiency and weak adaptability of anomaly detection in logistics scenarios.

[0006] To achieve the above and other related objectives, this invention provides a logistics anomaly detection method based on DINOv2, comprising: acquiring an original image of a logistics scene and preprocessing the original image to generate a uniform format image suitable for model input; simultaneously inputting the uniform format image into a pre-trained teacher network and a student network trained with normal samples, and simultaneously extracting local block features and global category token features from multiple intermediate Transformer layers of the teacher network and the student network; fusing the differences between the local block features output by the teacher network and the student network and the differences between the global category token features to calculate an anomaly score; and determining that the original image is abnormal when the fused anomaly score exceeds a preset threshold.

[0007] In one embodiment of the present invention, preprocessing the original image to generate a uniform format image suitable for model input includes: performing initial normalization processing on the original image and scaling it to a first preset size to obtain an initial image; performing dynamic region cropping on the initial image to obtain a cropped image containing only cargo regions; performing multi-objective adaptive segmentation on the cropped image to obtain multiple independent single cargo sub-images of different sizes; performing embossing enhancement processing on each single cargo sub-image; and uniformly scaling each single cargo sub-image after embossing feature enhancement processing to a second preset size required for model input as the uniform format image.

[0008] In one embodiment of the present invention, dynamic region cropping is performed on the initial image to obtain a cropped image containing only the goods region, including: calculating the sum of gray levels in the row direction of the initial image and locating the effective row range where the gray level value is greater than a set threshold to obtain first upper and lower boundaries where the sum of gray levels begins to rise and fall significantly; calculating the sum of absolute gradient values ​​in the row direction of the initial image, performing Gaussian smoothing on the sum of absolute gradient values, and locating the second upper and lower boundaries of the target using a peak detection algorithm; obtaining the cropping region boundary based on the first and second upper and lower boundaries; and cropping the initial image based on the cropping region boundary to obtain the cropped image.

[0009] In one embodiment of the present invention, multi-target adaptive segmentation is performed on the cropped image to obtain multiple independent single-item sub-images of different sizes, including: binarizing the cropped image and extracting contours from the image; selecting valid contours with an area not less than a preset area threshold from the extracted contours; segmenting the corresponding single-item sub-image from the cropped image based on the bounding rectangle of the valid contours; if no valid contours are selected, the cropped image itself is used as the single-item sub-image.

[0010] In one embodiment of the present invention, the embossing enhancement process for each of the individual cargo sub-images includes: performing embossing feature enhancement on the individual cargo sub-images by multi-directional illumination fusion to generate an initial embossing feature map; and performing noise suppression and contrast optimization on the initial embossing feature map to generate a final embossing feature map.

[0011] In one embodiment of the present invention, the single-item sub-image is subjected to multi-directional illumination fusion for relief feature enhancement to generate an initial relief feature map, including: converting the single-item sub-image into a grayscale image, and performing contrast enhancement and smoothing processing on the grayscale image to generate a height map; calculating the gradient of the height map in the horizontal and vertical directions to characterize the texture change rate of the height map surface; simulating multiple light sources in different directions to illuminate the height map based on the gradient, and calculating the illumination intensity in each direction based on the angle between the gradient and the illumination direction; taking the maximum value of the illumination intensity in all directions at each pixel position, and fusing them to generate the initial relief feature map.

[0012] In one embodiment of the present invention, noise suppression and contrast optimization of the initial relief feature map include: performing adaptive histogram equalization on the initial relief feature map to improve local texture contrast; and normalizing the pixel values ​​of the equalized relief feature map to a preset standard grayscale range.

[0013] In one embodiment of the present invention, the teacher network is a pre-trained DINOv2-ViT-B model, and the student network is a trained DINOv2-ViT-S model; local block features and global category token features are extracted synchronously from multiple intermediate Transformer layers of the teacher network and the student network, including: synchronously extracting the outputs of the 4th, 8th, and 12th layers of the Transformer encoder from the teacher network and the student network; for the output of each layer, the category token output at the beginning of the sequence is used as the global category token feature, and all image block token outputs in the sequence other than the category token are used as the local block features.

[0014] In one embodiment of the present invention, the calculation of the total loss function during the training of the student network includes the following steps: Local block features from the corresponding intermediate layers of the teacher network and the student network are dimensionality-reduced and upsampled to a preset number of channels and resolution using convolutional layers; global category token features from the corresponding intermediate layers of the teacher network and the student network are dimensionality-reduced to the same number of channels as the local block features using fully connected layers; local feature loss is calculated between the dimensionality-reduced local block features of the student network and the local block features of the teacher network; global feature loss is calculated between the dimensionality-reduced global category token features of the student network and the global category token features of the teacher network; the total loss function is obtained by summing the local feature loss and the dynamically weighted global feature loss; wherein the dynamic weights are preset according to the logistics scenario type, taking values ​​of 0.3 to 0.6 in scenarios where global anomalies are predominant, and taking values ​​of 0.1 to 0.3 in scenarios where local anomalies are predominant.

[0015] In one embodiment of the present invention, an anomaly score is calculated by fusing the differences between the local block features output by the teacher network and the student network, and the differences between the global category token features. This includes: for each of the multiple intermediate Transformer layers, calculating the differences between the local block features of the student network and the teacher network, and the differences between the global category token features, as local difference maps and global difference values ​​for that layer; performing element-wise multiplication on the local difference maps of each layer and aggregating them spatially to obtain a local anomaly score; multiplying the global difference values ​​of each layer to obtain a global anomaly score; and obtaining the anomaly score by multiplying the local anomaly score by the dynamically weighted global anomaly score.

[0016] The beneficial effects of this invention are as follows: This invention proposes a logistics anomaly detection method based on DINOv2. This method only requires normal samples to complete training, effectively solving the problem of scarce abnormal samples in logistics scenarios and significantly reducing data annotation costs. By constructing a teacher-student dual-branch network structure and integrating local block features and global category token features from multiple intermediate layers, it can simultaneously capture local subtle anomalies such as packaging damage and label detachment, as well as global structural anomalies such as shelf tilting and disordered stacking, thereby comprehensively improving detection accuracy and scenario adaptability. In addition, by introducing a dynamic weight mechanism, the detection strategy can be adaptively adjusted according to the anomaly characteristics of different logistics links, without the need for repeated model training, greatly improving deployment efficiency and system flexibility. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a logistics anomaly detection method provided in an embodiment of the present invention; Figure 2 This is a flowchart of raw image preprocessing provided in an embodiment of the present invention; Figure 3 A flowchart of dynamic region clipping provided in an embodiment of the present invention; Figure 4 A flowchart of multi-objective adaptive segmentation provided in an embodiment of the present invention; Figure 5A flowchart of an embodiment of the relief enhancement process provided by the present invention; Figure 6 A flowchart illustrating the generation of an initial relief feature map according to an embodiment of the present invention; Figure 7 An optimization flowchart of the initial relief feature map provided in an embodiment of the present invention; Figure 8 This is a flowchart of model feature extraction provided in an embodiment of the present invention; Figure 9 A flowchart illustrating the calculation of the total loss function provided in an embodiment of the present invention; Figure 10 This is a flowchart of an anomaly score calculation provided in an embodiment of the present invention. Detailed Implementation

[0019] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0020] It should be understood that the terminology used in the embodiments of this invention is for describing specific implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0023] Please see Figure 1 , Figure 1 An embodiment of the present invention provides a logistics anomaly detection method based on DINOv2, comprising steps S101 to S104.

[0024] Step S101: Acquire the original images of the logistics scene and preprocess the original images to generate a uniform format image suitable for model input. In this step, the logistics images can be, for example, images of different scenes (such as normally stacked shelves, well-packaged goods, and normally operating sorting lines) captured by cameras deployed in the logistics scene (such as warehouse rack cameras, sorting line barcode scanners, and in-vehicle monitoring). Alternatively, each frame of the captured video can also be used as the original image.

[0025] Please see Figure 2 In a specific embodiment of the present invention, the original image is preprocessed to generate a uniform format image suitable for model input, including steps S201 to S205.

[0026] Step S201: Perform initial normalization processing on the original image, scaling it to a first preset size to obtain the initial image. This first normalization process is to accelerate subsequent calculations and unify the data format. Subsequent operations such as cropping and segmentation are performed at a fixed and appropriate resolution, which is much faster than directly processing the original high-resolution image.

[0027] S202. Perform dynamic region cropping on the initial image to obtain a cropped image that only contains the cargo area.

[0028] Please see Figure 3 In a specific embodiment of the present invention, step S202 includes steps S301 to S304.

[0029] Step S301: Calculate the total grayscale value of the initial image along the row direction and locate the effective row range where the grayscale value is greater than a set threshold, thus obtaining the first upper and lower boundaries where the total grayscale value begins to rise and fall significantly. By calculating the total grayscale value of the image along the row direction and locating the effective row range based on a set threshold (e.g., grayscale value > 40), the upper and lower boundaries of the goods area are initially determined. This method can quickly filter out dark backgrounds (such as warehouse floors) and highlight brighter goods areas, providing a foundation for subsequent precise cropping and effectively improving the efficiency and accuracy of subsequent processing steps.

[0030] Step S302: Calculate the sum of absolute gradient values ​​in the row direction of the initial image, perform Gaussian smoothing on the sum of absolute gradient values, and use a peak detection algorithm to locate the second set of upper and lower boundaries of the target. By calculating the sum of absolute gradient values ​​in the row direction of the image, performing Gaussian smoothing and peak detection, the second set of upper and lower boundaries of the target are located. Gradient information can effectively capture the edge and texture changes of the goods. Even if the goods and background colors are similar, the target range can be accurately identified through structural features, forming an effective complement to grayscale features and improving the robustness of boundary localization.

[0031] Step S303: Obtain the cropping region boundary based on the first and second upper and lower boundaries. The final cropping region boundary is determined by taking the intersection of the first and second upper and lower boundaries. This fusion strategy ensures that the cropping region simultaneously meets the requirements for brightness and structural features, maximizing the retention of the effective cargo area and eliminating irrelevant background, thus providing high-quality input for subsequent segmentation and feature extraction.

[0032] Step S304: Crop the initial image according to the cropping region boundaries to obtain a cropped image. Cropping the initial image according to the determined cropping region boundaries yields an image containing only the core cargo area. This step significantly reduces the amount of data required for subsequent processing, lowers computational complexity, and avoids background interference, allowing the model to focus more on feature learning and anomaly detection of the cargo itself.

[0033] In one specific embodiment of the present invention, to avoid over-cropping, a judgment can be added: if the cropped area is too small (e.g., smaller than the original area)... Figure 1 / 2), automatically restore to the original image area. The fault tolerance mechanism of "restoring the original image if the cropped area is too small" is to prevent over-cropping in certain special scenarios (such as when the goods color is extremely similar to the background, or when the image quality is too low), thus avoiding the loss of key targets. This mechanism ensures the robustness of the preprocessing workflow, ensuring stable operation of the system even in complex environments.

[0034] Step S203: Perform multi-target adaptive segmentation on the cropped image to obtain multiple independent single-item sub-images of different sizes.

[0035] Please see Figure 4 In a specific embodiment of the present invention, step S203 includes: S401, binarizing the cropped image and extracting contours from the image; S402, selecting valid contours with an area not less than a preset area threshold from the extracted contours; S403, segmenting the corresponding single-item sub-image from the cropped image based on the bounding rectangle of the valid contours; S404, if no valid contours are selected, the cropped image itself is used as the single-item sub-image. This embodiment divides multiple goods in the image into independent single-target sub-images by binarizing the cropped image, extracting contours, and selecting valid targets. By setting an area threshold to filter noise and cropping sub-images based on bounding rectangles, feature interference between multiple goods is effectively avoided. The fault-tolerant design ensures that the original image can still be retained as input when segmentation fails, guaranteeing the continuity of the process.

[0036] Step S204: Perform embossing enhancement on each individual cargo sub-image. Embossing enhancement on each individual cargo sub-image aims to enhance the texture details of the cargo surface. This process, through simulating multi-directional lighting and contrast optimization, highlights microscopic features such as packaging wrinkles, indentations, and tears, enabling the subsequent feature extraction network to more accurately capture key information identifying anomalies.

[0037] Please see Figure 5 In a specific embodiment of the present invention, step S204 includes steps S501 and S502.

[0038] Step S501: Enhance the embossed features of a single cargo sub-image by multi-directional lighting fusion to generate an initial embossed feature map. By converting the sub-image into a height map and simulating multi-directional lighting based on gradients, a comprehensive embossed feature map is generated. This method can comprehensively enhance texture features in different directions, ensuring that details such as indentations on cartons and label edges are effectively extracted, overcoming the feature omissions that may be caused by single-directional lighting.

[0039] Please see Figure 6 In a specific embodiment of the present invention, step S501 includes: S601, converting a single product sub-image into a grayscale image, and performing contrast enhancement and smoothing processing on the grayscale image to generate a height map; S602, calculating the gradient of the height map in the horizontal and vertical directions to characterize the texture change rate of the height map surface; S603, based on the gradient, simulating multiple light sources from different directions to illuminate the height map, and calculating the illumination intensity in each direction based on the angle between the gradient and the illumination direction; S604, taking the maximum value of the illumination intensity in all directions at each pixel position, and fusing them to generate an initial relief feature map.

[0040] In this embodiment, the multiple light sources in different directions in step S603 can be, for example, eight uniformly distributed illumination directions, such as 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. In step S603, the illumination intensity in each direction can be calculated based on the cosine of the angle between the gradient and the illumination direction.

[0041] This process generates an initial relief feature map by constructing a height map, calculating gradients, simulating multi-directional lighting, and fusing the maximum values. The calculation of lighting intensity based on the angle between the gradient and the lighting direction aligns with the principles of photometric stereo vision, realistically reflecting surface undulations. The strategy of fusing the maximum values ​​ensures that significant textures from any direction are preserved, resulting in a detailed, orientation-insensitive enhanced feature map.

[0042] Step S502: Noise suppression and contrast optimization are performed on the initial relief feature map to generate the final relief feature map. In this step, local texture contrast is improved through adaptive histogram equalization, and the brightness range is unified through grayscale normalization. This effectively suppresses noise interference, enhances feature stability, and ensures the consistency of feature maps generated under different lighting conditions, providing a more reliable input for the model.

[0043] Please see Figure 7 In a specific embodiment of the present invention, noise suppression and contrast optimization are performed on the initial embossed feature map, including: S701, performing adaptive histogram equalization on the initial embossed feature map to improve local texture contrast; S702, normalizing the pixel values ​​of the equalized embossed feature map to a preset standard grayscale range (e.g., [0, 255]). By performing adaptive histogram equalization on the embossed feature map again, the grayscale distribution of local textures can be further stretched, making features such as label edges and minor packaging damage sharper. The subsequent grayscale normalization standardizes the pixel values ​​to a standard range, eliminating the interference of overall brightness differences on model judgment and improving the model's generalization ability under different imaging conditions.

[0044] Step S205: Scale each individual product sub-image after embossing feature enhancement to a second preset size required for model input, creating a uniform image format. This scaling step is necessary to match the input requirements of the deep learning model. Scaling each enhanced individual product sub-image to a fixed size required by the model is a crucial step in adapting to the deep learning model's input requirements. This operation resolves the image size inconsistency issue caused by dynamic cropping and segmentation, ensuring all input samples are aligned in spatial dimensions and satisfying the fixed input size constraints of the visual Transformer model.

[0045] Step S102: Simultaneously input the uniformly formatted image into both the pre-trained teacher network and the student network trained with normal samples. Then, simultaneously extract local block features and global category token features from multiple intermediate Transformer layers in both the teacher and student networks. The core of this method is to simultaneously input the uniformly formatted image into both the teacher and student networks and extract two types of features from the intermediate layers. The teacher network provides a reliable feature standard, and the student network learns to fit this standard through training. Extracting multi-level, multi-type features enables the model to simultaneously perceive local details and global structure, laying the foundation for comprehensive anomaly detection.

[0046] In this step, local patch features correspond to local details of the goods (such as packaging texture and label position); global class token features correspond to the overall structure of the goods (such as the number of shelf stacks and the orientation of packages). The shape of the local patch features extracted by the teacher network is [B,768,16,16], and the shape of the global class token features is [B,768]. The dimensions of these two features extracted by the student model are 384.

[0047] In a specific embodiment of the present invention, the teacher network is a pre-trained DINOv2-ViT-B model, and the student network is a trained DINOv2-ViT-S model. The teacher network, using the pre-trained DINOv2-ViT-B model, has a large number of parameters and strong feature extraction capabilities, providing a reliable reference as a frozen "standard feature library." The student network uses the lightweight DINOv2-ViT-S model, and through training, its output features are made as close as possible to those of the teacher network, significantly improving inference speed while maintaining accuracy, thus meeting the real-time detection requirements of logistics.

[0048] Please see Figure 8 In a specific embodiment of the present invention, local block features and global category token features are extracted synchronously from multiple intermediate Transformer layers of the teacher network and the student network, including: S801, extracting the outputs of the 4th, 8th and 12th layers of the Transformer encoder synchronously from the teacher network and the student network; S802, for the output of each layer, taking the category token output at the beginning of the sequence as the global category token feature, and taking all image block token outputs in the sequence other than the category token as local block features.

[0049] In the above steps, since the layers in the model are mostly indexed starting from 0, layers 4, 8, and 12 correspond to layers 3, 7, and 11 when calculated by index. For the teacher model, it has three layers, each corresponding to a local block feature and a global category token feature, so a teacher model can extract 6 features; similarly, a student model can also extract 6 features.

[0050] Please see Figure 9 In a specific embodiment of the present invention, the calculation of the total loss function during the training of the student network includes steps S901 to S905.

[0051] Step S901: Reduce the local block features from the intermediate layers of the teacher network and student network to 256 channels through convolutional layers, and upsample them to a preset number of channels and resolution (e.g., 64×64) to match the spatial resolution requirements of local anomalies in the logistics scenario.

[0052] Step S902: Reduce the global category token features from the corresponding intermediate layers of the teacher network and student network to the same number of channels as the local block features, i.e., 256 channels, through a fully connected layer.

[0053] Step S903: Calculate the local feature loss between the local block features of the reduced-dimensional student network and the local block features of the teacher network. The local feature loss is used to constrain the local feature matching between students and teachers, focusing on minor anomalies such as packaging tears and label detachment.

[0054] Step S904: Calculate the global feature loss between the global category token features of the reduced-dimensional student network and the global category token features of the teacher network. The global feature loss is used to constrain the global feature matching between students and teachers, focusing on structural anomalies such as shelf tipping and misaligned sorting lines.

[0055] Step S905: Obtain the total loss function by summing the local feature loss and the dynamically weighted global feature loss. This step can be expressed by the formula: Total Loss = Local Feature Loss + Dynamic Weights × Global Feature Loss.

[0056] In a specific embodiment of the present invention, the dynamic weights are preset according to the type of logistics scenario, taking values ​​of 0.3 to 0.6 in scenarios where global anomalies are predominant, and 0.1 to 0.3 in scenarios where local anomalies are predominant. The setting of dynamic weights is based on the anomaly characteristics of different logistics links. In warehousing / transportation scenarios, global structural anomalies such as shelf tipping and misaligned goods stacking pose a greater threat, so they are given higher weights for global losses; while in sorting / packaging links, more attention is paid to local detail anomalies such as damaged packaging and uneven labeling, so the global weights are reduced to highlight local feature matching. This adaptive mechanism allows the model to flexibly adapt to multiple scenarios without retraining.

[0057] Step S103: Calculate the anomaly score by fusing the differences between the local block features output by the teacher network and the student network, and the differences between the global category token features.

[0058] Please see Figure 10 In a specific embodiment of the present invention, step S103 includes steps S1001 to S1004.

[0059] Step S1001: For each of the multiple intermediate Transformer layers, calculate the differences between the local block features and the global category token features of the student network and the teacher network, respectively, as the local difference map and the global difference value of that layer. Local block features have spatial dimensions, and their difference calculation yields the difference value at each spatial location, thus forming a "local difference map"; while the global category token feature is a one-dimensional vector, and its difference calculation yields a scalar, hence called the "global difference value". The calculation process typically uses mean squared error or cosine distance to measure the difference between the corresponding feature vectors.

[0060] Step S1002: Element-wise multiplication of the local difference maps of each layer is performed and then aggregated in the spatial dimension to obtain the local anomaly score. Element-wise multiplication of the local difference maps of each layer is a non-linear fusion strategy that can amplify anomalous regions that show significant differences at all levels, thereby enhancing the model's sensitivity to persistent and significant local anomalies and improving the confidence of detection.

[0061] Step S1003: Multiply the global difference values ​​of each layer to obtain the global anomaly score. Multiplying the global difference values ​​of each layer aims to amplify the anomalous signals in the overall structure. If the goods deviate from the normal pattern at multiple semantic levels, their cumulative differences will be significantly amplified, making global anomalies such as overall overturning and severe misalignment reliably identifiable.

[0062] Step S1004: Obtain the anomaly score by multiplying the local anomaly score by the dynamically weighted global anomaly score. This step can be expressed as: Anomaly Score = Local Anomaly Score × (1 + Dynamic Weight × Global Anomaly Score). The intention behind this calculation is that the local anomaly score is the foundation, and when a global anomaly occurs, the dynamic weight λ acts as a multiplier, significantly improving the final score. This aligns with the logistics detection logic that "local anomalies require attention, while global anomalies pose a higher risk," ensuring that the scoring mechanism is consistent with the understanding of business risks.

[0063] Step S104: When the fusion anomaly score exceeds a preset threshold, the original image is determined to have an anomaly. When the fusion anomaly score exceeds a threshold determined by optimization methods such as ROC curves, the image is determined to have an anomaly. This determination mechanism is based on statistical learning principles, ensuring efficient detection with a controllable false alarm rate. After determining an anomaly, the system can trigger an audible and visual alarm, push alarm information to the monitoring center, or record the coordinates of the anomaly location for subsequent verification, thus forming a complete detection-early warning closed loop.

[0064] It should be noted that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0065] In summary, this invention designs a complete unsupervised anomaly detection process optimized for logistics scenarios. It comprehensively utilizes preprocessing techniques such as dynamic region clipping, multi-target segmentation, and embossed feature enhancement, along with a DINOv2-based dual-branch network and multi-level feature fusion mechanism, successfully achieving high-precision, adaptive logistics anomaly detection even under conditions of scarce anomaly samples. This method significantly improves accuracy, efficiency, and scenario adaptability, possessing significant industrial application value.

[0066] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A DINOv2-based logistics anomaly detection method, characterized in that, The method comprises the following steps: obtaining an original image of a logistics scene, and pre-processing the original image to generate a uniform format image suitable for model input; inputting the uniform format image into a pre-trained teacher network and a student network trained by normal samples at the same time, and synchronously extracting local block features and global class token features from multiple intermediate Transformer layers of the teacher network and the student network; fusing the differences between the local block features output by the teacher network and the student network, and the differences between the global class token features to calculate an anomaly score; when the fused anomaly score exceeds a preset threshold, determining that the original image is abnormal.

2. The DINOv2-based logistics anomaly detection method according to claim 1, characterized in that, The pre-processing of the original image to generate a uniform format image suitable for model input comprises: performing initial standardization processing on the original image to scale it to a first preset size to obtain an initial image; performing dynamic region cropping on the initial image to obtain a cropped image containing only the cargo region; performing multi-object adaptive segmentation on the cropped image to obtain multiple independent single-cargo subgraphs of different sizes; performing relief enhancement processing on each single-cargo subgraph; uniformly scaling each single-cargo subgraph after relief feature enhancement processing to a second preset size required by the model input to serve as the uniform format image.

3. The DINOv2-based logistics anomaly detection method according to claim 2, characterized in that, The dynamic region cropping of the initial image to obtain a cropped image containing only the cargo region comprises: calculating the total sum of gray values of the initial image in the row direction, and locating the effective row range with a gray value greater than a set threshold to obtain the first upper and lower boundaries where the total sum of gray values starts to rise and fall significantly; calculating the total sum of gradient absolute values of the initial image in the row direction, performing Gaussian smoothing on the total sum of gradient absolute values, and locating the second upper and lower boundaries of the target through a peak detection algorithm; obtaining the cropping region boundary according to the first upper and lower boundaries and the second upper and lower boundaries; cropping the initial image according to the cropping region boundary to obtain the cropped image.

4. The DINOv2-based logistics anomaly detection method according to claim 2, characterized in that, The multi-object adaptive segmentation of the cropped image to obtain multiple independent single-cargo subgraphs of different sizes comprises: performing binaryzation processing on the cropped image, and extracting the contours in the image; screening effective contours with an area not less than a preset area threshold from the extracted contours; segmenting the corresponding single-cargo subgraph from the cropped image based on the bounding rectangle of the effective contour; if no effective contour is screened, taking the cropped image itself as a single-cargo subgraph.

5. The DINOv2-based logistics anomaly detection method according to claim 2, characterized in that, The relief enhancement processing on each single-cargo subgraph comprises: performing multi-directional light fusion relief feature enhancement on the single-cargo subgraph to generate an initial relief feature map; performing noise suppression and contrast optimization on the initial relief feature map to generate a final relief feature map.

6. The DINOv2-based logistics anomaly detection method according to claim 5, characterized in that, The multi-directional light fusion relief feature enhancement on the single-cargo subgraph to generate an initial relief feature map comprises: converting the single-cargo subgraph into a gray-scale image, and performing contrast enhancement and smoothing processing on the gray-scale image to generate a height map; Gradient of the height map in horizontal and vertical directions is calculated to represent the rate of texture change of the height map surface; Based on the gradient, simulate the illumination of the height map by light sources in multiple different directions, and calculate the illumination intensity based on the gradient and the angle between the gradient and the light direction in each direction; Take the maximum value of the illumination intensity of all directions at each pixel position to generate the initial relief feature map.

7. The DINOv2-based logistics anomaly detection method according to claim 5, characterized in that, Noise suppression and contrast optimization are performed on the initial relief feature map, including: Adaptive histogram equalization is performed on the initial relief feature map to improve local texture contrast; The pixel values of the equalized relief feature map are normalized to a pre-set standard gray scale range.

8. The DINOv2-based logistics anomaly detection method of claim 1, wherein, The teacher network is a pre-trained DINOv2-ViT-B model, and the student network is a trained DINOv2-ViT-S model; From multiple intermediate Transformer layers of the teacher network and the student network, local block features and global class token features are synchronously extracted, including: The outputs of the 4th, 8th and 12th layers of the Transformer encoder are synchronously extracted from the teacher network and the student network; For the output of each layer, the class token at the beginning of the sequence is output as the global class token feature, and all image block token outputs in the sequence except the class token are output as the local block feature.

9. The DINOv2-based logistics anomaly detection method according to claim 8, characterized in that, When training the student network, the calculation of the total loss function includes the following steps: The local block features from the corresponding intermediate layers of the teacher network and the student network are respectively reduced in dimension by convolution layers and upsampled to a pre-set channel number and resolution; The global class token features from the corresponding intermediate layers of the teacher network and the student network are respectively reduced in dimension by fully connected layers to the same channel number as the local block features; The local feature loss between the reduced local block features of the student network and the local block features of the teacher network is calculated; The global feature loss between the reduced global class token features of the student network and the global class token features of the teacher network is calculated; The total loss function is obtained according to the sum of the local feature loss and the global feature loss weighted by a dynamic weight; Wherein, the dynamic weight is pre-set according to the logistics scene type, and the value is 0.3 to 0.6 in the scene where global anomaly is dominant, and the value is 0.1 to 0.3 in the scene where local anomaly is dominant.

10. The DINOv2-based logistics anomaly detection method of claim 9, wherein, Based on the differences between the local block features output by the teacher network and the student network, and the differences between the global class token features, an anomaly score is calculated, including: For each layer in the multiple intermediate Transformer layers, the differences between the local block features of the student network and the teacher network, and the differences between the global class token features, are calculated as the local difference map and the global difference value of the layer, respectively; The local difference maps of each layer are element-wise multiplied and aggregated in the spatial dimension to obtain the local anomaly score; The global difference values of each layer are multiplied to obtain the global anomaly score; The anomaly score is obtained according to a product of the local anomaly score and the global anomaly score weighted by the dynamic weight. The anomaly score is obtained according to a product of the local anomaly score and the global anomaly score weighted by the dynamic weight.

Images