A multi-source information checking system for product packaging boxes before warehousing

Abstract

The present application relates to the technical field of packaging box verification, and discloses a multi-source information verification system for product packaging boxes before storage. The system comprises a multi-source information acquisition module, which acquires packaging box stereo vision sequences, weight dynamic sampling and three-dimensional point cloud measurement data; a multi-modal feature learning module learns multi-scale space-time features from the vision data to generate vision space-time feature tensors; a dynamic weight analysis module decomposes the weight data in the frequency domain to generate weight frequency domain feature maps. A graph structure fusion module constructs a heterogeneous information graph from the two, generates a fusion verification graph feature through a graph neural network, an abnormal pattern recognition module generates an abnormal confidence distribution through a pre-trained variational autoencoder, an intelligent decision module outputs abnormal positioning and defect classification results accordingly, and an adaptive optimization module generates parameter adjustment strategies and transmits them to the storage system. The system realizes deep fusion of multi-source information, improves verification accuracy and intelligent level, and helps optimize the warehouse process.

Description

Technical Field

[0001] This invention relates to the field of packaging box verification technology, specifically a multi-source information verification system for product packaging boxes before they are put into storage. Background Technology

[0002] In modern logistics and manufacturing systems, the inspection of product packaging boxes upon arrival is a crucial link in ensuring the efficient operation of the supply chain and controllable product quality. With the expansion of the market and the diversification of product types, the specifications, materials, and packaging methods of packaging boxes are becoming increasingly complex, making traditional inspection methods inadequate for practical needs. Early on, the inspection of packaging boxes upon arrival relied heavily on manual labor, with inspectors judging whether the boxes met standards by visually inspecting their appearance and manually weighing them. This method not only consumes a significant amount of manpower but is also significantly affected by subjective human factors, easily leading to missed or incorrect inspections. This inefficiency is particularly pronounced in scenarios involving large volumes of goods continuously entering the warehouse, making it impossible to keep pace with automated production lines and intelligent warehousing systems.

[0003] To improve verification efficiency, some companies have introduced single-sensor verification equipment, such as using visual sensors to acquire images of the packaging box's appearance for defect detection, or using weight sensors to collect static weight data to determine if there are any missing or incorrect items in the packaging. However, verification methods based on a single data source have significant limitations. Relying solely on visual data makes it difficult to identify hidden problems such as abnormal product placement inside the packaging box or insufficient sealing tightness. These problems are often not directly manifested in the appearance but can affect subsequent storage and transportation safety. Relying solely on weight data makes it impossible to accurately locate explicit defects such as external damage or mislabeled labels, and static weight sampling is susceptible to interference from environmental vibrations and fluctuations in sensor accuracy, leading to reduced data reliability.

[0004] With the development of sensor technology, some verification systems have begun to attempt to fuse multi-source data. However, existing fusion methods mostly remain at the level of simple data overlay, failing to achieve deep feature-level fusion. For example, some systems simply combine visual inspection results with weight data through logical judgment, lacking the ability to explore the inherent correlations between the data. When feedback information from different data sources differs, the system struggles to make accurate judgments. Furthermore, existing anomaly detection algorithms are mostly based on traditional statistical models, which are poorly adaptable to nonlinear anomaly patterns in complex scenarios, and cannot effectively identify novel packaging defects or highly concealed anomalies. In addition, most verification systems can only output a simple "pass / fail" result, unable to pinpoint the specific packaging stage and defect type where the anomaly occurred. This necessitates additional manpower for investigation, increasing the overall cost of the verification process. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-source information verification system for product packaging boxes before they are put into storage, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a multi-source information verification system for product packaging boxes before they are put into storage, the system comprising:

[0007] The multi-source information acquisition module is used to acquire multi-source information data streams of the target product packaging box. The multi-source information data streams include stereoscopic visual sequence data of the packaging box, dynamic weight sampling data, and three-dimensional point cloud measurement data.

[0008] The multimodal feature learning module performs multi-scale spatiotemporal feature learning processing on the stereo vision sequence data to generate a visual spatiotemporal feature tensor.

[0009] The dynamic weight analysis module performs frequency domain feature decomposition processing on the dynamic weight sampling data to generate a weight frequency domain feature map.

[0010] The graph structure fusion module constructs a heterogeneous information graph by combining the visual spatiotemporal feature tensor with the weight frequency domain feature map, and generates a fusion verification graph feature by performing node feature propagation processing through a graph neural network.

[0011] The abnormal pattern recognition module calls a pre-trained variational autoencoder model to perform abnormal pattern reconstruction processing on the features of the fusion verification map, and generates an abnormal pattern confidence distribution, wherein the abnormal pattern confidence distribution reflects the abnormal correlation between the three-dimensional point cloud measurement data and the visual spatiotemporal features.

[0012] The intelligent decision-making module performs multi-criteria decision processing on the three-dimensional point cloud measurement data and stereo vision sequence data based on the confidence distribution of the anomaly pattern, and generates the verification decision result of the packaging box. The verification decision result is used to locate the packaging stage where the anomaly occurs and classify the defect.

[0013] The adaptive optimization module generates an adaptive adjustment strategy for packaging parameters based on the verification decision results, and transmits the adaptive adjustment strategy to the inbound execution system to initiate the parameter correction operation.

[0014] Preferably, the multimodal feature learning module includes:

[0015] The sequence segmentation unit performs time window division processing on the stereo vision sequence data to obtain multiple visual sequence segments;

[0016] The spatiotemporal coding unit calls a pre-trained 3D convolutional neural network model to perform spatiotemporal feature encoding on each visual sequence segment, generating a local spatiotemporal feature vector;

[0017] The graph construction unit treats each visual sequence segment as a graph node and constructs a visual sequence graph structure based on the spatiotemporal relationships between nodes.

[0018] The graph attention unit performs node feature enhancement processing on the visual sequence graph structure through a graph attention network, calculates the attention weight of each node, and generates an enhanced node feature set.

[0019] The global pooling unit performs global graph pooling on the enhanced node feature set to generate the visual spatiotemporal feature tensor.

[0020] Preferably, the dynamic weight analysis module includes:

[0021] The frequency domain transformation unit performs a fast Fourier transform on the dynamic weight sampling data to generate a weight frequency domain signal.

[0022] The frequency band decomposition unit decomposes the weight frequency domain signal into multiple preset frequency band sub-signals, and performs energy calculation processing on each frequency band sub-signal to generate a frequency band energy distribution vector;

[0023] The feature encoding unit calls a pre-trained autoencoder model to perform dimensionality reduction encoding on the frequency band energy distribution vector to generate a compressed frequency domain feature vector.

[0024] The spectrum generation unit maps the compressed frequency domain feature vector to a two-dimensional feature space and generates the weighted frequency domain feature spectrum through an interpolation algorithm.

[0025] Preferably, the graph structure fusion module includes:

[0026] The graph initialization unit defines each feature dimension of the visual spatiotemporal feature tensor as a graph node and each feature point of the weight frequency domain feature map as another type of graph node, thereby constructing an initial heterogeneous information graph.

[0027] Edge construction unit calculates the feature similarity between visual nodes and weight nodes, establishes cross-modal edge connections based on similarity thresholds, and generates heterogeneous graph edge sets;

[0028] The message passing unit performs multiple rounds of feature propagation processing on the heterogeneous graph through the message passing mechanism of the graph neural network to update node features;

[0029] The feature reading unit performs global graph pooling on the updated node features, extracts graph-level feature representations, and generates the fused verification graph features.

[0030] Preferably, the abnormal pattern recognition module includes:

[0031] The encoder unit inputs the fused verification map features into the encoder part of the variational autoencoder and performs latent space mapping processing to generate latent feature distribution parameters.

[0032] The sampling unit randomly samples potential feature vectors from the potential feature distribution parameters;

[0033] The decoder unit reconstructs the latent feature vector through the decoder part of the variational autoencoder to generate reconstructed verification features;

[0034] An anomaly scoring unit calculates the reconstruction error between the original fused verification map features and the reconstructed verification features, and generates the confidence distribution of the anomaly pattern based on the reconstruction error.

[0035] Preferably, the intelligent decision-making module includes:

[0036] The rule base construction unit extracts multi-criteria decision rules from historical validation data to build a decision rule base.

[0037] The feature mapping unit maps the anomaly pattern confidence distribution to the input space of the decision rule base to generate decision input features;

[0038] The rule triggering unit matches applicable rules from the decision rule base based on the decision input features, triggering a multi-criteria decision-making process.

[0039] The weight calculation unit calculates the dynamic weight of each decision criterion and performs weighted decision processing by combining the confidence of abnormal patterns.

[0040] The result generation unit outputs the weighted decision result as the verification decision result.

[0041] Preferably, the rule base construction unit includes:

[0042] The data mining unit performs association rule mining on historical verification data to extract frequent itemsets and association rules.

[0043] The rule optimization unit uses a genetic algorithm to perform multi-objective optimization on association rules and selects the Pareto optimal rule set.

[0044] The knowledge representation unit converts the Pareto optimal rule set into an executable decision rule base.

[0045] Preferably, the adaptive optimization module includes:

[0046] The strategy search unit performs Monte Carlo tree search processing in the strategy space based on the verification decision results to generate a set of candidate strategies;

[0047] The simulation evaluation unit performs a packaging process simulation for each candidate strategy and evaluates the strategy performance indicators.

[0048] The strategy selection unit selects the optimal candidate strategy based on performance indicators and uses it as the strategy for adaptive adjustment of packaging parameters.

[0049] The real-time deployment unit deploys adaptive adjustment strategies to the database execution system.

[0050] Preferably, the simulation evaluation unit includes:

[0051] Digital twin units, constructing a digital twin model of the packaging process;

[0052] The parameter injection unit injects the parameters of the candidate strategy into the digital twin model for virtual execution;

[0053] The performance indicator monitoring unit monitors key performance indicators during the virtual execution process, including packaging efficiency, defect rate, and resource consumption.

[0054] The scoring unit calculates a comprehensive score for each candidate strategy based on key performance indicators.

[0055] Preferably, the graph construction unit includes:

[0056] Node Relationship Calculator: Calculates node association strength based on temporal proximity and spatial similarity between visual sequence segments;

[0057] A dynamic threshold adjuster that automatically adjusts the node connection threshold based on the complexity of the visual sequence data;

[0058] The graph structure optimizer uses a spectral clustering algorithm to perform community detection and redundant edge pruning on the initial graph structure, generating an optimized visual sequence graph structure.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] The multi-source information acquisition module breaks through the limitations of traditional single data acquisition, simultaneously acquiring stereo vision sequence data, dynamic weight sampling data, and 3D point cloud measurement data. These three types of data reflect the characteristics of the packaging box from different dimensions. Stereo vision sequence data captures the appearance and dynamic changes, dynamic weight sampling data reflects the internal filling situation, and 3D point cloud measurement data provides accurate spatial structure parameters. The three complement each other, providing comprehensive and rich raw data support for subsequent verification work and avoiding verification deviations caused by incomplete data.

[0061] The multimodal feature learning module employs a multi-scale spatiotemporal feature learning method to process stereo vision sequence data. Compared to traditional single-scale feature extraction methods, it can simultaneously capture the detailed features and spatiotemporal variation patterns of the packaging box's appearance, such as morphological changes during the packaging process and subtle offsets in label position. The generated visual spatiotemporal feature tensor more accurately represents the visual attributes of the packaging box. The dynamic weight analysis module introduces frequency domain feature decomposition technology to process dynamic weight sampling data, transforming weight fluctuations that are difficult to detect in the time domain into frequency domain feature maps. This effectively identifies weight anomalies caused by internal product shaking, uneven packaging materials, etc., overcoming the shortcomings of static weight measurement being easily interfered with and enhancing the utilization value of weight data.

[0062] The graph structure fusion module innovatively constructs a heterogeneous information graph from visual spatiotemporal feature tensors and weight frequency domain feature maps. Leveraging the powerful node feature propagation capabilities of graph neural networks, it achieves deep fusion of different types of features. This fusion method is not a simple data splicing, but rather mines the intrinsic correlations between features. For example, it correlates the appearance deformation of a certain area of ​​a packaging box with the corresponding weight change. The generated fusion verification map features combine the advantages of both visual and weight dimensions, making the feature information more complete and correlated, providing high-quality feature input for subsequent anomaly identification.

[0063] The anomaly pattern recognition module employs a pre-trained variational autoencoder model for anomaly pattern reconstruction. This model possesses powerful nonlinear feature processing and pattern recognition capabilities, enabling it to accurately capture anomalous information from the fused verification map features. The generated anomaly pattern confidence distribution clearly reflects the anomalous correlation between 3D point cloud measurement data and visual spatiotemporal features. Compared to traditional anomaly recognition algorithms, it can effectively identify highly concealed and nonlinear anomaly patterns, reducing missed and false detections and improving the accuracy of anomaly recognition.

[0064] The intelligent decision-making module performs multi-criteria decision processing based on the confidence distribution of anomaly patterns. It can not only determine whether a package is qualified, but also accurately pinpoint the packaging stage and defect classification where the anomaly occurred. For example, it can distinguish between inadequate sealing during the sealing stage, incorrect or missing products during the filling stage, or cosmetic damage during transportation. This refined decision-making provides clear guidance for subsequent processing, eliminating the need for additional manpower, significantly shortening the problem-solving cycle, and improving the overall efficiency of the verification process.

[0065] The adaptive optimization module enables the system to operate in a closed loop. Based on the verification decision results, it generates adaptive adjustment strategies for packaging parameters and transmits these adjustments to the inbound execution system in real time. This mechanism promptly feeds back any issues discovered during verification to the front-end packaging stage. By correcting packaging parameters, it prevents the recurrence of similar anomalies, achieving collaborative optimization between verification and packaging. This drives continuous improvement throughout the entire production and warehousing process, enhancing the stability and adaptability of the enterprise's supply chain. Attached Figure Description

[0066] Figure 1 This is a schematic diagram illustrating the working principle of the multi-source information verification system for product packaging boxes before warehousing, as described in this invention.

[0067] Figure 2 A flowchart illustrating the operation of the multimodal feature learning module;

[0068] Figure 3 A flowchart illustrating the operation of the dynamic weight analysis module;

[0069] Figure 4 This is a confidence distribution diagram of anomaly patterns based on variational autoencoders;

[0070] Figure 5 This is a distribution diagram of rule triggering and defect stages in an intelligent decision-making system. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] Please see Figure 1This invention provides a multi-source information verification system for product packaging boxes before warehousing. The system includes: synchronously collecting stereoscopic visual sequence data, dynamic weight sampling data, and 3D point cloud measurement data of the target product packaging box to form an original multi-source information data stream. A multimodal feature learning module processes the stereoscopic visual sequence data and extracts a visual spatiotemporal feature tensor that can characterize the appearance, shape, and motion state of the packaging box through multi-scale spatiotemporal feature learning. At the same time, a dynamic weight analysis module performs frequency domain feature decomposition on the dynamic weight sampling data, transforming the weight fluctuation information in the time domain into energy distribution features in the frequency domain, generating a weight frequency domain feature map. A graph structure fusion module uses the above-mentioned visual spatiotemporal feature tensor and weight frequency domain feature map as heterogeneous nodes to construct a heterogeneous information graph, and uses a graph neural network to perform feature propagation and information aggregation between nodes on this graph, thereby generating a fusion verification graph feature that deeply integrates visual and weight information. The anomaly pattern recognition module invokes a pre-trained variational autoencoder model. This model reconstructs the features of the input fusion verification map by learning the distribution of normal packaging box data. It quantifies the degree of anomaly by calculating the reconstruction error, ultimately outputting an anomaly pattern confidence distribution reflecting the anomaly correlation between 3D point cloud measurement data and visual spatiotemporal features. Based on this confidence distribution and combined with preset multi-criteria decision rules, the intelligent decision module comprehensively analyzes the 3D point cloud measurement data and stereo vision sequence data, generating verification decision results that can locate the packaging stage where anomalies occur and classify defects. The adaptive optimization module dynamically generates packaging parameter adjustment strategies based on the verification decision results and transmits these strategies to the data entry execution system, triggering corresponding parameter correction operations, thus forming a closed-loop control from detection to feedback optimization.

[0073] Example 1: See Figure 2In specific implementation, the processing of stereo vision sequence data by the multimodal feature learning module begins with the operation of the sequence segmentation unit. The sequence segmentation unit receives the stereo vision sequence data stream from the multi-source information acquisition module and performs temporal window partitioning. This partitioning can employ a fixed-length window, such as each window covering a 0.5-second video frame sequence, or an adaptive window size adjustment strategy. The window length is dynamically calculated based on the average amplitude of scene changes in the sequence, for example, by triggering the window boundary by calculating the moving average of the differences between consecutive frames. This divides the continuous stereo vision sequence data into multiple temporally overlapping or consecutive visual sequence segments. Each visual sequence segment contains a series of stereo image pairs arranged in temporal order for subsequent fine-grained feature extraction. In some embodiments, the temporal window partitioning can also consider the speed of the packaging box moving on the conveyor belt, synchronously adjusting the window interval via encoder signals to ensure that each segment covers approximately the same physical displacement. The sequence segmentation unit outputs a set of temporally indexed visual sequence segments, providing structured input for spatiotemporal feature learning.

[0074] The spatiotemporal coding unit then processes each visual sequence segment. It invokes a pre-trained 3D convolutional neural network model, typically pre-trained on large-scale video action recognition datasets. This model's structure includes multiple 3D convolutional layers, 3D pooling layers, and activation function layers. The 3D convolutional kernels slide simultaneously across the spatial and temporal dimensions of the visual sequence segment, extracting patterns of pixel values ​​changing over time within local regions, such as movement of a packaging box edge, surface texture fluctuations, or label jitter. For each input visual sequence segment, the spatiotemporal coding unit normalizes it to a fixed size before feeding it into the 3D convolutional neural network. The network's forward propagation outputs a high-dimensional feature map, which is compressed through a global average pooling layer or a fully connected layer to generate a fixed-length local spatiotemporal feature vector. This local spatiotemporal feature vector captures joint information about motion appearance within the segment. It can be understood that the parameters of the 3D convolutional neural network model are frozen during the training phase; only inference computation is performed at this stage to ensure consistent processing speed. The spatiotemporal coding unit generates a corresponding local spatiotemporal feature vector for each visual sequence segment, and all these vectors constitute a preliminary feature set.

[0075] The graph construction unit then organizes the visual sequence segments into a graph structure. Each visual sequence segment is defined as a graph node, and the feature attributes of each node are the local spatiotemporal feature vectors generated by the spatiotemporal coding unit. The graph construction unit comprises three sub-components: a node relationship calculator, a dynamic threshold adjuster, and a graph structure optimizer. The node relationship calculator is responsible for calculating the association strength between any two nodes. The association strength is calculated based on two criteria: temporal proximity is measured by comparing the timestamp intervals of two visual sequence segments; the shorter the interval, the higher the temporal proximity score. Spatial similarity is evaluated by calculating the cosine similarity or negative Euclidean distance between the local spatiotemporal feature vectors corresponding to the two nodes; the higher the similarity value, the higher the spatial similarity score. The node relationship calculator weights and fuses the temporal proximity score and the spatial similarity score to obtain the final node association strength value. A dynamic threshold adjuster automatically adjusts a connection threshold based on the overall complexity of the current batch of stereo vision sequence data. The complexity metric can be the average variance of all local spatiotemporal feature vectors or the entropy of the feature distribution. When the complexity is high, the dynamic threshold adjuster appropriately lowers the connection threshold to allow more connections and prevent the graph structure from becoming too sparse. When the complexity is low, the threshold is increased to reduce redundant edges. The graph structure optimizer receives preliminary node and association strength information. It applies a spectral clustering algorithm to group nodes, identifies potential community structures, and then prunes and removes edges connecting different communities with association strength lower than the average strength within each community. The graph structure optimizer outputs an optimized visual sequence graph structure where the connections between nodes are more semantically relevant. Optionally, the graph building unit can also introduce node degree normalization to balance the influence of nodes.

[0076] The Graph Attention Unit (GNU) operates on an optimized visual sequence graph structure. It implements a Graph Attention Network (GNN) mechanism, which computes attention weights between each node and its neighbors. These weights are calculated using a learnable attention function that takes the current node's features and those of its neighbors as input and outputs a scalar weight representing the contribution of neighboring nodes to the current node's feature update. The GNN performs multiple rounds of message passing. In each round, each node aggregates the weighted feature information of all its neighbors and updates its own feature representation using a non-linear transformation function. After several iterations, each node's features are fused with the contextual information of its multi-hop neighbors, generating a set of enhanced node features. Essentially, the use of a GNN allows the model to focus on the neighborhood information most relevant to the current node. For example, in packaging box detection, it might focus more on segments of the same box surface area. The enhanced node feature set contains rich local and global spatiotemporal context. In some embodiments, the GNN can also integrate a multi-head attention mechanism, computing multiple independent attention operations in parallel and concatenating their results to capture different types of dependencies.

[0077] The global pooling unit finally aggregates the enhanced node feature set. It performs global graph pooling operations, commonly including global average pooling or global max pooling. Global average pooling averages the feature vectors of all nodes in the graph along each dimension, outputting a global feature vector. Global max pooling takes the maximum value in each feature dimension, outputting another global feature vector. In some implementations, the global pooling unit can combine the outputs of both pooling strategies to simultaneously retain average statistical information and salient feature responses. The resulting visual spatiotemporal feature tensor is a fixed-dimensional numerical vector that summarizes the overall spatiotemporal characteristics of the entire stereo vision sequence data. Optionally, a graph readout function can be added before the global pooling unit, such as using virtual nodes or attention pooling to weight and summarize node features. The visual spatiotemporal feature tensor, as the output of the multimodal feature learning module, is passed to the graph structure fusion module for further processing.

[0078] Example 2: See Figure 3In practical implementation, the dynamic weight analysis module begins processing the dynamic weight sampling data from the multi-source information acquisition module. The frequency domain transformation unit first performs Fast Fourier Transform (FFT) processing on the dynamic weight sampling data. This dynamic weight sampling data is a time-varying mass signal, typically obtained by continuous sampling by a high-precision dynamic scale as the packaging box passes through the weighing area. The frequency domain transformation unit applies the FFT algorithm to this discrete-time signal, converting it from the time domain to the frequency domain, generating a complex-form weight frequency domain signal. This signal contains amplitude and phase spectrum information, with the amplitude spectrum representing the intensity distribution of different frequency components. It can be understood that FFT processing can reveal periodic or quasi-periodic fluctuation patterns in the weight signal that are not obvious in the time domain. These patterns may be related to the swaying of items inside the packaging box, the vibration of packaging materials, or the natural frequency of the mechanical conveyor belt. The weight frequency domain signal output by the frequency domain transformation unit provides the basis for subsequent frequency band analysis.

[0079] The frequency band decomposition unit then processes the weight frequency domain signal. It divides the entire frequency range into multiple preset frequency bands. These band divisions can be based on logarithmic, linear, or prior knowledge specific to the application scenario, such as ultra-low frequency, low frequency, mid frequency, and high frequency bands. The unit segments the weight frequency domain signal according to these preset bands, resulting in a set of sub-band signals. Each sub-band signal contains the components of the original signal in the corresponding frequency range. Subsequently, the unit performs energy calculations on each sub-band signal. Energy calculations are typically performed by summing or integrating the squares of the amplitudes at all frequency points within the band, generating a scalar value representing the energy of that band. The energy values ​​of all bands are arranged sequentially to form a frequency band energy distribution vector. This vector summarizes the energy distribution characteristics of the weight dynamic sampling data in different frequency ranges in a compact numerical form. In some embodiments, the frequency band division can be adaptively adjusted according to the typical specifications of the packaging box and the characteristics of the contents. For example, heavy-duty boxes may focus more on low-frequency energy, while fragile packaging may focus more on mid-to-high frequency energy components.

[0080] The feature encoding unit then performs dimensionality reduction on the frequency band energy distribution vector. The feature encoding unit invokes a pre-trained autoencoder model, which typically consists of an encoder network and a decoder network. The encoder network, composed of multiple fully connected layers and nonlinear activation functions, is responsible for mapping the high-dimensional input vector to a low-dimensional latent space representation. The autoencoder model has been unsupervised pre-trained on a large amount of normal packaging box weight data, aiming to learn how to efficiently reconstruct the input data. Therefore, the encoder part learns to extract the most representative features from the input data. The feature encoding unit inputs the frequency band energy distribution vector into the encoder part of this pre-trained autoencoder model. The forward propagation of the encoder network outputs a compressed frequency domain feature vector with significantly reduced dimensionality. The compressed frequency domain feature vector retains the main patterns and information of the original frequency band energy distribution while eliminating redundancy and noise. Optionally, the autoencoder model can be a variational autoencoder or a denoising autoencoder, etc., to enhance the robustness of the features. The compressed frequency domain feature vector output by the feature encoding unit is ready for generating a visualization map.

[0081] The map generation unit is responsible for creating a weight frequency domain feature map. It maps compressed frequency domain feature vectors to a two-dimensional feature space. This mapping can be achieved through linear projection methods such as principal component analysis, or through nonlinear dimensionality reduction techniques such as t-distribution random neighborhood embedding. The purpose of the mapping is to visualize the high-dimensional features on a two-dimensional plane, facilitating the observation of the relative positional relationships between different samples. After mapping, the map generation unit generates a continuous map on the two-dimensional plane using interpolation algorithms. Common interpolation algorithms include bilinear interpolation, bicubic interpolation, or radial basis function-based interpolation. These algorithms calculate and fill the values ​​at other positions in the grid based on the position and value of discrete feature points in the two-dimensional space, ultimately generating a smooth, image-like weight frequency domain feature map. The weight frequency domain feature map can be viewed as a feature image, where color or grayscale depth represents the intensity or magnitude of the feature. In some embodiments, the map generation unit can also normalize the generated map to ensure that the values ​​are within the range of 0 to 1, facilitating subsequent processing.

[0082] In the graph structure fusion module, the graph initialization unit begins constructing a heterogeneous information graph. This unit receives a visual spatiotemporal feature tensor from the multimodal feature learning module and a weight frequency domain feature map from the dynamic weight analysis module. The graph initialization unit defines each feature dimension of the visual spatiotemporal feature tensor as a type of graph node, called a visual node. Each visual node represents a specific dimension or direction in the visual feature space. Simultaneously, the graph initialization unit defines each feature point on the weight frequency domain feature map as another type of graph node, called a weight node. Each weight node corresponds to a feature value at a specific location on the map. In this way, the graph initialization unit creates an initial heterogeneous information graph containing two types of heterogeneous nodes. The node set of the initial heterogeneous information graph is the union of the visual node set and the weight node set. It can be understood that the graph initialization process unifies data from different modalities and structures under a graph representation learning framework, laying the foundation for cross-modal information interaction.

[0083] The edge construction unit is responsible for establishing connections between nodes in the initial heterogeneous information graph. It calculates the feature similarity between each pair of visual and weight nodes. Since the original features of visual and weight nodes may fall within different numerical ranges and distributions, the edge construction unit may first standardize the node features. Then, it uses similarity metrics such as cosine similarity or Gaussian kernel-based similarity to calculate the association strength of cross-modal node pairs. After calculation, the edge construction unit sets a similarity threshold. This threshold can be a predefined fixed value or dynamically determined based on the similarity distribution of all potential connection pairs in the current graph, such as the median or a certain quantile of similarity values. The edge construction unit only retains connections between visual and weight node pairs whose similarity exceeds the set threshold, establishing undirected edges between these pairs to generate a cross-modal heterogeneous graph edge set. This edge set defines the paths that allow information flow within the heterogeneous information graph. Optionally, in addition to establishing heterogeneous edges, the edge construction unit can also establish connections within nodes of the same type based on feature similarity, forming homogeneous edges to promote feature smoothing within modalities.

[0084] The message passing unit performs feature propagation processing of the graph neural network on the constructed heterogeneous information graph. The message passing unit implements a graph neural network based on a message passing mechanism, which consists of multiple stacked graph convolutional layers or graph attention layers. In each layer, each node receives messages from its directly connected neighbor nodes. These messages are typically transformed feature vectors of the neighbor nodes. Each node aggregates all received neighbor messages and then combines them with features from its previous layer to generate a new feature representation through a learnable update function. Since the graph contains two types of nodes—visual nodes and weight nodes—and heterogeneous edges connecting them, information can flow bidirectionally between the visual and weight modalities. After multiple rounds of such message passing processing, the features of each node are integrated with information from its multi-hop neighbors in the graph, particularly the context of the other modality. The node features are updated; the features of visual nodes contain clues about the weight spectrum, and the features of weight nodes incorporate the spatiotemporal context of the visual modality. It is understandable that the number of message passing rounds needs careful selection. Too few rounds may lead to insufficient information fusion, while too many rounds may result in overly smoothed node features, losing their discriminative power.

[0085] The feature readout unit finally extracts graph-level representations from the updated heterogeneous information graph. It performs global graph pooling on the updated features of all nodes in the graph. Common pooling operations include global average pooling (averaging the feature vectors of all nodes by dimension) or global max pooling (maximizing the value of each feature dimension). Alternatively, the results of both pooling strategies can be concatenated. The feature readout unit outputs the pooled result as a fixed-dimensional vector, which is the fusion verification graph feature. This fusion verification graph feature integrates the visual dynamic sequence information and weight dynamic spectrum information of the packaging box, forming a unified and information-rich representation for subsequent anomaly pattern recognition. In some embodiments, the feature readout unit can use more complex graph pooling methods, such as global attention pooling, assigning different weights to different nodes before weighted summation to emphasize the contribution of more important nodes to the final representation. Optionally, the feature readout unit can also perform a nonlinear transformation on the node features before pooling to enhance representational power.

[0086] Example 3: In specific implementation, the core of the abnormal pattern recognition module is a pre-trained variational autoencoder model. This model is trained offline using fusion verification map features from a large number of normal packaging box samples to learn the distribution patterns of normal data in the latent space. The encoder unit, as the first part of the variational autoencoder, receives the fusion verification map features generated by the graph structure fusion module. The fusion verification map features are high-dimensional numerical vectors. The encoder unit typically consists of multiple fully connected layers. The last layer of the encoder network is divided into two parallel sub-layers. One sub-layer outputs the mean vector μ, and the other sub-layer outputs the logarithm log(σ²) of the variance vector. These two vectors together define the multidimensional Gaussian distribution followed by the latent feature vector z. The mean vector μ represents the center position of the latent space, while the logarithm log(σ²) is used for numerical stability. The variance σ² can be obtained through exponential operations. In this way, the encoder unit maps the input fusion verification map features to a set of distribution parameters in the latent space, completing the probabilistic mapping from the high-dimensional observation space to the low-dimensional continuous latent space. In some embodiments, the encoder unit may introduce a batch normalization layer or a dropout layer to improve the stability and generalization ability of training. The output of the encoder unit is the latent feature distribution parameters, namely the logarithm of the mean vector μ and the variance vector log(σ²).

[0087] The sampling unit then randomly samples from the distribution defined by the encoder unit to obtain a specific latent feature vector. The sampling unit uses a reparameterization technique to achieve this process. The reparameterization technique first samples a random noise vector from a standard Gaussian distribution N(0,I). Then, the latent eigenvectors are calculated through a deterministic transformation. , where ⊙ denotes element-wise multiplication. This method introduces randomness into the sampling process, but it separates this randomness from the model parameters during gradient calculation, thus supporting end-to-end training of the model via backpropagation. During the inference phase, the sampling unit performs the same operation, sampling one element from N(0,I). The sample, combined with μ and σ calculated from the current input, generates a specific latent feature vector z. The latent feature vector z is a point in the latent space, a compressed, probabilistic representation of the features of the original fusion verification map. It can be understood that the reparameterization technique is key to the effective operation of the variational autoencoder, enabling the model to optimize the loss function, which includes random sampling steps, during training using stochastic gradient descent.

[0088] The decoder unit is responsible for reconstructing the original input from the sampled latent feature vector. The decoder unit is the second part of the variational autoencoder, and its structure is typically symmetrical to the encoder unit, consisting of multiple fully connected layers and stacked nonlinear activation functions. The decoder unit takes the latent feature vector z output by the sampling unit as input, and through forward propagation of the network layers, gradually maps the low-dimensional latent representation back to the high-dimensional space where the original fusion verification map features reside, outputting a vector with the exact same dimension as the original fusion verification map features; this is called the reconstructed verification feature. Ideally, for a normal sample from the training distribution, the decoder unit can reconstruct the original features well from its latent encoding z, and the reconstructed verification feature will be very close to the original fusion verification map features. However, for an abnormal sample deviating from the normal distribution, because its latent encoding z may fall in a region with low training data density, the decoder unit's reconstruction ability will decrease, resulting in a significant difference between the reconstructed verification feature and the original features. In some embodiments, the first layer of the decoder unit may use different activation functions to adapt to the feature distribution, and the output of the decoder unit, the reconstructed verification feature, will be used for subsequent anomaly scoring.

[0089] The anomaly scoring unit ultimately calculates the confidence distribution of anomaly patterns. Based on the concept of reconstruction error, which can be influenced by the dimensions and absolute value of the feature itself, the anomaly scoring unit further processes the reconstruction error to obtain a standardized and interpretable anomaly confidence score. One approach is to calculate the reconstruction probability, i.e., to evaluate the log-likelihood of the original feature x given its latent code z. Assuming p(x|z) follows a Gaussian distribution, the log-likelihood is proportional to the negative mean squared error. Another, more direct approach is to map the reconstruction error to a confidence score between 0 and 1 using the following formula:

[0090]

[0091] in: This indicates the confidence level of the abnormal pattern for the current sample (the lower the value, the more abnormal it is). It is a scaling factor used to adjust the steepness of the confidence curve. It is the first The absolute reconstruction error of each feature dimension It is the total dimension of the feature vector. It is the original feature vector L2 norm, It is a very small constant to prevent division by zero errors.

[0092] In some embodiments, the anomaly pattern confidence distribution can be a multi-dimensional vector, where each element corresponds to a potential anomaly pattern, achieved by clustering or partitioning the latent space. Ultimately, the anomaly pattern confidence distribution is output to the intelligent decision-making module as the core input to the decision-making process. The unsupervised nature of the entire anomaly pattern recognition module enables it to be deployed and operated without relying on a large number of labeled anomaly samples, adapting to the real-world challenges of diverse anomaly types and scarce labeled data in industrial production.

[0093] See Figure 4 This figure demonstrates the application effect of variational autoencoders in packaging box anomaly detection. The graph shows the distribution of anomaly pattern confidence scores for all samples, distinguishing normal and anomaly samples using data points of different colors and shapes. The horizontal axis represents the sample index number, and the vertical axis represents the anomaly pattern confidence score calculated by the system, ranging from 0 to 1. Higher scores indicate a higher probability of the sample being normal. The red dashed line in the graph sets the threshold boundary for anomaly judgment; samples exceeding this boundary are identified as potential anomalies. The color intensity of each data point reflects the magnitude of the reconstruction error, a crucial evaluation metric for variational autoencoders, representing the degree of difference between the original and reconstructed features. Normal samples are typically concentrated in high-confidence regions with smaller reconstruction errors, while anomaly samples are distributed in low-confidence regions with larger reconstruction errors. This visualization helps to intuitively understand the model's ability to identify different types of packaging box defects.

[0094] Example 4: In specific implementation, the intelligent decision-making module transforms the confidence distribution of abnormal patterns into specific verification decision results through a structured multi-criteria decision-making process. The rule base construction unit is the foundation of the intelligent decision-making module. The rule base construction unit extracts decision rules from historical verification data. The historical verification data includes the fusion verification map features of a large number of packaging boxes in the past, the confidence distribution of abnormal patterns, and the verification result labels finally confirmed by humans or other authoritative methods. The data mining unit performs association rule mining processing on the historical verification data. For example, "when the value of the third dimension of the visual spatiotemporal feature tensor is greater than the threshold T1 and the low-frequency energy of the weight frequency domain feature spectrum is lower than the threshold T2, there is a high probability of corresponding to the packaging defect." The data mining unit outputs a series of such association rules, including rule antecedents and rule consequents, as well as the confidence and support indicators of the rules. The rule optimization unit then filters these initially mined association rules. It uses a genetic algorithm to perform multi-objective optimization on the association rules, encoding each rule as a chromosome. Optimization objectives include rule confidence, support, rule length, and adaptability to the latest data. Through selection, crossover, and mutation, the rule population iteratively evolves, ultimately selecting a Pareto-optimal rule set that is non-inferior across multiple objectives. A Pareto-optimal rule set means that no further optimization of a particular objective can be achieved without compromising other objectives. Finally, the knowledge representation unit converts the Pareto-optimal rule set into an executable decision rule base. Each rule is converted into a production rule form of "IF <condition> THEN <decision conclusion> WITH <weight>". The decision rule base is stored in a database table or XML file, facilitating rapid matching and retrieval by the rule triggering unit of the intelligent decision module. It can be understood that the construction of the decision rule base is an offline, periodic process that can be continuously updated and optimized as historical data accumulates.

[0095] The feature mapping unit is responsible for adapting the anomaly pattern confidence distribution output by the anomaly pattern recognition module to an input format acceptable to the decision rule base. The anomaly pattern confidence distribution may be a multi-dimensional vector or a probability distribution function. The feature mapping unit performs necessary preprocessing, including normalization to scale the values ​​to the [0,1] interval, feature selection to extract key dimensions, or feature transformations such as principal component analysis to reduce dimensionality and align with the feature space in the rule base conditions, generating decision input features. The decision input features are structured feature vectors, each dimension of which corresponds to the feature items of interest in the rule conditions of the decision rule base. The rule triggering unit matches the decision input features with applicable rules in the decision rule base. The rule triggering unit performs matching calculations between the decision input features and the antecedent conditions of each rule in the rule base. Matching calculations typically involve comparing whether the feature values ​​fall within the threshold range specified by the rule conditions, or calculating the similarity between the feature vector and the rule condition vector. When the matching degree exceeds a preset threshold, the rule is triggered. The rule triggering unit collects all triggered rules to form the set of active rules for the current decision cycle. Optionally, the rule triggering process can employ efficient pattern matching algorithms such as the RETE algorithm to handle situations with a large number of rules.

[0096] The weight calculation unit calculates a dynamic weight for each rule in the activation rule set. The dynamic weight calculation is based on multiple factors, including the rule's static confidence level, the frequency of recent triggering of the rule, and the degree of matching between the rule's consequent conclusion and the current decision input features. The weight calculation unit assigns an appropriate weight value to each decision criterion, which corresponds to the decision direction suggested by the rule's consequent, such as "determines that the contents are missing and suggests checking the filling stage" or "determines that the appearance is damaged and suggests checking the sealing stage." Subsequently, the weight calculation unit performs weighted decision processing in conjunction with the anomaly pattern confidence distribution. This weighted decision processing weights and fuses the consequent conclusions of each triggered rule according to its dynamic weight. Simultaneously, the anomaly pattern confidence distribution itself, as a global anomaly probability evidence, also participates in the final decision. For example, when the overall value of the anomaly pattern confidence distribution is extremely low, even if some rules suggest approval, the final decision may still tend to be negative. The result generation unit synthesizes the weighted calculation results and outputs the final verification decision result. The verification decision result not only includes a binary judgment of whether or not the item passes the warehousing process, but more importantly, it includes location information for the anomaly generation and packaging stage, as well as classification information for the defect type. See Table 1 for a simplified snippet of the decision rule base.

[0097] Table 1: Decision Rule Base Table

[0098] Rule ID Rule conditions (antecedents) Decision conclusion (subsequent event) Static confidence R001 Visual feature dimension 1 > 0.7 AND weight energy low frequency band < 0.3 Defect type: Inadequate sealing; Recommended inspection stage: Box sealing. 0.89 R002 Anomaly confidence score < 0.4 AND point cloud volume difference > 5% Defect type: Box deformation; Recommended inspection stage: Handling 0.92 R003 Visual feature dimension 5 is within the range of [0.2, 0.5]. Defect type: Labeling anomaly; Recommended inspection stage: Labeling 0.76

[0099] In some embodiments, the result generation unit can also output a measure of decision uncertainty, such as the entropy value based on the distribution of activation rule weights or the confidence level based on the voting mechanism, providing richer decision information to the downstream system. Optionally, the intelligent decision module can integrate a feedback learning mechanism, using the actual execution effect of the decision as a feedback signal to fine-tune rule weights or trigger incremental updates to the rule base. The verification decision results generated by the intelligent decision module are finally fed into the adaptive optimization module to drive the closed-loop optimization of the packaging parameters.

[0100] See Figure 5 This chart illustrates the decision-making process and rule triggering of the intelligent decision-making module in a multi-source information verification system. The bar chart shows the triggering frequency of different decision rules during the detection process, reflecting the activity level of each rule in practical application. Each bar corresponds to a specific decision rule, and the height of the bar indicates the number of times that rule is triggered throughout the detection process. Different colored bars help distinguish different rule categories, and the numbers above the bars indicate the specific number of triggers. This visualization helps analysts understand which rules play a key role in anomaly detection. The chart also includes statistical information on the decision results, showing the main defect types identified by the system and their corresponding production stages. This information is of significant guiding importance for optimizing packaging processes and improving production techniques. By analyzing the triggering patterns of the rules, decision thresholds can be further adjusted to improve the accuracy and efficiency of anomaly detection.

[0101] Example 5: In specific implementation, the adaptive optimization module initiates the parameter optimization process based on the verification decision results generated by the intelligent decision module. The strategy search unit performs Monte Carlo tree search processing in the predefined strategy space based on the verification decision results. For example, the verification decision results may indicate that "insufficient pressure during the sealing stage leads to poor sealing of the box, and the defect is classified as Class A." The strategy search unit transforms this semantic information into the initial state of the search. The strategy space consists of a series of adjustable packaging parameters and their value ranges. For example, the adjustable range of sealing pressure is [20, 50] kN, the adjustable range of top cover heat sealing temperature is [150, 200] degrees Celsius, and the adjustable range of conveyor belt speed is [0.5, 2.0] m / s. The strategy search unit constructs a search tree. The root node of the tree represents the current parameter setting state, and the child nodes represent the new states that can be reached by changing a certain parameter value. The Monte Carlo tree search processing includes four iterative steps: selection, expansion, simulation, and backtracking. In the selection phase, the strategy search unit... Starting from the root node, the upper bound confidence interval algorithm is used to recursively select child nodes until a node that is not fully expanded is reached. In the expansion phase, the policy search unit randomly selects an action from the unexplored actions, such as increasing the box-sealing pressure from the current 30 kN to 35 kN, thus adding a new child node to the search tree. In the simulation phase, the policy search unit starts from this new node and simulates a complete packing process using a randomized strategy or a simplified evaluation strategy until a simulation termination state is reached, and a simulation reward score is obtained. In the backtracking phase, the policy search unit backtracks the simulation results along the search path from the new node back to the root node, updating the visit count and cumulative reward value of all nodes on the path. Through multiple iterations of the above loop, the policy search unit gradually explores more promising regions in the policy space, ultimately generating a candidate policy set containing multiple potential parameter adjustment schemes. Each candidate policy in the candidate policy set corresponds to a specific set of parameter assignments. It can be understood that the advantage of Monte Carlo Tree Search is its ability to balance the exploration of unknown regions with the utilization of currently known excellent strategies.

[0102] The simulation evaluation unit performs packaging process simulation for each candidate strategy in the candidate strategy set. This unit includes a digital twin unit, which constructs a digital twin model of the packaging process. This model is a virtual mapping of the actual packaging production line, simulating the entire dynamic process of packaging boxes from empty box loading, item filling, box forming, sealing, to warehousing preparation through a physics engine and mathematical model. The digital twin model includes mechanical kinematics, thermodynamic transfer, material mechanical properties, and control system logic, enabling it to respond to input parameter settings in a near-real-world manner. The parameter injection unit injects specific parameter values ​​of the candidate strategies into the digital twin model, such as the sealing pressure, heat sealing temperature, and conveyor belt speed set in the candidate strategies. The values ​​are configured on the corresponding actuators of the virtual packaging line. During the virtual model's operation, the indicator monitoring unit monitors and records a series of predefined key performance indicators in real time. These key performance indicators typically include packaging efficiency, product defect rate, and resource consumption. After the simulation cycle ends, the indicator monitoring unit summarizes these indicator values. The scoring unit calculates a comprehensive score for each candidate strategy based on the monitored key performance indicators. The comprehensive scoring function can be a weighted sum of these indicators, for example: Comprehensive score = w1 * Packaging efficiency - w2 * Defect rate - w3 * Resource consumption, where the weights w1, w2, and w3 are pre-set according to the importance of the production target. The scoring unit calculates a quantitative comprehensive score for each candidate strategy for subsequent comparison and selection.

[0103] The strategy selection unit selects the optimal candidate strategy from the candidate strategy set based on the comprehensive score calculated by the simulation evaluation unit. The strategy selection unit compares the comprehensive scores of all candidate strategies and selects the strategy with the highest score as the packaging parameter adaptive adjustment strategy. In some embodiments, if the comprehensive scores of multiple candidate strategies are very close, the strategy selection unit may introduce a second criterion for decision-making, such as selecting a strategy with a smaller parameter adjustment range to maintain production stability, or selecting a strategy with lower resource consumption to optimize costs. The finally selected packaging parameter adaptive adjustment strategy is a clear set of instructions that specifies the target set values ​​of the parameters of each actuator. The real-time deployment unit deploys the adaptive adjustment strategy to the inbound execution system. The real-time deployment unit converts the parameter setting instructions contained in the strategy into communication protocol messages that the inbound execution system can recognize, and sends them to the lower-level programmable logic controller or distributed control system through the industrial network to initiate the parameter correction operation. After receiving the new parameters, the inbound execution system will adjust the corresponding actuators, such as adjusting the pressure valve of the pneumatic component, setting the temperature controller of the heat sealer, and changing the frequency of the frequency converter to adjust the speed of the conveyor motor, so that the packaging process runs according to the optimized parameters. Optionally, the real-time deployment unit may include a security verification step to ensure that new parameters are within the physical security limits of the device, avoiding equipment damage or production interruption due to over-adjustment.

[0104] In some embodiments, the operation of the adaptive optimization module may not be a one-time event, but rather form a closed loop. The strategy search unit can restart the Monte Carlo tree search process based on the results of a new round of verification decisions and production data after parameter adjustment, thereby achieving continuous parameter optimization. The digital twin unit can also continuously update and calibrate its own model using new data collected from the actual production line to improve the accuracy of simulation evaluation. The digital twin model of the simulation evaluation unit can be constructed to be more refined, for example, by introducing finite element analysis to simulate the stress distribution of the box under specific pressure, or by computational fluid dynamics to simulate the flow of hot air and heat conduction during the heat sealing process, thereby more accurately predicting the impact of parameter changes on packaging quality. The index monitoring unit can monitor a wider range of key performance indicators, such as the fatigue life of packaging materials and the wear prediction of key equipment components. The comprehensive scoring function of the scoring unit can also be designed to be more complex, incorporating more optimization objectives and constraints. Optionally, the real-time deployment unit can achieve incremental deployment, that is, instead of immediately adjusting the parameters to the target value, it gradually approaches the target value in several small steps to observe the system response and ensure a smooth transition. Through this simulation and search-based iterative optimization mechanism, the adaptive optimization module enables the packaging system to adaptively respond to various changes, such as fluctuations in raw material characteristics, equipment degradation, or changes in product specifications, ultimately achieving continuous improvement in packaging quality and efficiency.

[0105] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-source information verification system for product packaging boxes before warehousing, characterized in that, The system comprises: A multi-source information acquisition module for acquiring a multi-source information data stream of a target product packaging box, the multi-source information data stream comprising stereovision sequence data, weight dynamic sampling data and three-dimensional point cloud measurement data of the packaging box; A multi-modal feature learning module for performing multi-scale spatiotemporal feature learning processing on the stereovision sequence data to generate a visual spatiotemporal feature tensor; A dynamic weight analysis module for performing frequency domain feature decomposition processing on the weight dynamic sampling data to generate a weight frequency domain feature spectrum; A graph structure fusion module for constructing the visual spatiotemporal feature tensor and the weight frequency domain feature spectrum into a heterogeneous information graph, performing node feature propagation processing on the heterogeneous information graph through a graph neural network, and generating a fusion verification graph feature; An abnormal pattern recognition module for calling a pre-trained variational autoencoder model to perform abnormal pattern reconstruction processing on the fusion verification graph feature to generate an abnormal pattern confidence distribution, wherein the abnormal pattern confidence distribution reflects the abnormal correlation between the three-dimensional point cloud measurement data and the visual spatiotemporal feature; An intelligent decision-making module for performing multi-criteria decision processing on the three-dimensional point cloud measurement data and the stereovision sequence data based on the abnormal pattern confidence distribution to generate a verification decision result of the packaging box, the verification decision result being used for locating the packaging stage where the abnormality occurs and classifying the defects; An adaptive optimization module for generating a packaging parameter adaptive adjustment strategy according to the verification decision result and transmitting the adaptive adjustment strategy to a warehousing execution system to start a parameter correction operation.

2. The product package warehouse-in multi-source information verification system according to claim 1, wherein The multi-modal feature learning module comprises: A sequence segmentation unit for performing time window division processing on the stereovision sequence data to obtain a plurality of visual sequence segments; A spatiotemporal coding unit for calling a pre-trained 3D convolutional neural network model to perform spatiotemporal feature coding processing on each visual sequence segment to generate a local spatiotemporal feature vector; A graph construction unit for regarding each visual sequence segment as a graph node and constructing a visual sequence graph structure based on the spatiotemporal relationship between the nodes; A graph attention unit for performing node feature enhancement processing on the visual sequence graph structure through a graph attention network, calculating the attention weight of each node, and generating an enhanced node feature set; A global pooling unit for performing global graph pooling processing on the enhanced node feature set to generate the visual spatiotemporal feature tensor.

3. The system according to claim 1, wherein the product package case is a product package case for a product of a plurality of products, and the plurality of information includes information on the product. The dynamic weight analysis module comprises: A frequency domain transformation unit for performing fast Fourier transform processing on the weight dynamic sampling data to generate a weight frequency domain signal; A frequency band decomposition unit for decomposing the weight frequency domain signal into a plurality of preset frequency band sub-signals, performing energy calculation processing on each frequency band sub-signal, and generating a frequency band energy distribution vector; A feature coding unit for calling a pre-trained autoencoder model to perform dimension reduction coding processing on the frequency band energy distribution vector to generate a compressed frequency domain feature vector; A spectrum generation unit for mapping the compressed frequency domain feature vector to a two-dimensional feature space and generating the weight frequency domain feature spectrum through an interpolation algorithm.

4. The system according to claim 1, wherein the product package case is a product package case for a product of a plurality of products, and the product package case storage system is a product package case storage system for the product package case for the product. The graph structure fusion module comprises: The graph initialization unit defines each feature dimension of the visual spatio-temporal feature tensor as a graph node, defines each feature point of the weight frequency domain feature graph as another type of graph node, and constructs an initial heterogeneous information graph; The edge construction unit calculates the feature similarity between the visual nodes and the weight nodes, establishes a cross-modal edge connection based on a similarity threshold, and generates a heterogeneous graph edge set; The message passing unit updates the node features through multiple rounds of feature propagation processing on the heterogeneous graph by using a message passing mechanism of a graph neural network; The feature reading unit performs a global graph pooling operation on the updated node features, extracts a graph-level feature representation, and generates the fusion verification graph feature.

5. The product package warehouse-in multi-source information verification system of claim 1, wherein, The anomaly pattern recognition module includes: The encoder unit inputs the fusion verification graph feature into an encoder part of a variational autoencoder for latent space mapping processing to generate a latent feature distribution parameter; The sampling unit randomly samples a latent feature vector from the latent feature distribution parameter; The decoder unit reconstructs the latent feature vector through a decoder part of the variational autoencoder to generate a reconstructed verification feature; The anomaly score unit calculates a reconstruction error between the original fusion verification graph feature and the reconstructed verification feature, and generates the anomaly pattern confidence distribution based on the reconstruction error.

6. The system for checking multi-source information of product packaging cases before storage according to claim 1, wherein, The intelligent decision-making module includes: The rule base construction unit extracts multi-criteria decision rules from historical verification data to construct a decision rule base; The feature mapping unit maps the anomaly pattern confidence distribution to the input space of the decision rule base to generate decision input features; The rule triggering unit matches applicable rules in the decision rule base based on the decision input features to trigger a multi-criteria decision-making process; The weight calculation unit calculates the dynamic weight of each decision criterion and performs weighted decision-making processing in combination with the anomaly pattern confidence; The result generation unit outputs the weighted decision result as the verification decision result.

7. The product package warehouse-in prior information verification system according to claim 6, wherein The rule base construction unit includes: The data mining unit performs association rule mining processing on historical verification data to extract frequent item sets and association rules; The rule optimization unit performs multi-objective optimization processing on the association rules by using a genetic algorithm to screen a Pareto optimal rule set; The knowledge representation unit converts the Pareto optimal rule set into an executable decision rule base.

8. The system for checking multi-source information of product packaging cases before storage according to claim 1, wherein, The adaptive optimization module includes: The strategy search unit performs Monte Carlo tree search processing in a strategy space based on the verification decision result to generate a candidate strategy set; The simulation evaluation unit performs packaging process simulation processing on each candidate strategy to evaluate the strategy performance indicators; The strategy selection unit selects an optimal candidate strategy as a packaging parameter adaptive adjustment strategy according to the performance indicators; The real-time deployment unit deploys the adaptive adjustment strategy to an in-warehouse execution system.

9. The product package warehouse-in prior information verification system according to claim 8, wherein The simulation evaluation unit includes: The digital twin unit constructs a digital twin model of the packaging process; The parameter injection unit injects the parameters of the candidate strategy into the digital twin model for virtual execution; The indicator monitoring unit monitors the key performance indicators in the virtual execution process, including packaging efficiency, defect rate, and resource consumption; The scoring unit calculates the comprehensive score of each candidate strategy based on the key performance indicators.

10. The system for checking multi-source information of product packaging cases before storage according to claim 2, wherein, The graph construction unit includes: a node relationship calculator, which calculates node association strength based on time adjacency and spatial similarity between visual sequence segments; a dynamic threshold adjuster, which automatically adjusts node connection threshold according to complexity of visual sequence data; a graph structure optimizer, which performs community discovery and redundant edge pruning processing on the initial graph structure through a spectral clustering algorithm to generate an optimized visual sequence graph structure.

Images