An image recognition-based cross-border commodity compliance detection method and system

Abstract

The present application relates to the technical field of image recognition, in particular to a cross-border commodity compliance detection method and system based on image recognition, comprising the following steps: collecting a to-be-detected commodity image, and extracting a packaging label region and commodity appearance texture data in parallel; mapping to generate label semantics and visual form feature vectors; calculating feature distances using cross-attention, and constructing a graph-text semantic alignment matrix; comparing a dynamic safety threshold to determine a violation probability, and generating a final compliance detection result.In the present application, a bimodal feature mapping mechanism is constructed, commodity appearance and label text information are deeply fused, and graph-text semantic conflict features are accurately captured, effectively making up for the serious deficiency of traditional technology in feature extraction dimension, completely solving the difficult problem that disguised prohibited goods are difficult to identify in complex scenarios, and greatly improving the automation level and cargo clearance efficiency of cross-border logistics compliance detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for cross-border commodity compliance inspection based on image recognition. Background Technology

[0002] Image recognition technology involves automated processing techniques that utilize computer vision algorithms to extract features, classify, and understand specific targets. These techniques are widely applied in scenarios such as industrial inspection, security monitoring, and logistics sorting. Traditional image recognition-based methods for cross-border commodity compliance inspection involve acquiring product images using X-ray machines or industrial cameras, and then comparing the product's shape, color, or texture using a pre-set single visual template or a simple classifier to determine whether the product is prohibited or conforms to the declared category.

[0003] Traditional technologies rely solely on simple template comparisons based on single visual texture features, neglecting the deep semantic connections between the physical appearance of goods and the text on packaging labels. This results in a severe lack of feature extraction dimensions when dealing with complex prohibited goods with disguised label replacements and inconsistent image and text attributes. Consequently, these technologies cannot effectively meet the real-time dynamic and accurate screening needs of high-throughput cargo images in complex cross-border logistics scenarios, leading to a high risk of missed detections and serious customs clearance security risks in the compliance inspection of cross-border goods. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method and system for cross-border commodity compliance inspection based on image recognition.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a cross-border commodity compliance detection method based on image recognition, comprising the following steps: S1: Real-time acquisition of images of cross-border goods to be inspected through high-speed industrial cameras on the customs inspection line; use optical character recognition operators to perform text region localization and character segmentation on the images of cross-border goods to be inspected, and extract packaging label area data; use edge detection operators to perform contour fitting and texture segmentation on the images of cross-border goods to be inspected, and peel off the texture data of the product appearance. S2: Input the packaging label area data into a pre-trained semantic encoding network for semantic space alignment mapping to generate a label semantic feature vector. Use a multi-scale convolutional network to perform deep feature encoding and pyramid pooling on the product appearance texture data to construct a visual morphology feature vector. S3: Utilize the multi-head cross-attention mechanism to calculate the weighted Euclidean distance between the label semantic feature vector and the visual morphological feature vector in the feature space, quantify the logical matching degree of image and text information, and construct an image-text semantic alignment matrix in combination with preset semantic consistency constraints. S4: Input the image and text semantic alignment matrix into the preset risk assessment model for nonlinear regression analysis and feature weighting, calculate the predicted value of the violation probability, compare the predicted value of the violation probability with the dynamic security threshold based on historical customs clearance data, and generate the compliance test result of cross-border goods.

[0006] As a further aspect of the present invention, step S1 specifically comprises: S11: In response to the photoelectric sensor trigger signal on the customs inspection line, control the high-speed industrial camera to acquire high-resolution original images of the cross-border goods to be inspected in a multi-angle light source environment, and use a Gaussian filter to remove electronic noise generated during the image acquisition process to generate basic image data to be processed. S12: Input the basic image data into the text detection network based on differentiable binarization, predict the probability map and threshold map of text instances in the image, generate a binarized mask of the text region through the probability threshold fusion algorithm, and use the perspective transformation matrix to perform geometric correction and character slicing on the text region within the mask, thereby extracting the packaging label region data. S13: The multi-scale Canny edge detection operator is used to calculate the gradient magnitude and suppress non-maximum values ​​of the basic image data to identify the structural contour edges of the product appearance. The inverse mask of the text region binarization mask generated in S12 is used to filter out the texture interference of the text region. The color histogram features and gray-level co-occurrence matrix features of the remaining region are extracted to strip away the texture data of the product appearance.

[0007] As a further aspect of the present invention, step S2 specifically includes: S21: The character sequence in the packaging label area data is segmented and embedded with position encoding. It is then input into a pre-trained language model based on the Transformer architecture for bidirectional contextual semantic feature extraction. The long-distance dependency between characters is captured through the self-attention mechanism, and the output hidden state is globally averaged and pooled to generate the label semantic feature vector. S22: Input the product appearance texture data into a deep residual network for multi-level feature extraction, obtain feature maps at different resolutions, use spatial pyramid pooling layers to perform multi-scale grid partitioning and max pooling operations on the deep feature maps, and cascade and fuse the pooling results at different scales and perform dimensionality reduction processing to construct a visual morphological feature vector.

[0008] As a further aspect of the present invention, step S3 specifically comprises: S31: Map the label semantic feature vector to the query vector, and the visual morphological feature vector to the key vector and value vector. Calculate the dot product between the query vector and the key vector to scale the attention score using a multi-head attention mechanism, and generate an attention weight distribution to represent the importance of the features. S32: Based on the attention weight distribution, the corresponding dimensions of the label semantic feature vector and the visual morphological feature vector are weighted. For each semantic feature channel, the Euclidean distance between it and the corresponding visual feature channel is calculated, and the weighted sum is combined with the feature importance weight to quantify the difference between image and text features in the semantic space. S33: Combine the weighted distances calculated in S32 according to the arrangement order of the feature channels, and combine them with the preset semantic consistency constraints to normalize the distance values ​​and truncate outliers to construct a semantic alignment matrix for representing the degree of logical conflict between the text and the image.

[0009] As a further aspect of the present invention, step S4 specifically comprises: S41: Flatten the image-text semantic alignment matrix into a one-dimensional feature vector, input it into the risk assessment model consisting of multiple fully connected layers and nonlinear activation functions, perform nonlinear regression on the feature vector through the weight parameters optimized by the backpropagation algorithm, and output the predicted value of the violation probability that represents the level of violation risk of the product. S42: Obtain historical customs clearance compliance data for goods within a specified time period, calculate the statistical distribution characteristics of the risk values ​​of historical compliant goods, dynamically calculate the critical value for risk judgment based on the normal distribution assumption and confidence interval theory, and generate a dynamic safety threshold. S43: Use a comparator to compare the predicted probability of violation with the dynamic safety threshold. If the predicted value is greater than the threshold, it is judged as a high-risk violation. If the predicted value is less than or equal to the threshold, it is judged as a low-risk compliance. Based on this, a cross-border commodity compliance detection result containing risk level labels and confidence scores is generated.

[0010] As a further aspect of the present invention, the process of extracting packaging label area data specifically includes: A feature pyramid extractor containing multi-layer convolutional neural networks is constructed to extract shared feature maps of basic image data at different resolutions, and deformable convolutional layers are used to adapt to the irregular shapes of text regions. The prediction head is used to perform pixel-level classification and regression prediction on the shared feature map, generating a probability map representing the possibility of the text center region and a threshold map representing the text boundary threshold, respectively. The probability map and the threshold map are weighted and fused by the binarization formula to obtain a clear text instance binarization mask. The minimum bounding rectangle and rotation angle of the text instance are calculated based on the binarized mask. The tilted or distorted text region is corrected into a standard text image in the horizontal direction using the perspective transformation matrix. The standard text image is then segmented and serialized at the character level to obtain structured packaging label area data.

[0011] As a further aspect of the present invention, the process of constructing the visual morphological feature vector specifically includes: The product appearance texture data is input into the ResNet-50 backbone network and processed through four stages of residual block processing to extract the fourth stage feature map containing high-level semantic information and low-level detail information. The fourth-stage feature map is processed in parallel using a dilated spatial convolution pooling pyramid module. Feature sampling is performed using dilated convolution kernels with different dilation rates to capture multi-scale texture context information within different receptive fields. The output feature maps of each parallel branch are concatenated along the channel dimension, utilizing... The convolutional layer compresses the concatenated feature map to a preset target number of channels and performs global average pooling on the compressed feature map to obtain a one-dimensional deep visual feature representation, thereby constructing a visual morphological feature vector.

[0012] As a further aspect of the present invention, the calculation process of the weighted Euclidean distance specifically includes: Obtain the first weighted value after attention mechanism Each label semantic feature vector component With the Each visual morphological feature vector component And the attention importance weight corresponding to this dimension. ; The local differences of image and text features in each dimension are calculated using the weighted square difference formula, and the overall weighted Euclidean distance value is obtained by square root operation. Weighted Euclidean distance The calculation formula is: ; in, The weighted Euclidean distance between the semantic feature vector of the label and the visual morphological feature vector represents the distance between them. This represents the total dimension of the feature vector. Representing the Attention weight coefficients for each feature dimension The semantic feature vector of the representative label is in the first position. The numerical value of dimension, The visual morphological feature vector is represented in the first place. The numerical value of the dimension.

[0013] As a further aspect of the present invention, the process of generating the dynamic security threshold specifically includes: Retrieve the most recent data from the customs historical database. A historical safety sample set is constructed by predicting the probability of non-compliance for all historical goods that were deemed compliant and released within a given day. Calculate the arithmetic mean and standard deviation of all probability values ​​in the historical safety sample set, and combine them with the preset safety redundancy coefficient to determine the dynamic upper bound of risk assessment based on the statistical 3-sigma principle; Dynamic security threshold The calculation formula is: ; in, Represents a dynamic security threshold. This represents the total number of samples in the historical safe sample set. Representing the The predicted probability of non-compliance for a historical compliant product. The arithmetic mean of the predicted probability values ​​of all historical compliant products. This represents the preset safety redundancy coefficient.

[0014] A cross-border commodity compliance inspection system based on image recognition, the system being used to implement the aforementioned cross-border commodity compliance inspection method based on image recognition, the system comprising: The data acquisition and separation module is used to acquire images of cross-border goods to be inspected in real time through high-speed industrial cameras in the customs inspection line. It uses optical character recognition operators to perform text region localization and character segmentation to extract packaging label area data, and uses edge detection operators to perform contour fitting and texture segmentation to peel off the texture data of the product appearance. The dual-stream feature encoding module is used to input packaging label area data into the semantic encoding network to generate label semantic feature vectors, and input product appearance texture data into the multi-scale convolutional network to construct visual morphological feature vectors; The cross-modal alignment analysis module is used to calculate the weighted Euclidean distance in the feature space using a multi-head cross-attention mechanism and combine it with semantic consistency constraints to construct a graph-text semantic alignment matrix that quantifies the degree of graph-text logical matching. The risk assessment and decision-making module is used to input the image-text semantic alignment matrix into the risk assessment model to calculate the predicted value of the violation probability, and compare it with the dynamic security threshold generated based on historical customs clearance data to generate the compliance detection result of cross-border goods.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, a deep feature mapping mechanism based on visual and textual dual modalities is constructed. By utilizing cross-attention operation logic to deeply fuse the physical appearance of the product and the textual information of the packaging label, the semantic conflict features of the image and text are accurately captured. This enables the accurate identification of complex prohibited risks such as disguised label replacement and inconsistencies between image and text attributes. It effectively makes up for the shortcomings of traditional single-modal detection technology in terms of feature extraction dimensions, completely solves the problem of missing detection dimensions of single visual features in complex scenarios, and significantly improves the automation level, inspection accuracy and customs clearance efficiency of cross-border logistics compliance inspection. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the cross-border commodity compliance testing method of the present invention; Figure 2 This is a schematic diagram of the process for image acquisition and data stripping of the commodity to be inspected according to the present invention; Figure 3 This is a schematic diagram of the dual-stream feature encoding and vector construction process of the present invention; Figure 4 This is a schematic diagram illustrating the process of constructing the image-text semantic alignment matrix of the present invention; Figure 5 This is a schematic diagram of the process for assessing and generating violation risk results for this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the software-based technical solution is described in detail below with reference to system architecture diagrams and embodiments. It should be understood that the specific embodiments described herein are only for explaining the technical solutions of this invention and do not constitute a limitation on the scope of protection.

[0018] In the description of this invention, the system architecture relationships or data processing flows indicated by terms such as "layer," "module," "interface," "data flow," "client," and "server" are all defined based on the architecture diagram or flowchart corresponding to the embodiments. This way of describing is only used to clearly illustrate the logical relationships between the elements in the technical solution, and not to limit the physical deployment form. The term "multiple" includes two or more technical units, including but not limited to multiple data nodes, processing threads, service instances, or functional components and other scalable elements. The specific number is determined according to the actual business scenario and needs to be specifically specified.

[0019] Please see Figure 1 and Figure 2 This invention provides a technical solution: a method for compliance detection of cross-border goods based on image recognition, comprising the following steps: S1: High-speed industrial cameras on the customs inspection line capture images of cross-border goods to be inspected in real time. Optical character recognition (OCR) operators are used to perform text region localization and character segmentation on the images, and packaging label area data is extracted. Edge detection operators are used to perform contour fitting and texture segmentation on the images, stripping away the product's appearance texture data. The specific steps of S1 are as follows: S11: In response to the photoelectric sensor trigger signal on the customs inspection line, control the high-speed industrial camera to acquire high-resolution original images of the cross-border goods to be inspected in a multi-angle light source environment, and use a Gaussian filter to remove electronic noise generated during the image acquisition process to generate basic image data to be processed. S12: Input the basic image data into the text detection network based on differentiable binarization, predict the probability map and threshold map of text instances in the image, generate a binarized mask of the text region through the probability threshold fusion algorithm, and use the perspective transformation matrix to perform geometric correction and character slicing on the text region within the mask, thereby extracting the packaging label region data. S13: The multi-scale Canny edge detection operator is used to calculate gradient magnitude and suppress non-maximum values ​​in the base image data, identifying the structural contour edges of the product appearance. The inverse mask of the text region binarization mask generated in S12 is used to filter out texture interference in the text region, extracting the color histogram features and gray-level co-occurrence matrix features of the remaining region, thereby stripping away the product appearance texture data. The extraction process of packaging label area data specifically includes: A feature pyramid extractor containing multi-layer convolutional neural networks is constructed to extract shared feature maps of basic image data at different resolutions, and deformable convolutional layers are used to adapt to the irregular shapes of text regions. The prediction head is used to perform pixel-level classification and regression prediction on the shared feature map, generating a probability map representing the possibility of the text center region and a threshold map representing the text boundary threshold, respectively. The probability map and the threshold map are weighted and fused by the binarization formula to obtain a clear text instance binarization mask. The minimum bounding rectangle and rotation angle of the text instance are calculated based on the binarized mask. The tilted or distorted text region is corrected into a standard text image in the horizontal direction using the perspective transformation matrix. The standard text image is then segmented and serialized at the character level to obtain structured packaging label area data.

[0020] A photoelectric sensor monitors the movement of the conveyor belt on the customs inspection line with a response time of 10 milliseconds. When the conveyor belt speed stabilizes between 1.5 m / s and 2.0 m / s, the sensor captures a trigger signal indicating that a product has entered the detection area. This trigger signal is hardwired to the industrial camera control terminal, activating a global shutter with an exposure time set to 50 microseconds. With the assistance of a ring LED light source at a color temperature of 6500K, the industrial camera captures a freeze-frame of the high-speed moving product, obtaining a raw RGB image with a resolution of 5120×3840 pixels. The raw image is transmitted to the memory buffer of the image processing server via Gigabit Ethernet. A Gaussian smoothing filter algorithm is then used to preprocess the image. A frame of size [size missing] is constructed. A Gaussian convolution kernel was used, with a standard deviation parameter set to 1.5. This kernel was then applied to the original image matrix, and the weighted average value of the pixels within the kernel's coverage area was used to replace the value of the center pixel, in order to suppress electronic thermal noise generated during image acquisition due to photoelectric conversion.

[0021] The aforementioned Gaussian convolution kernel refers to a two-dimensional matrix generated based on the shape of a Gaussian function, with the largest value at the center and the value rapidly decreasing as it moves away from the center. It is used to perform weighted averaging of neighboring pixels in image smoothing processing.

[0022] Preprocessed base image data is input into the DBNet text detection model based on the ResNet-18 backbone network. The base image is first downsampled through five convolutional stages of the backbone network, generating multi-scale feature maps with strides of 4, 8, 16, and 32. These feature maps are then fed into a feature pyramid network, where upsampling and lateral connections fuse deep semantic information with shallow geometric details, outputting a feature map of uniform size. The prediction head outputs two single-channel images in parallel based on this feature map: a probability map representing the confidence that a pixel is a text region, and a threshold map representing the strength of the text-background boundary. During model training, the probability map generation is supervised by a binary cross-entropy loss function, and the threshold map generation is supervised by an L1 distance loss function. The AdamW optimizer is used to update the network weights with an initial learning rate of 0.001 until the loss value converges to below 0.05. During the inference phase, the probability map and threshold map are non-linearly fused using a differentiable binarization formula to generate a binarized mask. Regions with pixel values ​​greater than 0.6 are marked as text regions. Based on this mask, a connected component analysis algorithm is applied to extract the coordinates of the four vertices of the text box. A perspective transformation matrix is ​​then used to map the text region with tilt angles between -45 degrees and +45 degrees into a horizontal rectangular image, thus completing the extraction of the packaging label region data.

[0023] The aforementioned feature pyramid network refers to a deep learning model structure that combines the strong semantic information of high-level feature maps with the high-resolution information of low-level feature maps through top-down paths and lateral connections, thereby enhancing the model's ability to detect targets at multiple scales.

[0024] Simultaneously, edge detection is performed on the basic image data in parallel. The image is converted to grayscale, and the Sobel operator is applied to calculate the gradient magnitudes in the horizontal and vertical directions. A low threshold of 50 and a high threshold of 150 are set. The gradient magnitude map is subjected to double threshold filtering and non-maximum suppression to retain local maxima points in the gradient direction. Strong edge pixels are connected, and weak edge pixels are removed to generate a structural contour with a single pixel width. A reverse mask is constructed using the text region binarization mask generated by the aforementioned DBNet. This reverse mask is then bitwise ANDed with the edge contour map to completely shield the texture interference caused by the label text, retaining only the pattern and material texture of the product packaging itself. For the remaining area, the pixel distribution of the three RGB channels is statistically analyzed to construct a 256-dimensional color histogram. The gray-level co-occurrence matrices in the four directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees are calculated, and four texture descriptors—energy, contrast, entropy, and correlation—are extracted from them to complete the stripping of the product appearance texture data. Experimental data shows that when dealing with cross-border e-commerce parcels with complex backgrounds, the text localization accuracy of this combined strategy reaches 98.5%, which is 12% higher than the traditional MSER-based method.

[0025] Please see Figure 1 and Figure 3 S2: Input the packaging label area data into a pre-trained semantic encoding network for semantic space alignment mapping to generate label semantic feature vectors. Then, use a multi-scale convolutional network to perform deep feature encoding and pyramid pooling on the product appearance texture data to construct visual morphological feature vectors. The specific steps of S2 are as follows: S21: The character sequence in the packaging label area data is segmented and embedded with position encoding. It is then input into a pre-trained language model based on the Transformer architecture for bidirectional contextual semantic feature extraction. The long-distance dependency between characters is captured through the self-attention mechanism, and the output hidden state is globally averaged and pooled to generate the label semantic feature vector. S22: The product appearance texture data is input into a deep residual network for multi-level feature extraction, obtaining feature maps at different resolutions. Spatial pyramid pooling layers are used to perform multi-scale grid partitioning and max pooling operations on the deep feature maps. The pooling results at different scales are then cascaded, fused, and dimensionality-reduced to construct a visual morphology feature vector. The construction process of the visual morphology feature vector specifically includes: The product appearance texture data is input into the ResNet-50 backbone network and processed through four stages of residual block processing to extract the fourth stage feature map containing high-level semantic information and low-level detail information. The fourth-stage feature map is processed in parallel using a dilated spatial convolution pooling pyramid module. Feature sampling is performed using dilated convolution kernels with different dilation rates to capture multi-scale texture context information within different receptive fields. The output feature maps of each parallel branch are concatenated along the channel dimension, utilizing... The convolutional layer compresses the concatenated feature map to a preset target number of channels and performs global average pooling on the compressed feature map to obtain a one-dimensional deep visual feature representation, thereby constructing a visual morphological feature vector.

[0026] The extracted packaging label area data is a geometrically corrected sequence of text images. First, a CRNN engine is used to identify characters in the images, converting the image information into UTF-8 encoded text strings. The text strings are then cleaned, removing spaces and special characters, and segmented into sub-word unit sequences using a WordPiece word segmenter. These sub-word sequences are then input into a pre-trained BERT-base model. This model contains 12 Transformer encoder blocks. Each encoder block contains a multi-head self-attention mechanism layer and a feedforward neural network layer. In the input layer, each sub-word is mapped to a 768-dimensional word embedding vector, and positional encoding vectors generated by sine and cosine functions are superimposed. In the self-attention layer, the dot product similarity matrix between vectors is calculated, and attention weights are obtained after Softmax normalization, used to aggregate contextual information. After 12 layers of deep feature extraction, the 768-dimensional hidden state vector corresponding to the first special label CLS output from the last layer is extracted. This vector highly condenses key semantic information such as product name, ingredients, and origin in the product label, serving as the label semantic feature vector.

[0027] The WordPiece segmenter mentioned above refers to a word segmentation algorithm mainly used in natural language processing. It breaks words down into smaller sub-word units to solve the out-of-vocabulary word problem and improve the model's ability to generalize to word roots and affixes.

[0028] For the product appearance texture data, it is input into a ResNet-50 deep residual network. The data first passes through a... Convolutional layers and max-pooling layers reduce the spatial size to one-quarter of the original image. This is followed by four convolutional stages containing 3, 4, 6, and 3 residual blocks respectively. Each residual block employs a bottleneck structure... , , The system consists of three cascaded convolutional layers, with a skip connection mechanism introduced to mitigate the vanishing gradient problem. The fourth stage outputs a feature map with 2048 channels, preserving both high-level abstract semantics and low-level texture details. To accommodate the varying sizes of cross-border goods, a spatial pyramid pooling module is connected to the end of the ResNet-50. This module uses... , , Three grid scales divide the feature map into different regions. Max pooling is performed within each region, generating 1, 4, and 16 feature values ​​respectively. These 21 feature values ​​are flattened and concatenated to form a fixed-length visual feature representation. This representation is further reduced to 768 dimensions through a fully connected layer and L2 normalized to construct a visual morphological feature vector aligned with the text feature dimensions. Table 1 lists the parameter configurations and output dimensions of key network layers in the two-stream feature encoding process.

[0029] Table 1. Key Parameter Configuration Table for Feature Coding Network; As shown in Table 1, by strictly controlling the output dimension, it is ensured that the subsequent cross-modal alignment analysis can be performed in a unified metric space.

[0030] Please see Figure 1 and Figure 4 S3: Utilizing a multi-head cross-attention mechanism, calculate the weighted Euclidean distance between the label semantic feature vector and the visual morphological feature vector in the feature space to quantify the logical matching degree of the image and text information. Combined with preset semantic consistency constraints, construct an image-text semantic alignment matrix. The specific steps of S3 are as follows: S31: Map the label semantic feature vector to the query vector, and the visual morphological feature vector to the key vector and value vector. Calculate the dot product between the query vector and the key vector to scale the attention score using a multi-head attention mechanism, and generate an attention weight distribution to represent the importance of the features. S32: Based on the attention weight distribution, the corresponding dimensions of the label semantic feature vector and the visual morphological feature vector are weighted. For each semantic feature channel, the Euclidean distance between it and the corresponding visual feature channel is calculated, and the weighted sum is combined with the feature importance weight to quantify the difference between image and text features in the semantic space. S33: The weighted distances calculated in S32 are combined according to the arrangement order of feature channels. Combined with preset semantic consistency constraints, the distance values ​​are normalized and outlier truncation is performed to construct a text-image semantic alignment matrix to characterize the degree of text-image logical conflict. The calculation process of the weighted Euclidean distance specifically includes: Obtain the first weighted value after attention mechanism Each label semantic feature vector component With the Each visual morphological feature vector component And the attention importance weight corresponding to this dimension. ; The local differences of image and text features in each dimension are calculated using the weighted square difference formula, and the overall weighted Euclidean distance value is obtained by square root operation. Weighted Euclidean distance The calculation formula is: ; in, The weighted Euclidean distance between the semantic feature vector of the label and the visual morphological feature vector represents the distance between them. This represents the total dimension of the feature vector. Representing the Attention weight coefficients for each feature dimension The semantic feature vector of the representative label is in the first position. The numerical value of dimension, The visual morphological feature vector is represented in the first place. The numerical value of the dimension.

[0031] The 768-dimensional label semantic feature vector With 768-dimensional visual morphological feature vectors Input a multi-head cross-attention module. This module is configured with 8 parallel processing heads, each corresponding to a subspace dimension of 96. First, the label semantic feature vectors are processed... The cross-attention configuration in this embodiment involves multiplication with three learnable weight matrices. The label semantic feature vector is... It is mapped to a query vector, while the visual morphological feature vector is mapped to a query vector. It is simultaneously mapped to both key vectors and value vectors. Specifically, for the first... Each attention head calculates the dot product of the query vector and the key vector, and divides the result by the scaling factor. To prevent gradient vanishing, the result is processed by the Softmax function to generate an attention weight distribution vector of dimension 768. Among them, the first element Represents the eigenvector of the th The importance of each channel in the image-text alignment judgment task. For example, when detecting whether the brand logo and brand text are consistent, the feature channel responsible for encoding shape and spelling will receive a high weight of nearly 0.9, while the channel encoding background color may only have a weight of 0.1.

[0032] Based on the acquired attention weight distribution, a weighted Euclidean distance between image and text features is calculated. This distance aims to quantify the logical deviation between the product label's claimed content and the actual appearance. Using Python's NumPy library or PyTorch tensor operations, the label's semantic feature vector is first obtained. The Each component value and visual morphological feature vector The Each component has a numerical value. The difference between these two values ​​is subtracted and squared to obtain the component difference value. This component difference value is then multiplied by the corresponding attention weight. Repeat this operation for all 768 dimensions, summing all weighted difference values ​​and taking the square root of the sum to obtain the final weighted Euclidean distance.

[0033] Weighted Euclidean distance The calculation formula is: in, The weighted Euclidean distance between the semantic feature vector of the label and the visual morphological feature vector; The total dimension of the feature vector is 768 in this embodiment; Representing the Attention weight coefficients for each feature dimension are used to adjust the contribution of different feature channels to the final distance; The semantic feature vector of the representative label is in the first position. The numerical value of the dimension; The visual morphological feature vector is represented in the first place. The numerical value of the dimension.

[0034] The aforementioned attention weight coefficient refers to the numerical value generated in the attention mechanism by calculating the correlation between the query vector and the key vector. This value reflects the degree of attention the model pays to a specific input part when processing information.

[0035] Label semantic feature vector The fifth dimension value of 0.85 is related to the visual morphological feature vector. The value of the fifth dimension, 0.20, is substituted into the above calculation logic. Assume that the weights of this dimension are assigned by the attention mechanism. The value is 0.92. The locally weighted difference for this dimension is calculated by multiplying 0.92 by the square of the difference between 0.85 and 0.20, i.e. If we assume that the weighted sum of differences for other dimensions is 1.2, then the final weighted Euclidean distance result is... This result indicates a significant conflict between text and image features in this dimension.

[0036] The calculated distance values ​​are used as core elements to construct a text-image semantic alignment matrix, combined with pre-defined semantic consistency constraints. The constraints are implemented using a sigmoid activation function, mapping the unbounded distance values ​​to the range of 0 to 1; values ​​closer to 1 indicate more severe conflicts. Furthermore, a truncation threshold is set to filter out minute noise fluctuations with distances less than 0.1, ultimately forming a feature matrix describing the degree of logical matching between the text and images.

[0037] Please see Figure 1 and Figure 5 S4: Input the image-text semantic alignment matrix into the preset risk assessment model for nonlinear regression analysis and feature weighting to calculate the predicted violation probability. Compare the predicted violation probability with a dynamic security threshold based on historical customs clearance data to generate a cross-border commodity compliance inspection result. The specific steps of S4 are as follows: S41: Flatten the image-text semantic alignment matrix into a one-dimensional feature vector, input it into the risk assessment model consisting of multiple fully connected layers and nonlinear activation functions, perform nonlinear regression on the feature vector through the weight parameters optimized by the backpropagation algorithm, and output the predicted value of the violation probability that represents the level of violation risk of the product. S42: Obtain historical customs clearance compliance data for goods within a specified time period, calculate the statistical distribution characteristics of the risk values ​​of historical compliant goods, dynamically calculate the critical value for risk judgment based on the normal distribution assumption and confidence interval theory, and generate a dynamic safety threshold. S43: A comparator is used to compare the predicted probability of violation with a dynamic safety threshold. If the predicted value is greater than the threshold, it is judged as a high-risk violation; if the predicted value is less than or equal to the threshold, it is judged as a low-risk compliance. Based on this, a compliance detection result for cross-border goods, including a risk level label and a confidence score, is generated. The generation process of the dynamic safety threshold specifically includes: Retrieve the most recent data from the customs historical database. A historical safety sample set is constructed by predicting the probability of non-compliance for all historical goods that were deemed compliant and released within a given day. Calculate the arithmetic mean and standard deviation of all probability values ​​in the historical safety sample set, and combine them with the preset safety redundancy coefficient to determine the dynamic upper bound of risk assessment based on the statistical 3-sigma principle; Dynamic security threshold The calculation formula is: ; in, Represents a dynamic security threshold. This represents the total number of samples in the historical safe sample set. Representing the The predicted probability of non-compliance for a historical compliant product. The arithmetic mean of the predicted probability values ​​of all historical compliant products. This represents the preset safety redundancy coefficient.

[0038] The constructed image-text semantic alignment matrix is ​​flattened into a one-dimensional feature vector and input into the risk assessment model. This model employs a three-layer fully connected neural network architecture with 1024, 512, and 1 neurons respectively. The input layer receives the flattened vector and performs a non-linear transformation using the ReLU activation function. The hidden layer further abstracts the feature combination. The output layer uses the Sigmoid activation function to compress the output value into a closed interval between 0 and 1; this output value is the predicted violation probability. During model training, a historical product dataset containing compliance and violation labels is used. The prediction error is calculated using the binary cross-entropy loss function, and parameters are updated using a stochastic gradient descent algorithm with a momentum of 0.9, enabling the model to accurately capture the mapping relationship between the feature alignment matrix and the violation risk.

[0039] The binary cross-entropy loss function mentioned above is a commonly used loss function in binary classification problems. It measures the model's prediction error by calculating the difference between the predicted probability distribution and the true label distribution.

[0040] To ensure the objectivity and dynamic adaptability of the judgment, a dynamic safety threshold needs to be constructed. From the customs integrated business database, historical compliant goods records marked as released and for which no subsequent consumer complaints have been received within the past 30 days are extracted using SQL queries. The corresponding model-predicted risk values ​​are extracted from these records to construct a historical safety sample set. Assuming the sample set contains 10,000 data points, statistical analysis tools are used to calculate the arithmetic mean and standard deviation of these risk values. For example, the average risk value is calculated. Standard deviation Considering the extremely low tolerance of customs inspections for missed detection rates, a safety redundancy coefficient of 3 is set, following the 3-sigma principle in statistics, to cover a normal fluctuation range of 99.7%.

[0041] Substitute the parameter values ​​obtained from the above statistics into the dynamic safety threshold calculation logic. The calculation process is as follows: add the average value of 0.15 to three times the standard deviation of 0.09, which equals 0.24. At this point, the calculated dynamic safety threshold is 0.24. This result indicates that, based on the current customs clearance environment and historical data, any product with a risk prediction value exceeding 0.24 is a statistical outlier and is highly likely to be a non-compliant product.

[0042] Finally, a comparison and decision-making process is performed. The predicted violation probability is compared with the calculated threshold of 0.24. If the predicted value is 0.85, since 0.85 is greater than 0.24, the system determines the product as a high-risk violation and automatically generates an inspection report containing a risk label, a prediction confidence level of 85%, and a heatmap of the corresponding text-image discrepancy area. The system then instructs the assembly line sorting device to push the package into the manual review area. If the predicted value is 0.12, since 0.12 is less than or equal to 0.24, the system determines it as a low-risk compliance product and instructs it to be released. Table 2 lists the actual system performance data under different threshold setting strategies.

[0043] Table 2. Performance Comparison of Risk Assessment Strategies; As shown in Table 2, the dynamic threshold generation method based on the statistical characteristics of historical data described in this embodiment can significantly reduce the false alarm rate and greatly improve customs clearance efficiency while ensuring an extremely low false alarm rate of 0.5%.

[0044] A cross-border commodity compliance inspection system based on image recognition, the system being used to execute the aforementioned cross-border commodity compliance inspection method based on image recognition, the system comprising: The data acquisition and separation module is deployed at the customs inspection site and includes a high-resolution industrial camera, photoelectric sensors, and an image preprocessing unit. This module is equipped with a differentiable binarized text detection subunit based on a ResNet-18 backbone network and an edge texture separation subunit based on the Sobel operator and color histogram analysis. The specific execution process of this module is as follows: the photoelectric sensor triggers the camera to acquire images, and Gaussian filtering is used to remove noise. Subsequently, the text detection subunit predicts the probability map and threshold map of the text region, generates a text mask, corrects the text region, and outputs the packaging label area data. Simultaneously, the edge texture separation subunit uses a reverse mask to shield the text region, extracts the gradient magnitude and texture features of the remaining region, and outputs the product appearance texture data.

[0045] The dual-stream feature encoding module converts raw image data into high-dimensional feature vectors that are understandable to computers. This module integrates a CRNN character recognition engine, a BERT language model, and a ResNet-50 deep convolutional network. The specific execution process is as follows: First, the packaging label region data is converted into a text sequence, which is then input into the BERT model to extract a 768-dimensional label semantic feature vector containing contextual information. In parallel, the product appearance texture data is input into the ResNet-50 network to extract deep feature maps. Multi-scale features are aggregated through spatial pyramid pooling layers, and finally mapped to a 768-dimensional visual morphological feature vector through a fully connected layer, ensuring that the data from both modalities are aligned in the same dimensional space.

[0046] The cross-modal alignment analysis module is used to quantify the consistency of text and image information. Internally, this module includes a multi-headed cross-attention calculation unit and a weighted Euclidean distance calculation unit. The specific execution process is as follows: the label semantic feature vector is used as the query vector, and the visual morphological feature vector is used as the key and value vectors, respectively. These are input into the attention calculation unit to obtain the attention weight distribution. Subsequently, the weighted Euclidean distance calculation unit combines this weight distribution to calculate the weighted distance between the two feature vectors in the feature space. After sigmoid function mapping and truncation, this distance generates a text-image semantic alignment matrix. The value of this matrix directly reflects the degree of logical conflict between the product label description and the appearance features.

[0047] The risk assessment and decision-making module is the core of the system's decision-making process, comprising a pre-trained risk assessment neural network and a dynamic threshold generator. The module's execution process is as follows: the image-text semantic alignment matrix is ​​input into a three-layer fully connected network to calculate the predicted probability of violation. Simultaneously, the dynamic threshold generator analyzes historical customs clearance data from the past 30 days in real time, calculating a dynamic safety threshold based on the mean and standard deviation. A comparator within the module compares the predicted value with the dynamic threshold. If the predicted value is higher than the threshold, a high-risk alarm is triggered, and a detection report is generated instructing the sorting equipment to intercept the goods; otherwise, the goods are released.

[0048] The above embodiments illustrate preferred embodiments of the present invention. Any equivalent adjustments to the technical solution based on software engineering methods are within the scope of protection, including but not limited to: implementing algorithm logic using different programming languages, refactoring functional modules into services, adjusting data interaction protocols, and optimizing resource scheduling strategies. Any implementation scheme derived from reasonable modifications to the data processing flow, service call chain, or system architecture layer without departing from the core technology of the present invention should be considered within the protection scope defined by the technical solution of the present invention.

Claims

1. A method for compliance inspection of cross-border goods based on image recognition, characterized in that, Includes the following steps: S1: Real-time acquisition of images of cross-border goods to be inspected through high-speed industrial cameras on the customs inspection line; use optical character recognition operators to perform text region localization and character segmentation on the images of cross-border goods to be inspected, and extract packaging label area data; use edge detection operators to perform contour fitting and texture segmentation on the images of cross-border goods to be inspected, and peel off the texture data of the product appearance. S2: Input the packaging label area data into a pre-trained semantic coding network for semantic space alignment mapping to generate a label semantic feature vector. Use a multi-scale convolutional network to perform deep feature encoding and pyramid pooling on the product appearance texture data to construct a visual morphology feature vector. S3: Calculate the weighted Euclidean distance between the label semantic feature vector and the visual morphological feature vector in the feature space using a multi-head cross-attention mechanism, quantify the logical matching degree of the image and text information, and construct an image and text semantic alignment matrix in combination with preset semantic consistency constraints. S4: Input the image-text semantic alignment matrix into a preset risk assessment model for nonlinear regression analysis and feature weighting, calculate the predicted value of the violation probability, compare the predicted value of the violation probability with the dynamic security threshold based on historical customs clearance data, and generate the compliance detection result of cross-border goods.

2. The image recognition-based compliance inspection method for cross-border goods according to claim 1, characterized in that, The specific steps of S1 are as follows: S11: In response to the photoelectric sensor trigger signal on the customs inspection line, control the high-speed industrial camera to acquire high-resolution original images of the cross-border goods to be inspected in a multi-angle light source environment, and use a Gaussian filter to remove electronic noise generated during the image acquisition process to generate basic image data to be processed. S12: Input the basic image data into a text detection network based on differentiable binarization, predict the probability map and threshold map of text instances in the image, generate a binarized mask of the text region through a probability threshold fusion algorithm, and use a perspective transformation matrix to perform geometric correction and character slicing on the text region within the mask, thereby extracting the packaging label region data. S13: The gradient magnitude and non-maximum suppression are performed on the basic image data using the multi-scale Canny edge detection operator to identify the structural contour edges of the product appearance. The texture interference of the text region is filtered out using the inverse mask of the text region binarization mask generated in S12. The color histogram features and gray-level co-occurrence matrix features of the remaining region are extracted, thereby stripping the texture data of the product appearance.

3. The image recognition-based compliance inspection method for cross-border goods according to claim 1, characterized in that, The specific steps of S2 are as follows: S21: The character sequence in the packaging label area data is segmented and embedded with position encoding, and then input into a pre-trained language model based on the Transformer architecture for bidirectional contextual semantic feature extraction. The long-distance dependency between characters is captured through a self-attention mechanism, and the output hidden state is globally averaged and pooled to generate the label semantic feature vector. S22: Input the product appearance texture data into a deep residual network for multi-level feature extraction to obtain feature maps at different resolutions. Use a spatial pyramid pooling layer to perform multi-scale grid partitioning and max pooling operations on the deep feature maps. Cascade and fuse the pooling results at different scales and perform dimensionality reduction processing to construct the visual morphology feature vector.

4. The cross-border commodity compliance inspection method based on image recognition according to claim 1, characterized in that, The specific steps of S3 are as follows: S31: Map the label semantic feature vector to a query vector, map the visual morphological feature vector to a key vector and a value vector, and use a multi-head attention mechanism to calculate the dot product between the query vector and the key vector to scale the attention score, thereby generating an attention weight distribution to characterize the importance of the features. S32: Based on the attention weight distribution, the corresponding dimensions of the label semantic feature vector and the visual morphological feature vector are weighted. For each semantic feature channel, the Euclidean distance between it and the corresponding visual feature channel is calculated, and the weighted sum is combined with the feature importance weight to quantify the difference between image and text features in the semantic space. S33: Combine the weighted distances calculated in S32 according to the arrangement order of the feature channels, and combine them with the preset semantic consistency constraints to normalize the distance values ​​and truncate outliers to construct the image-text semantic alignment matrix used to characterize the degree of image-text logical conflict.

5. The image recognition-based compliance inspection method for cross-border goods according to claim 1, characterized in that, The specific steps of S4 are as follows: S41: Flatten the image-text semantic alignment matrix into a one-dimensional feature vector, input it into the risk assessment model composed of multiple fully connected layers and nonlinear activation functions, perform nonlinear regression operation on the feature vector through the weight parameters optimized by the backpropagation algorithm, and output the predicted value of the violation probability that represents the level of violation risk of the product. S42: Obtain historical customs clearance compliance data for goods within a specified time period, calculate the statistical distribution characteristics of the risk values ​​of historical compliant goods, dynamically calculate the critical value for risk judgment based on the normal distribution assumption and confidence interval theory, and generate the dynamic safety threshold. S43: Use a comparator to compare the predicted value of the violation probability with the dynamic security threshold. If the predicted value is greater than the threshold, it is determined to be a high-risk violation. If the predicted value is less than or equal to the threshold, it is determined to be a low-risk compliance. Based on this, generate the compliance detection result of the cross-border goods, which includes a risk level label and a confidence score.

6. The cross-border commodity compliance inspection method based on image recognition according to claim 2, characterized in that, The process of extracting the data from the packaging label area specifically includes: A feature pyramid extractor containing a multi-layer convolutional neural network is constructed to extract shared feature maps of the basic image data at different resolutions, and deformable convolutional layers are used to adapt to the irregular shape of the text region. The prediction head is used to perform pixel-level classification and regression prediction on the shared feature map, generating a probability map representing the possibility of the text center region and a threshold map representing the text boundary threshold, respectively. The probability map and the threshold map are weighted and fused by the binarization formula to obtain a clear text instance binarization mask. The minimum bounding rectangle and rotation angle of the text instance are calculated based on the binarized mask. The tilted or distorted text region is corrected into a standard text image in the horizontal direction using the perspective transformation matrix. The standard text image is then segmented and serialized at the character level to obtain the structured packaging label region data.

7. The image recognition-based compliance inspection method for cross-border goods according to claim 3, characterized in that, The process of constructing the visual morphological feature vector specifically includes: The product appearance texture data is input into the ResNet-50 backbone network and processed through four stages of residual block processing to extract the fourth stage feature map containing high-level semantic information and low-level detail information. The fourth-stage feature map is processed in parallel using a dilated spatial convolution pooling pyramid module. Feature sampling is performed using dilated convolution kernels with different dilation rates to capture multi-scale texture context information within different receptive fields. The output feature maps of each parallel branch are concatenated along the channel dimension, utilizing... The convolutional layer compresses the concatenated feature map to a preset target number of channels and performs global average pooling on the compressed feature map to obtain a one-dimensional depth visual feature representation, thereby constructing the visual morphological feature vector.

8. The image recognition-based cross-border commodity compliance inspection method according to claim 4, characterized in that, The calculation process of the weighted Euclidean distance specifically includes: Obtain the first weighted value after attention mechanism The label semantic feature vector components With the Each of the visual morphological feature vector components And the attention importance weight corresponding to this dimension. ; The local differences of image and text features in each dimension are calculated using the weighted square difference formula, and the overall weighted Euclidean distance value is obtained by square root operation. The weighted Euclidean distance The calculation formula is: ； in, The weighted Euclidean distance represents the relationship between the semantic feature vector of the label and the visual morphological feature vector. This represents the total dimension of the feature vector. Representing the Attention weight coefficients for each feature dimension The semantic feature vector of the label represents the first... The numerical value of dimension, The visual morphological feature vector represents the first... The numerical value of the dimension.

9. The cross-border commodity compliance inspection method based on image recognition according to claim 5, characterized in that, The process of generating the dynamic security threshold specifically includes: Retrieve the most recent data from the customs historical database. A historical safety sample set is constructed by predicting the probability of non-compliance for all historical goods that were deemed compliant and released within a given day. Calculate the arithmetic mean and standard deviation of all probability values ​​in the historical safety sample set, and combine them with the preset safety redundancy coefficient to determine the dynamic upper bound of risk assessment based on the statistical 3-sigma principle; The dynamic security threshold The calculation formula is: ； in, Represents the dynamic security threshold, This represents the total number of samples in the historical safe sample set. Representing the The predicted probability of non-compliance for a historical compliant product. The arithmetic mean of the predicted probability values ​​of all historical compliant products. This represents the preset safety redundancy coefficient.

10. A cross-border commodity compliance inspection system based on image recognition, characterized in that, The system is used to implement the image recognition-based cross-border commodity compliance detection method according to any one of claims 1-9, and the system includes: The data acquisition and separation module is used to acquire images of cross-border goods to be inspected in real time through high-speed industrial cameras in the customs inspection line, use optical character recognition operators to perform text region localization and character segmentation to extract packaging label area data, and use edge detection operators to perform contour fitting and texture segmentation to peel off the appearance texture data of the goods. The dual-stream feature encoding module is used to input the packaging label area data into the semantic encoding network to generate the label semantic feature vector, and input the product appearance texture data into the multi-scale convolutional network to construct the visual morphological feature vector. The cross-modal alignment analysis module is used to calculate the weighted Euclidean distance in the feature space using a multi-head cross-attention mechanism and combine it with semantic consistency constraints to construct the image-text semantic alignment matrix that quantifies the degree of image-text logical matching. The risk assessment and decision-making module is used to input the image-text semantic alignment matrix into the risk assessment model to calculate the predicted value of the violation probability, and compare it with the dynamic security threshold generated based on historical customs clearance data to generate the compliance detection result of the cross-border goods.

Images