Intelligent express parcel automatic sorting method based on deep learning

Abstract

The invention discloses an intelligent express parcel automatic sorting method based on deep learning, and the method comprises the steps: collecting a multi-view image, carrying out the standardization enhancement processing, carrying out the bidirectional space sequence coding through a Vision Mama structured state space model, obtaining a fusion visual feature map, inputting a preliminary classification network, and carrying out the primary classification processing, and according to the initial classification category, dynamically selecting expert sub-networks of the MoE-Mama hybrid expert network to complete deep feature reconstruction and fine classification, and finally, according to the final classification category, generating a control instruction to realize automatic guide positioning and accurate sorting. According to the method, the parcel sorting accuracy and efficiency are remarkably improved, and the method is suitable for intelligent logistics sorting operation in a complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and intelligent logistics, and particularly relates to an intelligent express package automatic sorting method based on deep learning. BACKGROUND

[0002] Intelligent logistics is an important development direction of the modern logistics industry, and efficient automatic sorting of express packages is one of the key technologies in the field of intelligent logistics. In recent years, with the continuous development of artificial intelligence technology, computer vision technology and automatic control technology, the automatic sorting system based on image recognition has gradually become the main technical route in the field of logistics sorting.

[0003] The traditional logistics sorting system early adopted a mechanical structure with manual assistance for operation, which had the disadvantages of low automation degree, insufficient sorting efficiency and easy human error, and was difficult to adapt to the growing demand for express business volume. In order to improve efficiency, relevant field technical personnel began to explore intelligent and automatic sorting technology based on image recognition and deep learning, typical schemes of which include image feature recognition methods based on convolutional neural network (CNN) and Transformer model.

[0004] The deep learning method represented by convolutional neural network has good application effect on image feature recognition task, mainly through multi-layer convolution structure to extract local features in package image, supplemented by pooling operation to reduce feature dimension, realizing feature extraction and preliminary classification. However, the convolutional neural network has obvious decline in recognition accuracy and robustness in the face of complex environment (such as obvious light change, multi-package occlusion stacking, and multiple morphologies interweaving), which is difficult to meet the demand of high-precision sorting of logistics automation system. In addition, due to the limited global modeling ability of CNN on spatial position relationship, the recognition result error is large, which directly affects the subsequent sorting decision.

[0005] The Transformer model realizes the modeling of long-distance dependence relationship of features in each region of the image through the self-attention mechanism, which enhances the image recognition ability of CNN in complex environment. However, the traditional Transformer model usually needs to cut the image into a large number of small image blocks, which significantly increases the computational complexity and leads to low recognition efficiency. At the same time, the self-attention mechanism of Transformer has the problem of too large calculation overhead, which is difficult to apply in the logistics sorting system with high real-time requirement.

[0006] Meanwhile, the current mainstream automatic sorting system of logistics adopts a single deep learning network to uniformly process all types of packages, without fully considering the differences in package types and forms, resulting in low sorting efficiency and unstable classification accuracy. Especially in the actual express logistics environment, different types, sizes and materials of packages coexist, and the processing system needs to have high flexibility and adaptability. The traditional single model decision strategy obviously lacks generalization ability and efficiency.

[0007] In addition, the existing logistics sorting system usually takes the visual feature extraction link and the subsequent decision sorting link as independent units, lacking effective information interaction and fusion mechanism. This kind of split design mode causes the loss of information transmission, which is not conducive to the improvement of the overall precision and efficiency of the system, and restricts the further development of intelligent logistics sorting technology.

[0008] Therefore, how to provide an intelligent express package automatic sorting method based on deep learning is a problem that those skilled in the art need to solve. SUMMARY

[0009] An object of the present application is to provide an intelligent express package automatic sorting method based on deep learning. The present application adopts a method of deep fusion of Vision Mamba structured state space model and MoE-Mamba hybrid expert network. The method processes the real-time collected multi-view package images through size standardization and multi-scale enhancement, then uses the Vision Mamba structured state space model to perform bidirectional spatial sequence coding and cross-scale feature fusion processing on the images to generate a fusion visual feature map, obtains the initial classification category of the package through a preliminary classification network, dynamically selects the expert sub-network in the MoE-Mamba hybrid expert network according to the initial classification category, and performs feature reconstruction and fine classification processing. Finally, according to the final classification category, the corresponding path information is obtained by querying the sorting path database, and a control instruction is generated to realize the automatic guiding and positioning of the package and the accurate sorting operation, effectively solving the problems of insufficient recognition accuracy and low sorting efficiency of multiple types and multiple forms of packages in complex environments, and having the beneficial effects of high recognition accuracy, strong generalization ability and significantly improved real-time processing efficiency.

[0010] According to the intelligent express package automatic sorting method based on deep learning of the present application, the method comprises the following steps:

[0011] S1, real-time multi-angle high-definition camera equipment is used to collect multi-view images of the package to be sorted, and size standardization and multi-scale image enhancement processing are performed to obtain a set of standardized multi-view images;

[0012] S2, encode the standardized multi-view image set in a bidirectional spatial sequence using a Vision Mamba structured state space model, to obtain a wrapped global contour feature map and a local texture feature map, respectively;

[0013] S3, interactively fuse the global contour feature map and the local texture feature map using a cross-scale feature fusion method to generate a fused visual feature map;

[0014] S4, input the fused visual feature map into a preliminary classification network for preliminary classification processing to obtain an initial classification category of the package;

[0015] S5, dynamically select a corresponding expert subnetwork in the MoE-Mamba hybrid expert network according to the initial classification category, and perform deep feature reconstruction and fine classification processing on the fused visual feature map to obtain a final classification category of the package;

[0016] S6, query a preset sorting path database according to the final classification category, extract corresponding automatic sorting path information, and generate a control instruction for driving a sorting device to perform automatic guided positioning and accurate sorting of the package.

[0017] Optionally, the S1 specifically comprises:

[0018] S11, use a multi-angle high-definition camera device with a resolution of 1920x1080 pixels to collect original images of the top view, side view and front view of the package to be sorted at a frequency of 30 frames per second;

[0019] S12, adjust the original images to standardized images with a uniform size of 512x512 pixels using a bilinear interpolation algorithm;

[0020] S13, perform gamma transformation enhancement processing on the standardized images:

[0021] O(x,y)=c×I(x,y) γ ;

[0022] Wherein, O(x,y) represents the output image after gamma transformation, I(x,y) represents the input standardized image, c represents the proportion coefficient, the value range is 0.8-1.2, and γ represents the gamma correction parameter, the value range is 0.4-0.7;

[0023] S14, perform multi-scale smoothing processing on the standardized images after gamma transformation enhancement using two-dimensional Gaussian filters of different scales respectively;

[0024] S15, the multi-scale smoothing processed normalized image is assigned with feature weight by scale, and is fused into a single image by linear weighted superposition of pixels, and the single image formed by each view fusion is combined to construct the normalized multi-view image set.

[0025] Optionally, the Vision Mamba structured state space model specifically comprises:

[0026] The input normalized multi-view image set is subjected to multi-scale convolution feature extraction by using a deep convolution operator to obtain preliminary visual features of different scales;

[0027] The preliminary visual features are subjected to state space modeling by using a structured state space model:

[0028]

[0029] wherein, represents a state vector of time t at scale s, represents a preliminary visual feature input vector of time t at scale s, represents an output state feature vector of time t at scale s, A is a state transition matrix, B is an input control matrix, C is a state output matrix, and D is an input direct-through matrix;

[0030] The output state feature vector is subjected to forward and backward state space encoding by using a bidirectional structured state space model;

[0031] Forward state space encoding:

[0032]

[0033] Backward state space encoding:

[0034]

[0035] wherein, respectively represent forward and backward state space encoding vectors of time t at scale s, A f , A b respectively represent forward and backward state transition matrices, B f , B b respectively represent forward and backward input control matrices;

[0036] The forward and backward state space encoding vectors are subjected to feature fusion processing:

[0037]

[0038] wherein, denotes a visual state feature vector of a scale s at time t fused by a bidirectional structured state space encoding mechanism, C f b denote forward and backward state output matrices respectively, D ′ denotes a bidirectional fusion pass-through matrix.

[0039] The visual state feature vectors of different scales are fused by a scale interaction fusion mechanism. The scale interaction fused visual feature vectors are obtained according to a preset scale fusion weight.

[0040] The scale interaction fused visual feature vectors are subjected to nonlinear mapping processing by a Gaussian error linear unit activation function to obtain a final visual feature output vector.

[0041] Optionally, the S2 specifically comprises:

[0042] S21, dividing each image in the standardized multi-view image set into non-overlapping image sub-regions with a size of 16x16 pixels, and flattening each sub-region into a one-dimensional initial feature vector with a length of 256;

[0043] S22, performing bidirectional state space encoding processing on each one-dimensional initial feature vector according to the Vision Mamba structured state space model in claim 3 to obtain corresponding forward state space feature vectors and backward state space feature vectors respectively;

[0044] S23, calculating bidirectional difference fusion feature vectors in the sub-region based on the complementary difference features of the forward state space feature vectors and the backward state space feature vectors, and the specific calculation formula is:

[0045]

[0046] wherein, denotes a bidirectional difference fusion feature vector of the sub-region r at time t, denote forward and backward state space feature vectors respectively, W f b denote forward and backward fusion weight matrices respectively, b denotes a fusion bias vector, and σ(·) denotes a Sigmoid activation function.

[0047] S24, rearranging the bidirectional difference fusion feature vectors of each sub-region according to the original image space position to construct a corresponding spatial sequence feature mapping matrix.

[0048] ​​S25, using an adaptive image feature attention mechanism to calculate feature weight coefficients of each sub-region in space on the spatial sequence feature mapping matrix, and performing element-by-element weighting reconstruction on the spatial sequence feature mapping matrix according to the feature weight coefficients to obtain a global contour feature map with a resolution of 32x32;

[0049] S26, using an adaptive two-dimensional Fourier frequency domain decomposition algorithm to extract feature spectrum components with a frequency range of 0.3f max ~0.6f max , wherein f max represents the highest frequency value after two-dimensional Fourier transform, and performing inverse Fourier transform reconstruction on the spectrum components to obtain a local texture feature map with a resolution of 32x32.

[0050] Optionally, the S3 specifically comprises:

[0051] S31, respectively performing up-sampling processing on the global contour feature map and the local texture feature map by a bicubic interpolation method with a scale factor of 2 to obtain a unified scale feature map with a resolution of 64x64;

[0052] S32, respectively performing smoothing processing on the unified scale feature map using a two-dimensional Gaussian low-pass filter, and the scale parameter of the filter is in a range of 0.5~1.0;

[0053] S33, respectively performing feature standardization processing on the smoothed unified scale feature map, and subtracting the mean value of the feature map from each element of the feature map and then dividing by the standard deviation of the feature map to obtain a standardized feature map;

[0054] S34, calculating a spatial element-by-element feature difference vector based on the standardized feature map:

[0055] d(x,y)=F g g(x,y)-F l l(x,y);

[0056] wherein d(x,y) represents a feature difference vector at position (x,y), Fg(x,y) and Fl(x,y) respectively represent feature values of the global contour standardized feature map and the local texture standardized feature map at position (x,y);

[0057] S35, generating a difference weight mapping matrix by using an adaptive weight mapping processing on the feature difference vector element-by-element:

[0058]

[0059] wherein M(x,y) represents a difference weight at position (x,y), and λ represents a weight adaptive mapping parameter, and the value range is 1.5~2.5;

[0060] S36, element-wise weighted fusion is performed on the global contour normalized feature map and the local texture normalized feature map according to the difference weight mapping matrix, to obtain a fused visual feature map:

[0061] F 融合 (x,y)=M(x,y)·F g (x,y)+[1-M(x,y)]·F l (x,y);

[0062] Wherein, F fusion (x, y) represents a feature value of the fused visual feature map at position (x, y).

[0063] Optionally, the preliminary classification network specifically comprises:

[0064] A convolution operation with a convolution kernel size of 3x3 is used to perform coarse-scale feature encoding processing on the input fused visual feature map, to obtain a preliminary coarse-scale feature map;

[0065] A convolution operation with a convolution kernel size of 1x1 is used to perform fine-scale feature encoding processing on the input fused visual feature map, to obtain a preliminary fine-scale feature map;

[0066] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are calculated through a full connection layer and a Softmax function to obtain an adaptive scale fusion weight vector;

[0067] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are subjected to channel-by-channel weighted fusion based on the adaptive scale fusion weight vector, to obtain a residual fusion feature map;

[0068] The residual fusion feature map is subjected to channel direction average pooling and maximum pooling processing, and the pooled features are subjected to two-layer full connection mapping processing with a nonlinear activation function, to obtain a channel attention weight vector;

[0069] The residual fusion feature map is subjected to channel-by-channel weighted processing based on the channel attention weight vector, to obtain a preliminary classification feature map;

[0070] After the preliminary classification feature map is flattened into a one-dimensional vector, it is input into a full connection classifier to generate a preliminary classification prediction class probability distribution.

[0071] Optionally, S4 specifically comprises:

[0072] S41, a maximum pooling operation with a size of 2x2 and a step of 2 is used to perform spatial down-sampling processing on the fused visual feature map, to obtain a reduced dimension visual feature map with a size of 32x32;

[0073] S42, input the dimension-reduced visual feature map into a preliminary classification network, and sequentially perform coarse-scale feature coding processing, fine-scale feature coding processing, adaptive scale fusion processing, and channel attention feature enhancement processing on the dimension-reduced visual feature map through the preliminary classification network to obtain a preliminary classification feature map;

[0074] S43, flatten the preliminary classification feature map into a one-dimensional feature vector, input the one-dimensional feature vector into a fully connected classifier of the preliminary classification network, and generate a probability distribution vector of an initial classification category of the parcel;

[0075] S44, determine the initial classification category corresponding to the element with the maximum value in the probability distribution vector as the initial classification category of the parcel.

[0076] Optionally, the MoE-Mamba hybrid expert network specifically comprises:

[0077] According to the initial classification category, an MoE-Mamba hybrid expert network composed of a plurality of parallel expert sub-networks and a gating network is constructed, each expert sub-network comprising a structured state space module and a residual feature enhancement module;

[0078] The gating network calculates a dynamic selection probability of each expert sub-network based on a global average pooling feature vector of the input fusion visual feature map;

[0079] According to the dynamic selection probability of the gating network, a corresponding expert sub-network is selected, and the fusion visual feature map is input into the selected expert sub-network;

[0080] The expert sub-network performs dimension-reduced feature transformation on the input fusion visual feature map to obtain a dimension-reduced input feature;

[0081] The expert sub-network uses the structured state space module to perform state space feature mapping on the dimension-reduced input feature and outputs a state space feature;

[0082] The expert sub-network uses the residual feature enhancement module to perform residual enhancement processing on the state space feature and outputs an enhanced expert feature;

[0083] According to the dynamic selection probability calculated by the gating network, the enhanced expert features output by all expert sub-networks are weighted and fused to obtain a fusion output feature of the MoE-Mamba hybrid expert network;

[0084] The fusion output feature is input into a fully connected classifier for fine classification processing to obtain a fine classification category probability distribution of the parcel.

[0085] Optionally, the S5 specifically comprises:

[0086] S51, dynamically selecting a unique corresponding expert sub-network from the MoE-Mamba hybrid expert network according to the initial classification category, and inputting the fused visual feature map into the selected expert sub-network;

[0087] S52, the selected expert sub-network uses dimension reduction feature transformation operation to perform feature dimension compression processing on the fused visual feature map, and obtains a dimension reduction feature vector;

[0088] S53, the selected expert sub-network uses a structured state space module to perform state space feature mapping processing on the dimension reduction feature vector, and obtains a state space feature vector;

[0089] S54, the selected expert sub-network uses a residual feature enhancement module to perform residual enhancement processing on the state space feature vector, and obtains an enhanced expert feature vector;

[0090] S55, performing spatial feature reconstruction on the enhanced expert feature vector through a convolution operation with a size of 3*3 and a step of 1 and a global average pooling operation, to obtain a deep feature reconstruction vector with a length of 512;

[0091] S56, inputting the deep feature reconstruction vector into a first fully connected layer containing 256 hidden nodes and a second fully connected classification layer containing 20 output nodes, and obtaining a probability distribution vector wrapping the final classification category through a Softmax function;

[0092] S57, determining the corresponding final classification category according to the element with the maximum value in the probability distribution vector as the wrapped final classification category.

[0093] Optionally, the S6 specifically comprises:

[0094] S61, performing a category index matching operation in a preset sorting path database according to the final classification category, to obtain a unique sorting path code corresponding to the final classification category;

[0095] S62, extracting corresponding automatic sorting path information from the sorting path database according to the unique sorting path code, the automatic sorting path information including sorting target position coordinates, path node numbers, path order sequences and path control parameters;

[0096] S63, performing spatial coordinate conversion processing on the automatic sorting path information, comparing the sorting target position coordinates with the initial position coordinates of the package in real time, and calculating an automatic guide positioning vector:

[0097] V 导引 =(X 目标 -X 初始 ,Y 目标 -Y 初始Z 目标 -Z 初始 );

[0098] Wherein, X 目标 , Y 目标 , Z 目标 Respectively, the coordinate value of the sorting target position coordinate in the space rectangular coordinate system, X 初始 , Y 初始 , Z 初始 Respectively, the coordinate value of the real-time location coordinate of the package in the space rectangular coordinate system;

[0099] S64, according to the automatic guiding positioning vector, the space movement trajectory point array coordinates of the sorting equipment are calculated, and the movement trajectory is interpolated and smoothed;

[0100] S65, according to the sorting path control parameter, the sorting control instruction of the execution unit of the sorting equipment is generated in real time, and the instruction integrity and safety check are carried out;

[0101] S66, the sorting control instruction after the check is sent to the execution unit of the sorting equipment in real time, the sorting equipment is driven to move accurately along the movement trajectory point array coordinates, and the automatic guiding positioning of the package is realized;

[0102] S67, when the package moves to the target sorting position, the sorting equipment is controlled to execute the action to complete the accurate sorting work.

[0103] The beneficial effects of the present application are:

[0104] (1) the present application adopts the Vision Mamba structured state space model to realize bidirectional space sequence coding and cross-scale fusion of standardized multi-view images, effectively realizes accurate visual feature extraction of multiple types and multiple forms of packages in complex environment, significantly improves package recognition accuracy in complex scenarios, reduces visual recognition error, and improves package classification accuracy and stability.

[0105] (2) the present application introduces a preliminary classification network to realize coarse and fine scale coding, adaptive scale fusion and channel attention enhancement processing of the fused visual feature map, realizes efficient prediction of the primary classification category, significantly improves the recognition speed and accuracy of the preliminary classification stage, and shows better adaptability in the real-time operation environment of express package automatic sorting.

[0106] (3) The application has high classification accuracy and generalization ability. The expert subnetworks in the MoE-Mamba hybrid expert network are dynamically selected to reconstruct and finely classify the deep features of the fused visual feature maps, effectively solving the problem of flexible processing of multiple package classification in the prior art, breaking through the performance bottleneck of a single network model, and significantly improving the final package classification accuracy and decision efficiency, thereby effectively improving the practical application ability of the intelligent sorting system.

[0107] (4) The application realizes accurate automatic guiding and positioning of the package and sorting operation by performing spatial coordinate conversion on the automatic sorting path information and real-time safety verification of the sorting control instruction, significantly improves the control accuracy and operation stability of the automatic sorting equipment, effectively reduces the operation error and equipment operation risk, and enhances the engineering applicability of the automatic sorting system in actual complex operation environment. BRIEF DESCRIPTION OF DRAWINGS

[0108] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, which together with the embodiments of the application, serve to explain the application, and do not constitute a limitation on the application. In the drawings:

[0109] Fig. 1 A schematic diagram of the overall process of an intelligent express package automatic sorting method based on deep learning is provided for the application;

[0110] Fig. 2 A schematic diagram of the VisionMamba structured state space model process of an intelligent express package automatic sorting method based on deep learning is provided for the application;

[0111] Fig. 3 A schematic diagram of the MoE-Mamba hybrid expert network structure of an intelligent express package automatic sorting method based on deep learning is provided for the application. DETAILED DESCRIPTION

[0112] The application will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams that only schematically illustrate the basic structure of the application, and therefore only show the components related to the application.

[0113] REFERENCE Figs. 1-3 An intelligent express package automatic sorting method based on deep learning includes the following steps:

[0114] S1, real-time acquisition of multi-angle high-definition camera equipment for multi-view images of packages to be sorted, and size standardization and multi-scale image enhancement processing to obtain a set of standardized multi-view images;

[0115] S2, encode the standardized multi-view image set in a bidirectional spatial sequence using a Vision Mamba structured state space model, to obtain a wrapped global contour feature map and a local texture feature map, respectively;

[0116] S3, interactively fuse the global contour feature map and the local texture feature map using a cross-scale feature fusion method to generate a fused visual feature map;

[0117] S4, input the fused visual feature map into a preliminary classification network for preliminary classification processing to obtain an initial classification category of the package;

[0118] S5, dynamically select a corresponding expert subnetwork in the MoE-Mamba hybrid expert network according to the initial classification category, and perform deep feature reconstruction and fine classification processing on the fused visual feature map to obtain a final classification category of the package;

[0119] S6, query a pre-set sorting path database according to the final classification category, extract the corresponding automatic sorting path information, and generate a control instruction for driving the sorting equipment to perform automatic guided positioning and accurate sorting of the package.

[0120] The standardized multi-view image set in the application is specifically the original images of top view, side view and front view collected synchronously and in real time by three high-definition camera devices, the image size is adjusted to 512x512 pixels by a bilinear interpolation method, then multi-scale enhancement processing is performed by gamma transformation, and then scale smoothing and feature fusion are performed on the image using a two-dimensional Gaussian filter to obtain standardized input image data. The MoE-Mamba hybrid expert network in the application includes a plurality of pre-trained expert subnetworks, each subnetwork structure includes a dimension compression module, a state space mapping module and a residual enhancement module, and performs dimension reduction, spatial mapping and enhancement processing on the input visual feature map to obtain deep reconstruction features for final fine classification.

[0121] The application realizes efficient extraction and fusion of multi-view feature information of express packages by constructing a Vision Mamba and MoE-Mamba combined visual feature coding and fine classification framework, improves the recognition accuracy and stability of package categories, and at the same time, the automatic guided positioning method based on the final classification result significantly improves the accuracy of sorting path selection and enhances the adaptability of the sorting equipment in complex logistics environment.

[0122] In the embodiment, the S1 specifically includes:

[0123] S11, use a multi-angle high-definition camera device with a resolution of 1920x1080 pixels to collect original images of the top view, side view and front view of the package to be sorted in real time at a frequency of 30 frames per second;

[0124] S12, adopt a bilinear interpolation algorithm to adjust the original image to a standardized image with a uniform size of 512*512 pixels;

[0125] S13, perform gamma transformation enhancement processing on the standardized image:

[0126] O(x,y)=c×I(x,y) γ ;

[0127] Wherein, O(x,y) represents the output image after gamma transformation, I(x,y) represents the input standardized image, c represents the proportion coefficient, the value range is 0.8~1.2, and γ represents the gamma correction parameter, the value range is 0.4~0.7;

[0128] S14, the standardized image after gamma transformation enhancement is respectively subjected to multi-scale smoothing processing by using two-dimensional Gaussian filters with different scales;

[0129] S15, the standardized image after multi-scale smoothing processing is subjected to feature weight distribution by scale, and is fused into a single image by a linear weighted superposition manner pixel by pixel, and then each single image formed by fusing each view angle is combined to construct the standardized multi-view image set.

[0130] In the specific implementation process of the application, the two-dimensional Gaussian filter processing respectively adopts three different Gaussian filter kernels with scale parameters of 0.5, 0.75 and 1.0 to perform pixel-by-pixel convolution operation on the standardized image, so as to generate feature maps with different smoothing scales. The feature weight distribution adopts one-dimensional weight vector mapping based on image pixel position, and the feature maps of each scale are weighted and fused into a single scale image, and then the single scale images of the top view, the side view and the front view are combined to obtain the standardized multi-view image set with uniform size and scale feature enhancement, so as to ensure that the subsequent steps effectively extract image features and complete accurate classification processing.

[0131] The application effectively enhances the detailed features of the image under different scales by multi-scale Gaussian filtering and smoothing processing and feature weight fusion by scale, reduces the noise interference in the image processing process, significantly improves the accuracy and robustness of subsequent feature extraction and classification recognition, and meets the demand of high-precision automatic sorting of express parcels in actual logistics environment.

[0132] The Vision Mamba structured state space model specifically includes:

[0133] The standardized multi-view image set is subjected to multi-scale convolution feature extraction by using a deep convolution operator, so as to obtain preliminary visual features with different scales;

[0134] adopting a structured state space model to perform state space modeling on the preliminary visual feature:

[0135]

[0136] wherein, denotes a state vector at time t at scale s, denotes a preliminary visual feature input vector at time t at scale s, denotes an output state feature vector at time t at scale s, A is a state transition matrix, B is an input control matrix, C is a state output matrix, and D is an input pass-through matrix;

[0137] adopting a bidirectional structured state space model to perform forward and backward state space encoding on the output state feature vector, respectively;

[0138] forward state space encoding:

[0139]

[0140] backward state space encoding:

[0141]

[0142] wherein, denote forward and backward state space encoding vectors at time t at scale s, respectively, A f , A b denote forward and backward state transition matrices, respectively, B f , B b denote forward and backward input control matrices, respectively;

[0143] performing feature fusion processing on the forward and backward state space encoding vectors:

[0144]

[0145] wherein, denotes a visual state feature vector at time t at scale s after fusion by a bidirectional structured state space encoding mechanism, C f , C b denote forward and backward state output matrices, respectively, D ′ denotes a bidirectional fusion pass-through matrix;

[0146] performing scale interaction fusion on visual state feature vectors at different scales according to a preset scale fusion weight to obtain a fused visual feature vector after scale interaction fusion;

[0147] The fused visual feature vector after scale interaction fusion is subjected to nonlinear mapping processing by using a Gaussian error linear unit activation function to obtain a final visual feature output vector.

[0148] In this embodiment, S2 specifically includes:

[0149] S21, each image in the standardized multi-view image set is divided into non-overlapping image sub-regions with a size of 16x16 pixels, and each sub-region is flattened into a one-dimensional initial feature vector with a length of 256;

[0150] S22, according to the Vision Mamba structured state space model of claim 3, bidirectional state space encoding processing is performed on each one-dimensional initial feature vector to obtain corresponding forward state space feature vectors and backward state space feature vectors, respectively;

[0151] S23, based on the complementary difference features of the forward state space feature vectors and the backward state space feature vectors, a bidirectional difference fusion feature vector in the sub-region is calculated, and the specific calculation formula is:

[0152]

[0153] wherein, indicates the bidirectional difference fusion feature vector of the sub-region r at time t, indicate the forward and backward state space feature vectors, respectively, W f , W b indicate the forward and backward fusion weight matrices, respectively, b indicates the fusion bias vector, and σ(·) indicates the Sigmoid activation function;

[0154] S24, the bidirectional difference fusion feature vectors of each sub-region are rearranged according to the spatial position of the original image to construct a corresponding spatial sequence feature mapping matrix;

[0155] S25, the spatial sequence feature mapping matrix is subjected to adaptive image feature attention mechanism to calculate the feature weight coefficients of each sub-region in space, and the spatial sequence feature mapping matrix is weighted and reconstructed element by element according to the feature weight coefficients to obtain a global contour feature map with a resolution of 32x32.

[0156] S26, the spatial sequence feature mapping matrix is subjected to adaptive two-dimensional Fourier frequency domain decomposition algorithm to extract feature spectral components with a frequency range of 0.3f max ~0.6f max , wherein f max indicates the highest frequency value after two-dimensional Fourier transform, and the spectral components are subjected to inverse Fourier transform reconstruction to obtain a local texture feature map with a resolution of 32x32.

[0157] In the embodiment, the multi-scale convolution feature extraction specifically includes depth convolution operators with kernel sizes of 3x3, 5x5 and 7x7 respectively, and zero padding is used in the convolution operation to keep the output image size the same as the input image size; the forward state space encoding and backward state space encoding processing are specifically implemented in parallel by a structured state space model, and then feature fusion processing is completed by channel-by-channel linear weighting fusion; the scale interaction fusion processing is realized by channel-by-channel linear interpolation between different scale state space encoding feature maps and adaptive scale fusion weight matrix for pixel-by-pixel weighted fusion; the mapping coefficient of the Gaussian error linear unit activation function processing is fixed at 0.5 to ensure the stability of the nonlinear mapping processing.

[0158] The application realizes bidirectional state space encoding and scale fusion of visual features through a structured state space model, significantly improves the expression ability of visual features, effectively enhances the feature robustness and generalization performance in the package classification task, and improves the accuracy and stability of automatic sorting of express packages in complex scenes.

[0159] In the embodiment, the S3 specifically includes:

[0160] S31, the global contour feature map and the local texture feature map are respectively up-sampled by a bicubic interpolation method with a scale factor of 2 to obtain a unified scale feature map with a resolution of 64x64;

[0161] S32, the unified scale feature map is respectively smoothed by a two-dimensional Gaussian low-pass filter, and the scale parameter of the filter is in the range of 0.5-1.0;

[0162] S33, the smoothed unified scale feature map is respectively subjected to feature standardization processing, and the feature standardization processing is obtained by subtracting the mean value of the feature map from each element and dividing by the standard deviation of the feature map;

[0163] S34, a spatial element feature difference vector is calculated based on the standardized feature map:

[0164] d(x,y)=F g (x,y)-F l (x,y);

[0165] Wherein, d(x,y) represents the feature difference vector at position (x,y), Fg(x,y) and Fl(x,y) represent the feature values of the global contour standardized feature map and the local texture standardized feature map at position (x,y) respectively;

[0166] S35, the feature difference vector is subjected to adaptive weight mapping processing to generate a difference weight mapping matrix:

[0167]

[0168] wherein M(x, y) represents the difference weight at position (x, y), and λ represents the weight adaptive mapping parameter, and the value range is 1.5-2.5;

[0169] S36, element-wise weighted fusion is performed on the global contour normalized feature map and the local texture normalized feature map according to the difference weight mapping matrix, and a fused visual feature map is obtained:

[0170] F 融合 (x, y) = M(x, y) F g (x, y) + [1-M(x, y)] F l (x, y);

[0171] wherein F fusion (x, y) represents the feature value of the fused visual feature map at position (x, y).

[0172] In the embodiment, when the unified scale feature map is up-sampled by using a bicubic interpolation algorithm, the radius of the interpolation kernel is set to 2, and the boundary mirror filling method is used in the interpolation process to prevent numerical distortion of the feature map boundary; the convolution kernel size of the two-dimensional Gaussian low-pass filter is 7x7 pixels, and the convolution kernel normalization processing is used in the filtering process to ensure the stability of the feature map value after the filtering operation; the spatial position corresponding method is used for pixel-by-pixel alignment calculation in the calculation process of the element-wise feature difference vector, which avoids the feature error caused by pixel position misalignment; the difference weight adaptive mapping parameter λ is set to 2.0 according to the actual application environment, so as to ensure the stability and consistency of the difference weight mapping matrix in the calculation process.

[0173] The application realizes effective fusion of global contour features and local texture features through scale unification, standardization processing and difference weight adaptive fusion strategy, significantly improves the expression accuracy and adaptability of visual features, improves the robustness and distinguishability of express package image features under complex background and lighting conditions, and thus improves the accuracy and practicality of automatic sorting operation.

[0174] The preliminary classification network specifically includes:

[0175] The input fused visual feature map is processed by using a convolution operation with a convolution kernel size of 3x3 to obtain a preliminary coarse scale feature map;

[0176] The input fused visual feature map is processed by using a convolution operation with a convolution kernel size of 1x1 to obtain a preliminary fine scale feature map;

[0177] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector;

[0178] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector;

[0179] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector;

[0180] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector;

[0181] The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector.

[0182] In the embodiment, the S4 specifically comprises:

[0183] S41, a maximum pooling operation with a size of 2x2 and a step of 2 is performed on the fusion visual feature map to obtain a reduced dimension visual feature map with a size of 32x32.

[0184] S42, the reduced dimension visual feature map is input into a preliminary classification network, and the preliminary classification network is sequentially subjected to coarse-scale feature encoding processing, fine-scale feature encoding processing, adaptive scale fusion processing, and channel attention feature enhancement processing to obtain a preliminary classification feature map.

[0185] S43, the preliminary classification feature map is flattened into a one-dimensional feature vector, which is input into a full connection classifier of the preliminary classification network to generate a probability distribution vector of the initial classification class of the package.

[0186] S44, the initial classification class corresponding to the element with the maximum value in the probability distribution vector is determined as the initial classification class of the package.

[0187] In the embodiment, the coarse-scale feature encoding process and the fine-scale feature encoding process of the preliminary classification network both adopt batch normalization operation, the batch data size is set to 64 in the batch normalization process, and an insertion is inserted before the activation function to stabilize the network training process; the adaptive scale fusion processing dynamically adjusts the scale fusion weight vector through the training data set, and the initialization value of the weight vector is uniformly set to 0.5; in the channel attention feature enhancement processing, the number of hidden nodes of the full connection layer is set to 128, and the nonlinear activation function is selected as the ReLU function to ensure the nonlinear mapping of the feature enhancement process is stable and effective; the number of output nodes of the full connection classifier is equal to the number of preset package categories, which is set to 20 in the embodiment.

[0188] The application effectively improves the adaptability and discrimination ability of the classification network to the fused visual feature map through the coarse and fine scale fusion and channel attention mechanism of the preliminary classification network, significantly optimizes the accuracy and stability of the initial classification category prediction of the package, and lays a good foundation for subsequent fine classification processing.

[0189] The MoE-Mamba hybrid expert network specifically includes:

[0190] According to the initial classification category, a MoE-Mamba hybrid expert network composed of a plurality of parallel expert sub-networks and a gating network is constructed, each expert sub-network includes a structured state space module and a residual feature enhancement module;

[0191] The gating network calculates the dynamic selection probability of each expert sub-network based on the global average pooling feature vector of the input fused visual feature map;

[0192] According to the dynamic selection probability of the gating network, the corresponding expert sub-network is selected, and the fused visual feature map is input into the selected expert sub-network;

[0193] The expert sub-network performs dimension reduction feature transformation on the input fused visual feature map to obtain a dimension reduction input feature;

[0194] The expert sub-network uses the structured state space module to perform state space feature mapping on the dimension reduction input feature and outputs a state space feature;

[0195] The expert sub-network uses the residual feature enhancement module to perform residual enhancement processing on the state space feature and outputs an enhanced expert feature;

[0196] According to the dynamic selection probability calculated by the gating network, the enhanced expert features output by all expert sub-networks are weighted and fused to obtain the fusion output feature of the MoE-Mamba hybrid expert network;

[0197] The fusion output feature is input into a full connection classifier for fine classification processing to obtain a fine classification category probability distribution of the package.

[0198] In this embodiment, the S5 specifically includes:

[0199] S51, dynamically selecting a unique corresponding expert sub-network from the MoE-Mamba hybrid expert network according to the initial classification category, and inputting the fusion visual feature map into the selected expert sub-network;

[0200] S52, the selected expert sub-network uses a dimension reduction feature transformation operation to perform feature dimension compression processing on the fusion visual feature map to obtain a dimension reduction feature vector;

[0201] S53, the selected expert sub-network uses a structured state space module to perform state space feature mapping processing on the dimension reduction feature vector to obtain a state space feature vector;

[0202] S54, the selected expert sub-network uses a residual feature enhancement module to perform residual enhancement processing on the state space feature vector to obtain an enhanced expert feature vector;

[0203] S55, the enhanced expert feature vector is subjected to spatial feature reconstruction through a convolution operation with a size of 3x3 and a step of 1 and a global average pooling operation to obtain a deep feature reconstruction vector with a length of 512;

[0204] S56, the deep feature reconstruction vector is input into a first full connection layer containing 256 hidden nodes and a second full connection classification layer containing 20 output nodes, and a probability distribution vector of the final classification category of the package is obtained through a Softmax function;

[0205] S57, determining the final classification category corresponding to the element with the maximum value in the probability distribution vector as the final classification category of the package.

[0206] In this embodiment, the MoE-Mamba hybrid expert network contains no less than 20 expert sub-networks, and each expert sub-network has the same structure but independent parameter training; the dimension reduction feature transformation operation uses a full connection transformation matrix with a dimension of 1024x512 for linear mapping, ensuring that the dimension of the feature vector after compression is half of the original dimension; the state transition matrix and the input control matrix in the structured state space module determine the initial parameters through random initialization, and are iteratively optimized in the training process; the residual feature enhancement module is composed of two serial full connection layers, the first layer contains 1024 hidden nodes, the second layer contains 512 hidden nodes, and is connected through a nonlinear activation function ReLU to ensure the stability of the feature enhancement effect.

[0207] The application selects an expert sub-network and a deep feature reconstruction operation dynamically, performs personalized fine classification on fusion visual feature maps of different initial classification categories, effectively improves the recognition accuracy and generalization ability of the classification network on complex visual features, and significantly enhances the accuracy and reliability of the final classification category prediction of the package.

[0208] In the embodiment, the S6 specifically includes:

[0209] S61, performing a category index matching operation in a preset sorting path database according to the final classification category, to obtain a unique sorting path code corresponding to the final classification category;

[0210] S62, extracting corresponding automatic sorting path information from the sorting path database according to the unique sorting path code, wherein the automatic sorting path information includes a sorting target position coordinate, a path node number, a path order sequence, and a path control parameter;

[0211] S63, performing spatial coordinate conversion processing on the automatic sorting path information, comparing the sorting target position coordinate with an initial position coordinate of the package in real time, and calculating an automatic guiding positioning vector:

[0212] V 导引 =(X 目标 -X 初始 ,Y 目标 -Y 初始 ,Z 目标 -Z 初始 );

[0213] wherein X 目标 , Y 目标 , and Z 目标 are coordinate values of the sorting target position coordinate in a spatial rectangular coordinate system, and X 初始 , Y 初始 , and Z 初始 are coordinate values of the position coordinate of the package in real time in the spatial rectangular coordinate system;

[0214] S64, calculating spatial movement trajectory point array coordinates of the sorting device according to the automatic guiding positioning vector, and performing interpolation smoothing processing on the movement trajectory;

[0215] S65, generating sorting control instructions for the execution unit of the sorting device in real time according to the sorting path control parameter, and performing instruction integrity and safety verification;

[0216] S66, sending the verified sorting control instructions to the execution unit of the sorting device in real time, driving the sorting device to move accurately along the movement trajectory point array coordinates, and realizing automatic guiding positioning of the package;

[0217] S67、When the package moves to the target sorting position, the sorting device is controlled to perform an action to complete the accurate sorting operation.

[0218] In this embodiment, the sorting path database is preconfigured with path information of at least 100 classification categories, each path information including unique code, coordinate position, node number, sequence, and control parameters, and the database adopts a relational database structure; the spatial rectangular coordinate system used in the spatial coordinate conversion operation is a three-dimensional rectangular coordinate system established with the initial installation position of the sorting device as the origin; the interpolation smoothing processing specifically adopts a cubic spline interpolation algorithm for spatial trajectory optimization to ensure the continuity and smoothness of the sorting device movement; and the sorting control instruction integrity and safety check adopts a redundant cyclic check algorithm (CRC) for verification, with a 16-bit check code length to ensure that the instruction data is not tampered with or damaged during transmission.

[0219] The present application effectively improves the stability and safety of the sorting device movement trajectory by combining the explicit automatic guide positioning calculation and precise interpolation smoothing processing with the instruction verification mechanism, significantly improving the reliability and accuracy of the package automatic guide positioning and accurate sorting operation.

[0220] Example 1

[0221] To verify the feasibility of the present application, the present application is applied in the intelligent express sorting center environment of a large logistics company B. Company B is a large logistics enterprise that processes more than 500,000 packages per day, and sorting efficiency and accuracy directly affect the overall operational efficiency and user satisfaction of the company. In recent years, with the rapid development of the e-commerce industry, the traditional express package sorting method of B company has gradually exposed serious shortcomings, manifested as low sorting efficiency, high classification error rate, slow response speed of sorting equipment, etc. Especially during large-scale promotional holidays such as "Double Eleven" and "618", the sorting error rate increases significantly, seriously affecting the timeliness of package delivery and customer satisfaction.

[0222] In view of the above problems, B company deployed the intelligent express package automatic sorting system based on Vision Mamba combined with MoE-Mamba of the present application from September to November 2024. This embodiment selects a core area express sorting center of B company for system deployment, which processes more than 100,000 packages per day, and the sorting equipment includes automatic sorting robots, intelligent conveyor belts, multi-angle high-definition camera equipment, etc.

[0223] In the implementation of the present application, first, the high-definition camera equipment real-time acquisition of multi-view package image is standardized and multi-scale enhanced processing, so that the resolution of each package image is unified to 512x512 pixels, in order to facilitate subsequent unified processing. Subsequently, the system uses the Vision Mamba structured state space model to process the standardized image in a bidirectional spatial sequence, extracting the contour and texture features of the package. Next, the fused visual feature map is processed by spatial dimension reduction sampling, coarse scale and fine scale coding, adaptive scale fusion and channel attention enhancement, generating an initial package classification result. Then, according to the initial classification result, the corresponding MoE-Mamba hybrid expert network is dynamically selected, and the package is further extracted and classified to obtain the accurate final classification category. Finally, the system queries the sorting path database according to the final classification result in real time, calculates the guidance positioning vector and spatial movement trajectory, and generates real-time control instructions to drive the sorting equipment to accurately perform sorting operations.

[0224] After implementing the system, B Company conducted a detailed data statistics and analysis on the sorting efficiency and accuracy of the sorting center in December 2024, obtaining the specific data shown in Table 1 as follows:

[0225] Table 1: Comparison of B Company's sorting center before and after implementing the present application

[0226]

[0227]

[0228] From the above Table 1 data, it can be clearly seen that after deploying the present application, the sorting efficiency and accuracy of the sorting center have been significantly improved. The average daily package processing capacity has increased from 10.2 million pieces before system deployment to 11.5 million pieces; the accuracy rate of package sorting has increased from 92.5% to 99.3%, and the error rate has decreased from 7.5% to 0.7%; the average processing time for a single package has been shortened from 2.6 seconds to 1.2 seconds. During the "Double Eleven" and other e-commerce promotion peak periods, the daily average number of accurately sorted packages has also increased from 11.0 million to 13.9 million.

[0229] In terms of control instruction accuracy, before implementing the present application, the equipment instruction execution accuracy was 90.4%, which improved to 98.9% after deployment, significantly improving the control stability and operation precision of the sorting equipment. At the same time, due to the realization of automatic guidance positioning and smooth trajectory control by the system, the equipment response delay has also been significantly shortened, from 450 milliseconds to 120 milliseconds. In addition, the customer complaint rate due to misclassification has also decreased from 5.6 per thousand to 0.3, greatly improving customer satisfaction and company brand reputation.

[0230] In a typical verification case, on the peak day of November 11, 2024, the system processed 150,000 parcels in total, with an accurate sorting rate of 99.8%. Especially during the peak period of 12:00-13:00, the sorting system accurately processed 18,000 parcels, with a classification accuracy rate of 100%. System monitoring data shows that during this period, the control command transmission and execution error is always less than 1 millisecond, achieving error-free operation of accurate parcel sorting.

[0231] The above examples clearly verify the effectiveness of the technical solutions of the present application. Through the advanced visual processing and expert network mechanism of Vision Mamba combined with MoE-Mamba network, the present application solves the significant defects of traditional sorting technology in terms of low processing efficiency, high sorting error rate and insufficient precision of control commands, and realizes large-scale, high-precision and high-efficiency intelligent express sorting operation, providing strong technical support for B company and promoting the further improvement of the automation and intelligence level of the logistics industry.

[0232] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art within the technical scope disclosed by the present application, according to the technical solutions and inventive concepts of the present application, can make equivalent replacement or change, which should be covered within the protection scope of the present application.

Claims

1. An intelligent express package automatic sorting method based on deep learning, characterized in that, It comprises the following steps: S1, real-time acquisition of multi-angle high-definition camera equipment for multi-view images of packages to be sorted, and size standardization and multi-scale image enhancement processing are performed to obtain a standardized multi-view image set; S2, using a Vision Mamba structured state space model to bidirectional spatial sequence coding on the standardized multi-view image set, respectively obtaining the global contour feature map and the local texture feature map of the package; S3, using a cross-scale feature fusion method to interactively fuse the global contour feature map and the local texture feature map to generate a fused visual feature map; S4, inputting the fused visual feature map into a preliminary classification network for primary classification processing to obtain the initial classification category of the package; S5, dynamically selecting the corresponding expert subnetwork in the MoE-Mamba hybrid expert network according to the initial classification category, and performing deep feature reconstruction and fine classification processing on the fused visual feature map to obtain the final classification category of the package; S6, according to the final classification category, querying the preset sorting path database, extracting the corresponding automatic sorting path information, and generating control instructions for driving the sorting equipment to perform automatic guiding positioning and accurate sorting of the package. 2.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The S1 specifically comprises: S11, using a multi-angle high-definition camera equipment with a resolution of 1920*1080 pixels to real-time collect the original images of the top view, side view and front view of the package to be sorted at a frequency of 30 frames per second; S12, using a bilinear interpolation algorithm to adjust the original image to a standardized image with a uniform size of 512*512 pixels; S13, performing gamma transformation enhancement processing on the standardized image: O(x, y) = c x I(x, y) γ ; Wherein, O(x,y) represents the output image after gamma transformation, I(x,y) represents the input standardized image, c represents the proportion coefficient, the value range is 0.8-1.2, and γ represents the gamma correction parameter, the value range is 0.4-0.7; S14, performing multi-scale smoothing processing on the gamma transformation enhanced standardized image using two-dimensional Gaussian filters of different scales respectively; S15, performing feature weight distribution on the standardized image after multi-scale smoothing processing, and fusing into a single image through pixel-by-pixel linear weighted superposition, and then combining the single images formed by each view to construct the standardized multi-view image set. 3.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The Vision Mamba structured state space model specifically comprises: Using a deep convolution operator to perform multi-scale convolution feature extraction on the input standardized multi-view image set to obtain preliminary visual features of different scales; Using a structured state space model to model the preliminary visual features in a state space: wherein, x(t) represents a state vector at time t at scale s, z(t) represents a preliminary visual feature input vector at time t at scale s, y(t) represents an output state feature vector at time t at scale s, A is a state transition matrix, B is an input control matrix, C is a state output matrix, and D is an input pass-through matrix. Using a bidirectional structured state space model to perform forward and backward state space coding on the output state feature vector respectively; Forward state space coding: Backward state space coding: wherein, respectively denote the forward and backward state space encoding vectors at time t at scale s, A f , A b respectively denote the forward and backward state transition matrices, B f , B b respectively denote the forward and backward input control matrices; Performing feature fusion processing on the forward and backward state space coding vectors: wherein, denotes the visual state feature vector at time t under scale s after fusion by the bidirectional structured state space encoding mechanism, C f , C b denote the forward and backward state output matrices, respectively, D ′ denotes the pass-through matrix for bidirectional fusion; Visual state feature vectors of different scales According to the preset scale fusion weight, scale interaction fusion is performed to obtain a fused visual feature vector after scale interaction fusion. Using a Gaussian error linear unit activation function to perform nonlinear mapping processing on the fused visual feature vector after scale interaction fusion to obtain the final visual feature output vector. 4.The intelligent express package automatic sorting method based on deep learning of claim 1, wherein, The S2 specifically comprises: S21, each image in the standardized multi-view image set is divided into non-overlapping image sub-regions with a size of 16x16 pixels, and each sub-region is flattened into a one-dimensional initial feature vector with a length of 256; S22, according to the Vision Mamba structured state space model of claim 3, bidirectional state space encoding processing is performed on each one-dimensional initial feature vector to obtain corresponding forward state space feature vectors and backward state space feature vectors respectively; S23, based on the complementary difference features of the forward state space feature vectors and the backward state space feature vectors, a bidirectional difference fusion feature vector in the sub-region is calculated, and the specific calculation formula is: wherein, denotes the bidirectional differential fusion feature vector of the sub-region r at time t, denote the forward and backward state space feature vectors, respectively, W f , W b denote the forward and backward fusion weight matrices, respectively, b denotes the fusion bias vector, and σ(·) denotes the Sigmoid activation function; S24, the bidirectional difference fusion feature vectors of each sub-region are rearranged according to the spatial position of the original image to construct a corresponding spatial sequence feature mapping matrix; S25, the spatial sequence feature mapping matrix is calculated using an adaptive image feature attention mechanism to obtain feature weight coefficients of each sub-region in space, and the spatial sequence feature mapping matrix is weighted and reconstructed element by element according to the feature weight coefficients to obtain a global contour feature map with a resolution of 32x32; S26, the spatial sequence feature mapping matrix is decomposed by using an adaptive two-dimensional Fourier frequency domain decomposition algorithm, and a feature spectrum component with a frequency range of 0.3f max ~0.6f max is extracted, wherein f max represents the highest frequency value after two-dimensional Fourier transform, and inverse Fourier transform reconstruction is performed on the spectrum component to obtain a local texture feature map with a resolution of 32x32. 5.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The S3 specifically comprises: S31, the global contour feature map and the local texture feature map are respectively up-sampled by a bicubic interpolation method with a scale factor of 2 to obtain a unified scale feature map with a resolution of 64x64; S32, the unified scale feature map is respectively smoothed using a two-dimensional Gaussian low-pass filter, and the scale parameter of the filter is in the range of 0.5-1.0; S33, the smoothed unified scale feature map is respectively subjected to feature standardization processing, and the feature standardization processing is performed by subtracting the mean value of the feature map from each element and then dividing by the standard deviation of the feature map to obtain a standardized feature map; S34, a spatial element-wise feature difference vector is calculated based on the standardized feature map: d(x,y) = F g (x,y) - F l (x,y); Wherein, d(x,y) represents the feature difference vector at position (x,y), Fg(x,y) and Fl(x,y) represent the feature values of the global contour standardized feature map and the local texture standardized feature map at position (x,y) respectively; S35, the feature difference vector is processed element by element using an adaptive weight mapping to generate a difference weight mapping matrix: Wherein, M(x,y) represents the difference weight at position (x,y), and λ represents the weight adaptive mapping parameter, which is in the range of 1.5-2.5; S36, the global contour standardized feature map and the local texture standardized feature map are weighted and fused element by element according to the difference weight mapping matrix to obtain a fused visual feature map: F 融合 (x,y) = M(x,y) · F g (x,y) + [1 - M(x,y)] · F l (x,y); Wherein, Ffusion(x,y) represents the feature value of the fused visual feature map at position (x,y). 6.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The preliminary classification network specifically comprises: A convolution operation with a convolution kernel size of 3x3 is used to perform coarse-scale feature encoding processing on the input fused visual feature map to obtain a preliminary coarse-scale feature map; A convolution operation with a convolution kernel size of 1x1 is used to perform fine-scale feature encoding processing on the input fused visual feature map to obtain a preliminary fine-scale feature map; The preliminary coarse-scale feature map and the preliminary fine-scale feature map are respectively subjected to channel direction average pooling processing and maximum pooling processing, and the feature vectors after the pooling processing are subjected to full connection layer and Softmax function calculation to obtain an adaptive scale fusion weight vector; The preliminary coarse-scale feature map and the preliminary fine-scale feature map are subjected to channel-by-channel weighted fusion based on the adaptive scale fusion weight vector to obtain a residual fusion feature map; The residual fusion feature map is subjected to channel direction average pooling and maximum pooling processing, and the pooled features are subjected to two-layer full connection mapping processing with a nonlinear activation function to obtain a channel attention weight vector; The residual fusion feature map is subjected to channel-by-channel weighted processing based on the channel attention weight vector to obtain a preliminary classification feature map; After the preliminary classification feature map is flattened into a one-dimensional vector, it is input into a full connection classifier to generate a preliminary classification prediction class probability distribution. 7.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The S4 specifically comprises: S41, performing spatial down-sampling processing on the fusion visual feature map by using a maximum pooling operation with a size of 2x2 and a step of 2 to obtain a dimension-reduced visual feature map with a size of 32x32; S42, inputting the dimension-reduced visual feature map into a preliminary classification network, and sequentially performing coarse-scale feature encoding processing, fine-scale feature encoding processing, adaptive scale fusion processing, and channel attention feature enhancement processing through the preliminary classification network to obtain a preliminary classification feature map; S43, flattening the preliminary classification feature map into a one-dimensional feature vector and inputting it into a full connection classifier of the preliminary classification network to generate a probability distribution vector of an initial classification class of the package; S44, determining the initial classification class corresponding to the element with the maximum value in the probability distribution vector as the initial classification class of the package. 8.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The MoE-Mamba hybrid expert network specifically comprises: According to the initial classification class, an MoE-Mamba hybrid expert network composed of a plurality of parallel expert sub-networks and a gating network is constructed, and each expert sub-network includes a structured state space module and a residual feature enhancement module; The gating network calculates the dynamic selection probability of each expert sub-network based on the global average pooling feature vector of the input fusion visual feature map; According to the dynamic selection probability of the gating network, a corresponding expert sub-network is selected, and the fusion visual feature map is input into the selected expert sub-network; The expert sub-network performs dimension-reduced feature transformation on the input fusion visual feature map to obtain a dimension-reduced input feature; The expert sub-network uses the structured state space module to perform state space feature mapping on the dimension-reduced input feature and outputs a state space feature; The expert sub-network uses the residual feature enhancement module to perform residual enhancement processing on the state space feature and outputs an enhanced expert feature; According to the dynamic selection probability calculated by the gating network, the enhanced expert features output by all expert sub-networks are weighted and fused to obtain a fusion output feature of the MoE-Mamba hybrid expert network; The fusion output feature is input into a full connection classifier for fine classification processing to obtain a fine classification class probability distribution of the package. 9.The intelligent express package automatic sorting method based on deep learning of claim 1, wherein, The S5 specifically comprises: S51, dynamically selecting a unique corresponding expert sub-network from the MoE-Mamba hybrid expert network according to the initial classification category, and inputting the fused visual feature map into the selected expert sub-network; S52, the selected expert sub-network uses dimension reduction feature transformation operation to perform feature dimension compression processing on the fused visual feature map, and obtains a dimension reduction feature vector; S53, the selected expert sub-network uses a structured state space module to perform state space feature mapping processing on the dimension reduction feature vector, and obtains a state space feature vector; S54, the selected expert sub-network uses a residual feature enhancement module to perform residual enhancement processing on the state space feature vector, and obtains an enhanced expert feature vector; S55, the enhanced expert feature vector is subjected to spatial feature reconstruction through a convolution operation with a size of 3x3 and a step of 1 and a global average pooling operation, and a depth feature reconstruction vector with a length of 512 is obtained; S56, input the depth feature reconstruction vector into a first fully connected layer containing 256 hidden nodes and a second fully connected classification layer containing 20 output nodes, and obtain a probability distribution vector wrapping the final classification category through the Softmax function; S57, determine the corresponding final classification category according to the element with the maximum value in the probability distribution vector as the final classification category of the package. 10.The intelligent express package automatic sorting method based on deep learning according to claim 1, characterized in that, The S6 specifically comprises: S61, performing a category index matching operation in the preset sorting path database according to the final classification category, and obtaining a unique sorting path code corresponding to the final classification category; S62, extracting the corresponding automatic sorting path information from the sorting path database according to the unique sorting path code, the automatic sorting path information including sorting target position coordinates, path node number, path order sequence and path control parameters; S63, performing spatial coordinate conversion processing on the automatic sorting path information, comparing the sorting target position coordinates with the initial position coordinates of the package in real time, and calculating an automatic guide positioning vector: V 导引 = (X 目标 - X 初始 , Y 目标 - Y 初始 , Z 目标 - Z 初始 ); wherein X 目标 , Y 目标 , Z 目标 are coordinate values of the sorting target position coordinates in a space rectangular coordinate system, and X 初始 , Y 初始 , Z 初始 are coordinate values of the real-time position coordinates of the package in the space rectangular coordinate system, respectively. S64, calculating the spatial movement trajectory point array coordinates of the sorting equipment according to the automatic guide positioning vector, and performing interpolation smoothing processing on the movement trajectory; S65, generating sorting control instructions in real time according to the sorting path control parameters, and performing instruction integrity and safety verification; S66, sending the verified sorting control instructions to the execution unit of the sorting equipment in real time to drive the sorting equipment to move accurately along the movement trajectory point array coordinates, and realize the automatic guide positioning of the package; S67, when the package moves to the target sorting position, control the sorting equipment to perform actions to complete the accurate sorting work.

Images