A parcel intelligent sorting method and system based on deep learning

Abstract

The application discloses a kind of based on deep learning's parcel intelligent sorting method and system, comprising: S1: using single camera acquisition parcel image and pre-processing, obtain the parcel image after pre-processing;S2: design frequency domain enhanced multi-scale feature extraction branch, and generate illumination correction image;Design topological perception type deformable convolution branch, calculate the wrinkle distribution thermodynamic diagram, then calculate geometric regular image, standardization barcode image;S3: extract standardization barcode feature, design enhanced channel-space dual attention mechanism, extract channel-space joint feature, then extract barcode boundary box and inclination angle, and identify barcode content;S4: according to barcode content and sorting window mapping table, generate swing arm control instruction trigger swing arm link.This application can solve the problem that traditional single visual angle visual scheme in swing arm type sorting machine, due to parcel surface wrinkle, reflection, shadow interference leads to barcode recognition rate insufficient and sorting error.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of parcel sorting, in particular to a parcel intelligent sorting method and system based on deep learning. BACKGROUND

[0002] In the field of logistics sorting, automatic identification and sorting of parcels are key links to improve overall operation efficiency. Traditional parcel barcode identification mainly relies on traditional computer vision technology based on image processing, such as edge detection, template matching, and one-dimensional / two-dimensional barcode scanning algorithm, etc. These methods usually need to be in ideal shooting conditions to achieve high recognition rate. However, in actual application, the parcel is often placed unstable or the surface is uneven, causing the barcode image to deform or be partially occluded, which makes it difficult for traditional algorithms to accurately extract the barcode area and complete decoding. In addition, in high-speed pipeline environment, the parcel moves fast and the image acquisition time window is short, which further aggravates the recognition problem caused by posture tilt and complex background. At the same time, the existing system is sensitive to light changes, and when encountering reflection, shadow or low contrast, the recognition performance decreases significantly, resulting in high miss and misidentification rate, especially in multi-angle and non-standard placement. Therefore, an intelligent identification method that can adapt to complex environment, has strong robustness and high precision is needed to improve the automation level and operation efficiency of the parcel sorting system. SUMMARY

[0003] Therefore, the present application aims to provide a parcel intelligent sorting method and system based on deep learning to solve the problem of insufficient barcode recognition rate and sorting error caused by parcel surface wrinkles, reflection and shadow interference in the traditional single-view visual solution in the swing arm sorting machine.

[0004] A parcel intelligent sorting method based on deep learning, comprising:

[0005] S1: acquiring a parcel image with a single camera and preprocessing to obtain a preprocessed parcel image;

[0006] S2: designing a frequency domain enhanced multi-scale feature extraction branch according to the preprocessed parcel image, extracting an illumination compensation mask, and generating an illumination corrected image; designing a topological perception type deformable convolution branch to predict local wrinkle distribution and obtain a wrinkle distribution heat map; combining the illumination corrected image and the wrinkle distribution heat map to calculate a geometric regularized image and a standardized barcode image;

[0007] The frequency domain enhanced multi-scale feature extraction branch comprises: extracting frequency domain features by two-dimensional fast Fourier transform, extracting real and imaginary tensors by down-sampling and linear layers respectively, generating multi-scale attention features by combining attention mechanisms of three scales, and generating illumination compensation masks by complex tensor splicing and convolution layers; the topological perception type deformable convolution branch comprises: generating a deformable feature map by deformable convolution, generating a topological graph structure by combining graph construction algorithms, and generating a wrinkle distribution heat map by convolution operation and partial derivative;

[0008] S3: According to the standardized barcode image, extract the standardized barcode feature, design an enhanced channel-space dual attention mechanism, extract the channel-space joint feature, and then extract the barcode boundary box and the tilt angle, and identify the barcode content;

[0009] The enhanced channel-space dual attention mechanism comprises: generating a channel weight matrix by a channel attention mechanism, generating a spatial weight matrix by combining global average pooling features and a spatial attention mechanism, and finally generating a channel-space joint feature by weighted fusion of the spatial weight matrix and a hollow convolution;

[0010] S4: According to the barcode content and the sorting window mapping table, generate the swing arm control instruction to trigger the swing arm link, and complete the package sorting.

[0011] Further, the S1 step further comprises:

[0012] S11: According to the single camera fixed on the sorting transmission line, the package image is collected to obtain the original RGB image;

[0013] S12: The original RGB image is converted into a gray image by using a three-channel weighted average algorithm;

[0014] S13: The gray image is median filtered to remove image noise and obtain a denoised package image by using a method of sorting local neighborhood pixel values and taking the middle value;

[0015] S14: The denoised package image is histogram equalized to obtain a preprocessed package image.

[0016] Further, the S2 step further comprises:

[0017] According to the wrinkle distribution heat map, a spatial deformation grid is obtained by a deformation field generation function, a local deformation stability constraint term is generated by combining Jacobian matrix calculation and determinant calculation, and a geometric regular image is generated by bilinear interpolation sampling.

[0018] Further, the S2 step further comprises:

[0019] According to the illumination correction image and the geometric regular image, an edge mask is extracted through a Sobel operator, channel attention mechanism and feature splicing operation are combined to generate channel attention weights, the channel attention weights are weighted, details are enhanced by combining a Laplacian sharpening operator, and a standardized barcode image is generated.

[0020] Further, the S2 step further comprises:

[0021] S21: According to the preprocessed parcel image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract an illumination compensation mask, and the calculation method is:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] wherein, is a frequency domain feature, is a two-dimensional fast Fourier transform, is a preprocessed parcel image, is an i-th scale query vector, i is a scale index, , is an i-th scale linear layer, is down-sampling, is an extraction real part tensor operation, is an i-th scale key vector, is an extraction imaginary part tensor operation, is a multi-scale attention feature, are first, second and third scale query vectors respectively, are first, second and third scale key vectors respectively, is a Softmax function, is a feature dimension, is an illumination compensation mask, is a Sigmoid function, is a convolution layer, is a complex tensor splicing, feature splicing;

[0028] S22: According to the illumination compensation mask and the preprocessed parcel image, an illumination correction image is generated by a Retinex algorithm, and the calculation method is:

[0029]

[0030] wherein, is an illumination corrected image, is a Retinex algorithm;

[0031] S23: According to the pre-processed package image, a topological perception type deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map, and the calculation method is:

[0032]

[0033]

[0034]

[0035] wherein, is a deformable feature map, is a deformable convolution, is a shift parameter, is a topological graph structure, is a graph construction algorithm, is a wrinkle distribution heat map, is a convolution operation, is a partial derivative of the deformable feature map in the horizontal coordinate direction, is a partial derivative of the deformable feature map in the vertical coordinate direction;

[0036] S24: According to the wrinkle distribution heat map and the illumination corrected image, a geometric regular image is obtained through a deformation field generation function, a Jacobian matrix and interpolation calculation, and the calculation method is:

[0037]

[0038]

[0039]

[0040] wherein, is a spatial deformation grid, is a deformation field generation function, is a Jacobian matrix, is a Jacobian matrix calculation, is a geometric regular image, is a bilinear interpolation sampling, is a Lagrange multiplier, is a determinant calculation;

[0041] S25: According to the illumination corrected image and the geometric regular image, a standardized barcode image is generated by fusion, and the calculation method is:

[0042]

[0043]

[0044]

[0045] wherein, is an edge mask, is a Sobel operator, is a channel attention weight, is a channel attention mechanism, is a normalized barcode image, is a Hadamard product, is a Laplacian sharpening operator.

[0046] It needs to be further explained that the frequency domain enhanced multi-scale feature extraction branch and the topological perception deformable convolution branch designed in the S2 step of the present application realize the collaborative correction of uneven illumination and surface deformation of the package image. Specifically:

[0047] The core of the frequency domain enhanced multi-scale feature extraction branch is to realize the accurate modeling of the illumination distribution through the collaborative design of frequency domain decomposition and multi-scale attention mechanism. After the image is mapped to the frequency domain space through two-dimensional fast Fourier transform, the real part tensor and the imaginary part tensor are extracted as the query vector (Q) and the key vector (K) of the attention mechanism. This design directly utilizes the physical characteristic differences of the real part and the imaginary part in the frequency domain: the real part reflects the global trend of the image brightness distribution (such as the brightness gradient of the reflective area), and the imaginary part represents the local change of the texture edge (such as the high-frequency details of the shadow boundary). Different scales of attention mechanism can capture the macroscopic law and local anomaly of the illumination distribution, respectively. After the fusion of the attention features of the three scales, the consistency of the global illumination trend is retained, and the compensation ability of the local illumination abnormal area is strengthened. Compared with the limitation of the traditional Retinex algorithm which generates an illumination mask depending on the local statistical characteristics, the present application realizes the dual improvement of the frequency domain scale adaptability and the spatial distribution robustness of the illumination compensation mask through the frequency domain-space dual domain interactive modeling under the dynamic illumination condition, and the compensation accuracy is significantly improved.

[0048] Topology-aware deformable convolution branches primarily address non-rigid deformation problems involving wrinkles on a surface. This branch generates deformable feature maps through deformable convolution and then uses partial derivatives to calculate the gradient field (i.e., the horizontal and vertical gradient components) of the feature maps. The direction and intensity of the gradient field directly reflect the orientation and severity of the wrinkles (e.g., vertical wrinkles dominated by horizontal gradients or horizontal wrinkles dominated by vertical gradients). By combining the gradient field with graph construction algorithms, the generated topological graph structure not only enhances the feature responses of local wrinkled regions but also models the global spatial distribution of wrinkles through the connectivity between graph nodes. Inter-dependencies (e.g., continuity constraints between adjacent folds); Compared to the limitations of traditional STN affine transformation, which can only handle rigid deformations such as overall rotation and scaling, this scheme generates a non-rigid deformation field driven by a gradient field. Under the condition that there are local folds on the package surface, it can achieve pixel-level deformation correction and avoid the problem of barcode texture breakage caused by local overstretching. This feature is particularly significant when dealing with differences in package materials (such as the mixed sorting of flexible packaging and rigid cardboard boxes). The corrected geometrically regular image retains the key features of the barcode while effectively suppressing artifact interference in the folded area.

[0049] In the process of generating geometrically regular images, this invention introduces a local deformation stability constraint term. This constraint term quantifies the rate of change of the area in a local region, thereby constraining the differentiability and local smoothness of the deformation field. Compared with the deformation field generation mechanism that relies solely on fixed parameters in traditional methods, the constraint term generated by the present invention through determinant calculation can dynamically suppress local overstretching or folding caused by uneven distribution of wrapping wrinkles. The corrected geometric structure integrity is significantly improved, providing a high-fidelity geometric basis for the subsequent generation of standardized barcode images.

[0050] The illumination-corrected image and geometrically regularized image obtained by combining the outputs of the two branches are further enhanced by edge perception and attention fusion, and then the details are enhanced by the Laplacian sharpening operator to form a standardized barcode image. Compared with the prior art, the present invention significantly improves the robustness to dynamic illumination interference and irregular deformation in high-speed pipeline scenes. Especially in the case of complex wrinkles on the package surface, reflective labels, or tilted placement, its standardized image generation efficiency and feature integrity are better than single-scale feature extraction or rigid transformation correction methods.

[0051] Furthermore, step S3 also includes:

[0052] S31: Based on the standardized barcode image, extract standardized barcode features using the YOLOv7 network and path aggregation network. The calculation method is as follows:

[0053]

[0054] in, to standardize the barcode features, to a path aggregation network, to a YOLOv7 network;

[0055] S32: According to the standardized barcode features, design an enhanced channel-space dual attention mechanism to enhance the key region features, obtain the channel-space joint features, and the calculation method is:

[0056]

[0057]

[0058]

[0059]

[0060]

[0061] wherein, is a channel weight matrix, is a channel feature, is a global average pooling feature, is a global average pooling, is a spatial weight matrix, is a spatial attention mechanism, is an element-wise addition, is a channel-space joint feature, is a dilated convolution;

[0062] S33: According to the channel-space joint features, extract the barcode bounding box and the tilt angle, and the calculation method is:

[0063]

[0064] wherein, is a barcode bounding box, is a tilt angle, is a bounding box decoding function based on a preset anchor box;

[0065] S34: According to the barcode bounding box and the tilt angle, generate an angle normalized barcode image through a spatial transformation network, and identify the barcode content, and the calculation method is:

[0066]

[0067]

[0068]

[0069] wherein, affine transformation matrix, spatial transformation network, angle normalization barcode image, barcode content, connectional temporal classification decoder, convolutional recurrent neural network.

[0070] It needs to be further explained that in the S3 step of the present application, the core of the enhanced channel-space dual attention mechanism is to realize the dual-domain constraint of feature enhancement through the cooperative modeling of the channel weight matrix and the spatial weight matrix.

[0071] Specifically, after the channel attention mechanism generates the channel weight matrix, the global average pooling feature is further taken as an auxiliary input, and an element-by-element addition operation is performed with the output of the spatial attention mechanism, so that the spatial weight matrix not only depends on the spatial distribution of the local feature map (such as the gradient direction of the barcode edge), but also fuses the global texture statistical characteristics represented by the global average pooling feature; for example, when there is complex background interference (such as printed text or patterns) on the surface of the package, the channel weight matrix will preferentially enhance the channel features related to the barcode texture, and the spatial weight matrix will guide the attention to the overall outline of the barcode area, avoiding false response in the local area. Compared with the limitation of the traditional channel-space attention mechanism that generates spatial weights based only on local features, the present application not only preserves local details but also enhances the global consistency of the barcode area, especially for the working condition where there is a large area of interference label on the surface of the package, the anti-interference ability is significantly improved; in addition, the present application expands the receptive field through the hollow convolution, so that the feature fusion process can cover a larger range of context information, further suppressing the interference of false artifacts similar to the barcode in the complex background.

[0072] For the problem of barcode rotation, the present application determines the spatial position of the barcode area through the bounding box coordinates, and calculates the affine transformation matrix combined with the inclination angle, so that the barcode characters in the corrected image are always in the horizontal direction; the parameter learning process of the affine transformation network is jointly trained with the subsequent barcode content recognition module, so that the angle correction process can directly respond to the needs of the recognition module; compared with the traditional method which only relies on bounding box regression for angle estimation, the present application can directly model the arrangement rule of the barcode characters through the cooperative design of the spatial transformation network and the connectional temporal classification decoder; for example, when the barcode on the surface of the package is inclined at an arbitrary angle, the affine transformation matrix generated by the affine transformation network can project the barcode area to a unified reference coordinate system, so that the recognition module always decodes based on the forward arrangement of the character sequence, thereby breaking the dependence of the template matching method on fixed angles, which is particularly crucial when dealing with angle changes caused by package rotation in high-speed sorting scenarios.

[0073] Further, the S4 step further comprises:

[0074] S41: According to the barcode content and the sorting window mapping table, the address information in the barcode content is matched with the sorting window mapping table through a rule matching algorithm to obtain a target delivery window identifier;

[0075] S42: According to the target delivery window identifier, the swing timing parameter and the motor driving strength parameter bound to the target delivery window are called through a table lookup matching algorithm to obtain a swing arm control instruction containing pulse width and driving time length;

[0076] S43: According to the swing arm control instruction, in combination with the package to position signal detected by the photoelectric sensor, the swing arm rod is triggered to complete package sorting.

[0077] The application also discloses a package intelligent sorting system based on deep learning, comprising:

[0078] An image acquisition module: a single camera is used to acquire a package image and perform preprocessing to obtain a preprocessed package image;

[0079] A light deformation joint correction module: according to the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed, a light compensation mask is extracted, and a light correction image is generated; a topological perception type deformable convolution branch is designed to predict local wrinkle distribution and obtain a wrinkle distribution heat map; in combination with the light correction image and the wrinkle distribution heat map, a geometric regular image and a standardized barcode image are calculated;

[0080] A barcode recognition module: according to the standardized barcode image, a standardized barcode feature is extracted, an enhanced channel-space dual attention mechanism is designed, a channel-space joint feature is extracted, a barcode bounding box and an inclination angle are further extracted, and barcode content is recognized;

[0081] A swing arm control module: according to the barcode content and the sorting window mapping table, a swing arm control instruction is generated to trigger the swing arm rod to complete package sorting.

[0082] Compared with the prior art, the application has the following beneficial effects:

[0083] (1) The application is aimed at the problems of insufficient barcode recognition rate and sorting error in the swing arm type sorting machine caused by package surface wrinkles, reflection and shadow interference in the traditional single view visual scheme, the quality of the standardized image is significantly improved through light and deformation joint correction; in combination with the channel-space dual attention mechanism and the affine transformation network, high-precision barcode positioning and rotation correction in a complex background are realized; finally, swing arm control signals are generated according to the recognition results to complete accurate sorting actions, effectively improving the barcode recognition rate and system robustness.

[0084] (2) The application designs a frequency domain enhanced multi-scale feature extraction branch and a topological perception type deformable convolution branch, realizes the joint correction of uneven illumination and surface deformation, the frequency domain enhanced multi-scale feature extraction branch extracts image frequency domain features by using two-dimensional fast Fourier transform, and respectively takes the real part and the imaginary part as the query vector and the key vector of the attention mechanism, then outputs the illumination compensation mask through the fusion of three scale attention mechanisms, more accurately models the global trend and local anomaly of the illumination distribution, and is significantly better than the traditional Retinex algorithm; the topological perception type deformable convolution branch generates a gradient field through deformable convolution and partial derivative calculation, generates a wrinkle distribution heat map combined with graph construction algorithm, and realizes pixel-level non-rigid correction through the deformation field generation function and the local stability constraint term, breaks through the limitation that the traditional STN affine transformation can only process rigid deformation, and is especially suitable for flexible packaging and hard carton mixed sorting scenes, and significantly improves the quality of geometric regular images and the clarity of standardized barcode images.

[0085] (3) The application improves the stability and adaptability of barcode recognition through the cooperative design of the enhanced channel-space dual attention mechanism and the affine transformation network; the enhanced channel-space dual attention mechanism fuses the channel weight matrix and the space weight matrix, while retaining local details, it strengthens the global consistency of the barcode area, so that the system can still focus on the target barcode in a complex background, avoiding false detection of interference labels or patterns; for the barcode rotation problem, the affine transformation network estimates the boundary box and the inclination angle jointly to generate an affine transformation matrix, realizing the output of the angle normalized image; this process is jointly trained with the recognition module, so that the angle correction directly responds to the recognition demand, and compared with the traditional method, it is more suitable for barcode recognition at any angle, and has stronger adaptability and generalization ability in high-speed sorting scenes. BRIEF DESCRIPTION OF DRAWINGS

[0086] Figure 1 A flowchart of a parcel intelligent sorting method based on deep learning provided by the application is shown in the figure;

[0087] Figure 2 An algorithm flowchart of the frequency domain enhanced multi-scale feature extraction branch provided by the application is shown in the figure;

[0088] Figure 3 An algorithm flowchart of the topological perception type deformable convolution branch provided by the application is shown in the figure;

[0089] Figure 4 A comparison example graph of a certain parcel with a wrinkled barcode before and after correction provided by the application is shown in the figure, Figure 4 (a) is a sample graph of a wrinkled barcode before correction; Figure 4 (b) is a sample graph of a wrinkled barcode after correction. DETAILED DESCRIPTION

[0090] The application will be further described below with reference to the drawings, but the application is not limited in any way by the following description, and any transformation or replacement based on the teaching of the application shall fall within the protection scope of the application.

[0091] Embodiment 1: a deep learning-based parcel intelligent sorting method, as shown in Figure 1 , comprising the following steps:

[0092] S1: acquiring a parcel image by using a single camera and performing preprocessing to obtain a preprocessed parcel image, comprising:

[0093] S11: acquiring a parcel image by using a single camera fixed on a sorting transmission line to obtain an original RGB image;

[0094] S12: converting the original RGB image into a gray-scale image by using a three-channel weighted average algorithm;

[0095] S13: performing median filter processing on the gray-scale image by using a method of sorting and taking the middle value of local neighborhood pixel values to remove image noise and obtain a denoised parcel image;

[0096] S14: performing histogram equalization processing on the denoised parcel image to obtain a preprocessed parcel image.

[0097] S2: designing a frequency domain enhanced multi-scale feature extraction branch according to the preprocessed parcel image, extracting an illumination compensation mask, and generating an illumination correction image; designing a topological perception type deformable convolution branch to predict local wrinkle distribution and obtain a wrinkle distribution heat map; combining the illumination correction image and the wrinkle distribution heat map to calculate a geometric regularized image and a standardized barcode image, comprising:

[0098] S21: designing a frequency domain enhanced multi-scale feature extraction branch according to the preprocessed parcel image, extracting an illumination compensation mask, as shown in Figure 2 , the calculation method is:

[0099]

[0100]

[0101]

[0102]

[0103]

[0104] wherein, is a frequency domain feature, is a two-dimensional fast Fourier transform, This is the preprocessed image of the package. Let i be the query vector for the i-th scale, where i is the scale index. , For the i-th scale linear layer, To reduce sampling, For operations to extract the real part of the tensor, Let be the key vector at the i-th scale. For operations to extract the imaginary part of the tensor, For multi-scale attention features, These are query vectors representing the first, second, and third scales, respectively. These are the key vectors at the first, second, and third scales, respectively. For the Softmax function, For feature dimension, For illumination compensation mask, For the Sigmoid function, It is a convolutional layer. For complex tensor splicing, Feature splicing;

[0105] S22: Based on the illumination compensation mask and the preprocessed wrapped image, an illumination-corrected image is generated using the Retinex algorithm. The calculation method is as follows:

[0106]

[0107] in, For illumination-corrected images, The Retinex algorithm;

[0108] S23: Based on the preprocessed package image, design a topology-aware deformable convolution branch to predict the local wrinkle distribution and obtain a wrinkle distribution heatmap, such as... Figure 3 As shown, the calculation method is as follows:

[0109]

[0110]

[0111]

[0112] in, This is a deformable feature map. For deformable convolution, For offset parameters, It is a topological graph structure. Graph construction algorithms This is a heat map showing the distribution of folds. For convolution operations, a partial derivative of the deformable feature map in the horizontal direction, a partial derivative of the deformable feature map in the vertical direction;

[0113] S24: According to the wrinkle distribution heat map and the light correction image, the geometric regular image is obtained through the deformation field generation function, the Jacobian matrix and the interpolation calculation, and the calculation method is:

[0114]

[0115]

[0116]

[0117] wherein, is a spatial deformation grid, is a deformation field generation function, is a Jacobian matrix, is a Jacobian matrix calculation, is a geometric regular image, is a bilinear interpolation sampling, is a Lagrange multiplier, is a determinant calculation;

[0118] S25: According to the light correction image and the geometric regular image, the standardized barcode image is fused and generated, and the calculation method is:

[0119]

[0120]

[0121]

[0122] wherein, is an edge mask, is a Sobel operator, is a channel attention weight, is a channel attention mechanism, is a standardized barcode image, is a Hadamard product, is a Laplacian sharpening operator.

[0123] For example, Figure 4 is a contrastive image of a barcode image with wrinkles before and after correction in the training sample of the present application; as Figure 4As shown in (a), before correction, obvious distortion and local occlusion appeared in the lower right corner area, which not only affected the geometric integrity of the barcode but also may cause traditional recognition algorithms to fail to accurately extract barcode information, thereby reducing sorting efficiency and accuracy. This invention utilizes a frequency-domain enhanced multi-scale feature extraction branch to generate an illumination compensation mask, combines a topology-aware deformable convolution branch to predict the wrinkle distribution heatmap, locates the wrinkle region, and achieves pixel-level non-rigid correction through a deformation field generation function; Figure 4 As shown in (b), the corrected barcode clearly displays its complete structure, providing high-quality input for subsequent recognition.

[0124] S3: Based on the standardized barcode image, extract standardized barcode features, design an enhanced channel-spatial dual attention mechanism to extract joint channel-spatial features, then extract the barcode bounding box and tilt angle, and identify the barcode content, including:

[0125] S31: Based on the standardized barcode image, extract standardized barcode features using the YOLOv7 network and path aggregation network. The calculation method is as follows:

[0126]

[0127] in, To standardize barcode features, For path aggregation network, For YOLOv7 networks;

[0128] S32: Based on standardized barcode features, an enhanced channel-spatial dual attention mechanism is designed to enhance key region features and obtain joint channel-spatial features. The calculation method is as follows:

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] in, This is the channel weight matrix. As a channel feature, This is a global average pooling feature. For global average pooling, This is the spatial weight matrix. For spatial attention mechanisms, To add element by element, It is a channel-space joint feature. For dilated convolution;

[0135] S33: Based on the channel-space joint features, extract the barcode bounding box and tilt angle, calculated as follows:

[0136]

[0137] in, For barcode bounding boxes, The tilt angle, This is a bounding box decoding function based on a preset anchor box;

[0138] S34: Based on the barcode bounding box and tilt angle, an angle-normalized barcode image is generated through a spatial transformation network, and the barcode content is identified. The calculation method is as follows:

[0139]

[0140]

[0141]

[0142] in, Let be the affine transformation matrix. For spatial transformation networks, For angle-normalized barcode images, For barcode content, To connect to the temporal classification decoder, It is a convolutional recurrent neural network.

[0143] Specifically, for extreme scenarios with tilt angles greater than 45°, this invention also provides an enhanced angle normalization calculation method to replace step S34. The calculation method is as follows:

[0144]

[0145] in, For image rotation operations, It is a symbolic function.

[0146] S4: Based on the barcode content and the sorting window mapping table, generate swing arm control commands to trigger the swing arm levers, including:

[0147] S41: Based on the barcode content and the sorting window mapping table, the address information in the barcode content is matched with the key value of the sorting window mapping table using a rule matching algorithm to obtain the target delivery window identifier;

[0148] S42: Based on the target delivery window identifier, retrieve the swing arm swing timing parameters and motor drive strength parameters bound to the target delivery window through a lookup table matching algorithm to obtain the swing arm control command containing pulse width and drive duration;

[0149] S43: Based on the control command of the swing arm and the package arrival signal detected by the photoelectric sensor, the swing arm is triggered to complete the package sorting.

[0150] Example 2: This invention also discloses a deep learning-based intelligent parcel sorting system, comprising:

[0151] Image acquisition module: Uses a single camera to acquire images of the package and performs preprocessing to obtain preprocessed images of the package;

[0152] The illumination deformation joint correction module: Based on the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image; a topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heatmap; the illumination-corrected image and the wrinkle distribution heatmap are combined to calculate the geometrically regularized image and the standardized barcode image.

[0153] Barcode recognition module: Based on standardized barcode images, it extracts standardized barcode features, designs an enhanced channel-space dual attention mechanism to extract channel-space joint features, extracts barcode bounding boxes and tilt angles, and recognizes barcode content;

[0154] Swing arm control module: Based on the barcode content and the sorting window mapping table, it generates swing arm control commands to trigger the swing arm levers and complete the package sorting.

[0155] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0157] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A deep learning-based intelligent parcel sorting method, characterized in that, Includes the following steps: S1: Use a single camera to capture images of the package and perform preprocessing to obtain preprocessed images of the package; S2: Based on the preprocessed package image, design a frequency domain enhanced multi-scale feature extraction branch to extract the illumination compensation mask and generate an illumination-corrected image; design a topology-aware deformable convolution branch to predict the local wrinkle distribution and obtain a wrinkle distribution heatmap. By combining illumination-corrected images with wrinkle distribution heatmaps, geometrically regularized images and standardized barcode images are calculated. The frequency domain enhanced multi-scale feature extraction branch includes: extracting frequency domain features through two-dimensional fast Fourier transform, extracting real and imaginary tensors using downsampling and linear layers respectively, generating multi-scale attention features by combining three-scale attention mechanisms, and generating illumination compensation masks through complex tensor concatenation and convolutional layers; the topology-aware deformable convolution branch includes: generating deformable feature maps through deformable convolution, generating topological graph structures by combining graph construction algorithms, and generating fold distribution heatmaps through convolution operations and partial derivatives; S3: Based on the standardized barcode image, extract the standardized barcode features, design an enhanced channel-space dual attention mechanism, extract the channel-space joint features, extract the barcode bounding box and tilt angle, and identify the barcode content; The enhanced channel-spatial dual attention mechanism includes: generating a channel weight matrix through a channel attention mechanism, generating a spatial weight matrix by combining global average pooling features with a spatial attention mechanism, and finally generating a channel-spatial joint feature through weighted fusion of the spatial weight matrix and dilated convolution. S4: Based on the barcode content and the sorting window mapping table, generate swing arm control instructions to trigger the swing arm levers and complete the package sorting.

2. The intelligent parcel sorting method based on deep learning according to claim 1, characterized in that, Step S1 includes: S11: Obtain the original RGB image by capturing package images using a single camera fixed on the sorting conveyor line; S12: Use a three-channel weighted average algorithm to convert the original RGB image into a grayscale image; S13: The grayscale image is processed by median filtering using the method of sorting the local neighborhood pixel values ​​and taking the median value to remove image noise and obtain the noise-reduced wrapper image. S14: Perform histogram equalization on the denoised package image to obtain the preprocessed package image.

3. The intelligent parcel sorting method based on deep learning according to claim 1, characterized in that, Step S2 includes: Based on the heat map of the fold distribution, a spatial deformation mesh is obtained through the deformation field generation function. By combining the calculation of the Jacobian matrix and the determinant, a local deformation stability constraint term is generated, and a geometrically regular image is generated through bilinear interpolation sampling.

4. The intelligent parcel sorting method based on deep learning according to claim 1, characterized in that, Step S2 includes: Based on the illumination-corrected image and the geometrically regularized image, edge masks are extracted using the Sobel operator. Channel attention weights are generated by combining the channel attention mechanism and feature stitching operation. The channel attention weights are then used for weighting, and the Laplacian sharpening operator is used to enhance details, thereby generating a standardized barcode image.

5. The intelligent parcel sorting method based on deep learning according to claim 2, characterized in that, Step S2 includes: S21: Based on the preprocessed package image, design a frequency domain enhanced multi-scale feature extraction branch to extract the illumination compensation mask. The calculation method is as follows: in, For frequency domain characteristics, For two-dimensional fast Fourier transform, This is the preprocessed image of the package. Let i be the query vector for the i-th scale, where i is the scale index. , For the i-th scale linear layer, To reduce sampling, For operations to extract the real part of the tensor, Let be the key vector at the i-th scale. For operations to extract the imaginary part of the tensor, For multi-scale attention features, These are query vectors representing the first, second, and third scales, respectively. These are the key vectors at the first, second, and third scales, respectively. For the Softmax function, For feature dimension, For illumination compensation mask, For the Sigmoid function, It is a convolutional layer. For complex tensor splicing, Feature splicing; S22: Based on the illumination compensation mask and the preprocessed wrapped image, an illumination-corrected image is generated using the Retinex algorithm. The calculation method is as follows: in, For illumination-corrected images, The Retinex algorithm; S23: Based on the preprocessed package image, design a topology-aware deformable convolution branch to predict the local wrinkle distribution and obtain a wrinkle distribution heatmap. The calculation method is as follows: in, This is a deformable feature map. For deformable convolution, For offset parameters, It is a topological graph structure. Graph construction algorithms This is a heat map showing the distribution of folds. For convolution operations, Let be the partial derivative of the deformable feature map along the horizontal axis. is the partial derivative of the deformable feature map along the vertical axis; S24: Based on the heat map of wrinkle distribution and the illumination-corrected image, a geometrically regularized image is obtained through deformation field generation function, Jacobian matrix, and interpolation calculation. The calculation method is as follows: in, For spatial deformation mesh, For the deformation field generating function, For Jacobian matrices, Calculate the Jacobian matrix. For geometrically regular images, For bilinear interpolation sampling, For Lagrange multipliers, For determinant calculation; S25: Generate a standardized barcode image by fusing the illumination-corrected image and the geometrically regularized image. The calculation method is as follows: in, For edge masking, For Sobel operators, For channel attention weights, For channel attention mechanism, To standardize barcode images, For Hadama accumulation, This is the Laplacian sharpening operator.

6. The intelligent parcel sorting method based on deep learning according to claim 5, characterized in that, Step S3 includes: S31: Based on the standardized barcode image, extract standardized barcode features using the YOLOv7 network and path aggregation network. The calculation method is as follows: in, To standardize barcode features, For path aggregation network, For YOLOv7 networks; S32: Based on standardized barcode features, an enhanced channel-space dual attention mechanism is designed to enhance key region features and obtain joint channel-space features. The calculation method is as follows: in, This is the channel weight matrix. As a channel feature, This is a global average pooling feature. For global average pooling, This is the spatial weight matrix. For spatial attention mechanisms, To add element by element, It is a channel-space joint feature. For dilated convolution; S33: Based on the channel-space joint features, extract the barcode bounding box and tilt angle, calculated as follows: in, For barcode bounding boxes, The tilt angle, This is a bounding box decoding function based on a preset anchor box; S34: Based on the barcode bounding box and tilt angle, an angle-normalized barcode image is generated through a spatial transformation network, and the barcode content is identified. The calculation method is as follows: in, Let be the affine transformation matrix. For spatial transformation networks, For angle-normalized barcode images, For barcode content, To connect to the temporal classification decoder, It is a convolutional recurrent neural network.

7. The intelligent parcel sorting method based on deep learning according to claim 6, characterized in that, Step S4 includes: S41: Based on the barcode content and the sorting window mapping table, the address information in the barcode content is matched with the key value of the sorting window mapping table using a rule matching algorithm to obtain the target delivery window identifier; S42: Based on the target delivery window identifier, retrieve the swing arm swing timing parameters and motor drive strength parameters bound to the target delivery window through a lookup table matching algorithm to obtain the swing arm control command containing pulse width and drive duration; S43: Based on the control command of the swing arm and the package arrival signal detected by the photoelectric sensor, the swing arm is triggered to complete the package sorting.

8. A deep learning-based intelligent parcel sorting system, characterized in that, include: Image acquisition module: Uses a single camera to acquire images of the package and performs preprocessing to obtain preprocessed images of the package; The illumination deformation joint correction module: Based on the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image; a topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heatmap; the illumination-corrected image and the wrinkle distribution heatmap are combined to calculate the geometrically regularized image and the standardized barcode image. Barcode recognition module: Based on standardized barcode images, it extracts standardized barcode features, designs an enhanced channel-space dual attention mechanism to extract channel-space joint features, extracts barcode bounding boxes and tilt angles, and recognizes barcode content; The swing arm control module generates swing arm control commands based on the barcode content and the sorting window mapping table to trigger the swing arm levers and complete the package sorting. To achieve a deep learning-based intelligent parcel sorting method as described in any one of claims 1-7.

Images