Intelligent parcel sorting method and system based on deep learning

Abstract

The invention discloses an intelligent parcel sorting method and system based on deep learning, and the method comprises the steps: S1, collecting a parcel image through a single camera, and carrying out the preprocessing of the parcel image, and obtaining a preprocessed parcel image; s2, designing a frequency domain enhanced multi-scale feature extraction branch, and generating an illumination correction image; designing a topology perception type deformable convolution branch, calculating a fold distribution thermodynamic diagram, and then calculating a geometric regular image and a standardized bar code image; s3, standardized barcode features are extracted, an enhanced channel-space double attention mechanism is designed, channel-space joint features are extracted, a barcode bounding box and an inclination angle are extracted, and barcode content is identified; and S4, according to the bar code content and the sorting window mapping table, generating a swing arm control instruction to trigger a swing arm rod piece. According to the invention, the problems of insufficient bar code recognition rate and sorting errors caused by surface wrinkles, light reflection and shadow interference of parcels in a swing arm type sorting machine in a traditional single-visual-angle vision scheme can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of parcel sorting, and in particular to a deep learning-based intelligent parcel sorting method and system. Background Art

[0002] In the logistics and sorting sector, automatic parcel identification and sorting are critical to improving overall operational efficiency. Traditional parcel barcode recognition relies primarily on traditional computer vision techniques based on image processing, such as edge detection, template matching, and one-dimensional / two-dimensional barcode scanning algorithms. These methods typically require ideal imaging conditions to achieve high recognition rates. However, in real-world applications, parcels often experience deformation or partial occlusion of barcode images due to unstable placement or uneven surfaces, making it difficult for traditional algorithms to accurately extract and decode the barcode area. Furthermore, in high-speed assembly line environments, the rapid movement of parcels and the short image acquisition window further exacerbate recognition challenges caused by tilted postures and complex backgrounds. Furthermore, existing systems are sensitive to lighting variations. Recognition performance significantly degrades in the presence of reflections, shadows, or low contrast, resulting in high rates of missed and false recognitions. This is particularly true when parcels are positioned at multiple angles or in non-standard situations. Therefore, there is an urgent need for intelligent recognition methods that can adapt to complex environments, exhibit strong robustness, and provide high accuracy to improve the automation and operational efficiency of parcel sorting systems. Summary of the Invention

[0003] In view of this, the present invention aims to provide a deep learning-based intelligent parcel sorting method and system to solve the problems of insufficient barcode recognition rate and sorting errors caused by wrinkles, reflections, and shadows on the package surface in traditional single-view vision solutions in swing-arm sorting machines.

[0004] A deep learning-based intelligent parcel sorting method, comprising: S1: Use a single camera to capture and preprocess the package image to obtain a preprocessed package image; S2: Based on the preprocessed package image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map. The illumination-corrected image and the wrinkle distribution heat map are combined to calculate the geometrically regularized image and the standardized barcode image. The frequency-domain enhanced multi-scale feature extraction branch includes: extracting frequency-domain features through two-dimensional fast Fourier transform, using downsampling and linear layers to extract real and imaginary tensors respectively, combining three-scale attention mechanisms to generate multi-scale attention features, and generating illumination compensation masks through complex tensor splicing and convolution layers; the topology-aware deformable convolution branch includes: generating a deformable feature map through deformable convolution, generating a topological map structure through a graph construction algorithm, and generating a wrinkle distribution heat map through convolution operations and partial derivatives; S3: Based on the standardized barcode image, extract the standardized barcode features, design an enhanced channel-space dual attention mechanism, extract the channel-space joint features, extract the barcode bounding box and tilt angle, and recognize the barcode content; The enhanced channel-space dual attention mechanism includes: generating a channel weight matrix through the channel attention mechanism, combining the global average pooling feature with the spatial attention mechanism to generate a spatial weight matrix, and finally generating a channel-space joint feature through weighted fusion and void convolution of the spatial weight matrix; S4: Generate a swing arm control instruction based on the barcode content and the sorting window mapping table to trigger the swing arm lever to complete the package sorting.

[0005] Furthermore, the step S1 further includes: S11: A single camera fixed on the sorting transmission line captures the package image to obtain an original RGB image; S12: Convert the original RGB image into a grayscale image using a three-channel weighted average algorithm; S13: performing median filtering on the grayscale image by sorting the local neighborhood pixel values ​​and taking the median value to remove image noise, thereby obtaining a denoised package image; S14: performing histogram equalization processing on the denoised package image to obtain a preprocessed package image.

[0006] Furthermore, the step S2 further includes: According to the wrinkle distribution thermal map, a spatial deformation grid is obtained by the deformation field generation function. The local deformation stability constraint term is generated by combining the Jacobian matrix calculation and the determinant calculation, and a geometrically regular image is generated by bilinear interpolation sampling.

[0007] Furthermore, the step S2 further includes: According to the illumination-corrected image and the geometrically regular image, the edge mask is extracted by the Sobel operator. The channel attention weight is generated by combining the channel attention mechanism and the feature splicing operation. The channel attention weight is used for weighting, and the Laplacian sharpening operator is combined to enhance the details to generate a standardized barcode image.

[0008] Furthermore, the step S2 further includes: S21: Based on the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask. The calculation method is: in, is the frequency domain feature, is the two-dimensional fast Fourier transform, is the preprocessed package image, is the query vector of the i-th scale, i is the scale index, , is the linear layer of the i-th scale, For downsampling, To extract the real part of the tensor operation, is the key vector of the i-th scale, To extract the imaginary part of the tensor operation, is the multi-scale attention feature, are the query vectors of the first, second and third scales respectively, are the key vectors of the first, second and third scales respectively, is the Softmax function, is the feature dimension, is the lighting compensation mask, is the Sigmoid function, is the convolutional layer, is complex tensor concatenation, Feature splicing; S22: Generate a lighting correction image using the Retinex algorithm based on the lighting compensation mask and the preprocessed package image. The calculation method is: in, Correct the image for lighting, It is the Retinex algorithm; S23: Based on the preprocessed package image, a topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain the wrinkle distribution heat map. The calculation method is: in, is the deformable feature map, is a deformable convolution, is the offset parameter, is a topological structure, Building algorithms for graphs, is the heat map of wrinkle distribution, is the convolution operation, is the partial derivative of the deformable feature map in the horizontal direction, is the partial derivative of the deformable feature map in the ordinate direction; S24: Based on the wrinkle distribution thermal map and the illumination correction image, a geometrically regular image is obtained through deformation field generation function, Jacobian matrix and interpolation calculation. The calculation method is: in, is a spatially deformable grid, is the deformation field generating function, is the Jacobian matrix, is the Jacobian matrix calculation, is a geometrically regular image, is bilinear interpolation sampling, is the Lagrange multiplier, Calculate the determinant; S25: Generate a standardized barcode image by fusing the illumination correction image and the geometric regularization image. The calculation method is: in, is the edge mask, is the Sobel operator, is the channel attention weight, is the channel attention mechanism, To standardize the barcode image, For Hadamard, is the Laplacian sharpening operator.

[0009] It should be further explained that the frequency-domain enhanced multi-scale feature extraction branch and topology-aware deformable convolution branch designed in step S2 of the present invention achieve the coordinated correction of uneven illumination and surface deformation of the package image. Specifically: The core of the frequency-domain enhanced multi-scale feature extraction branch lies in the coordinated design of frequency-domain decomposition and a multi-scale attention mechanism to achieve accurate modeling of illumination distribution. After mapping the image to the frequency domain via a two-dimensional fast Fourier transform, this branch extracts the real and imaginary tensors as the query vector (Q) and key vector (K) of the attention mechanism, respectively. This design directly exploits the physical property differences between the real and imaginary parts in the frequency domain: the real part reflects the global trend of the image brightness distribution (such as the brightness gradient in reflective areas), while the imaginary part characterizes the local changes in texture edges (such as the high-frequency details at shadow boundaries). Attention mechanisms at different scales can respectively capture the macroscopic regularities and local anomalies of the illumination distribution. The fusion of attention features at three scales not only preserves the consistency of global illumination trends but also enhances the compensation capability for areas of local illumination anomalies. Compared to the traditional Retinex algorithm, which relies on local statistical properties to generate illumination masks, the present invention achieves both frequency-domain scale adaptability and spatial distribution robustness of the illumination compensation mask under dynamic illumination conditions through frequency-space dual-domain interactive modeling, significantly improving compensation accuracy.

[0010] The topology-aware deformable convolution branch is mainly aimed at the non-rigid deformation problem of wrapped surface wrinkles. After generating a deformable feature map through deformable convolution, this branch further uses partial derivatives to calculate the gradient field of the feature map (i.e., the transverse and longitudinal gradient components). The direction and intensity of the gradient field directly reflect the direction and severity of the wrinkles (such as longitudinal wrinkles dominated by transverse gradients or transverse wrinkles dominated by longitudinal gradients). By combining the gradient field with the graph construction algorithm, the generated topological graph structure not only strengthens the characteristic response of the local wrinkle area, but also models the global spatial distribution of wrinkles through the connection relationship between the graph nodes. Inter-fold dependencies (such as continuity constraints between adjacent wrinkles); compared to the limitation of traditional STN affine transformation that can only handle rigid deformations such as overall rotation and scaling, this scheme generates non-rigid deformation fields driven by gradient fields. Under the condition that there are local wrinkles on the package surface, it can achieve pixel-level deformation correction and avoid the problem of barcode texture breakage caused by local over-stretching. This feature is particularly significant when dealing with differences in package materials (such as mixed sorting of flexible packaging and rigid cartons). The corrected geometrically regular image retains the key features of the barcode while effectively suppressing artifact interference in the wrinkled area.

[0011] In the process of generating geometrically regular images, the present invention introduces a local deformation stability constraint term , this constraint term quantitatively evaluates the area change rate of the local region, thereby constraining the differentiability and local smoothness of the deformation field; compared with the traditional method that only relies on the deformation field generation mechanism of fixed parameters, the constraint term generated by the determinant calculation in the present invention can dynamically suppress the local over-stretching or folding phenomenon caused by the uneven distribution of wrapping wrinkles. The integrity of the corrected geometric structure is significantly improved, providing a high-fidelity geometric basis for the subsequent generation of standardized barcode images.

[0012] The illumination-corrected image and geometrically regular image obtained by combining the output calculations of the two branches are further enhanced with details through the Laplacian sharpening operator after edge perception and attention fusion to form a standardized barcode image. Compared with the existing technology, the present invention significantly improves the robustness to dynamic lighting interference and irregular deformation in high-speed assembly line scenarios, especially when there are complex wrinkles, reflective labels or tilted packages on the surface. Its standardized image generation efficiency and feature integrity are better than those of single-scale feature extraction or rigid transformation correction methods.

[0013] Furthermore, the step S3 further includes: S31: Based on the standardized barcode image, the standardized barcode features are extracted through the YOLOv7 network and the path aggregation network. The calculation method is: in, To standardize barcode features, is a path aggregation network, It is the YOLOv7 network; S32: Based on the standardized barcode features, an enhanced channel-space dual attention mechanism is designed to enhance the key area features and obtain the channel-space joint features. The calculation method is: in, is the channel weight matrix, is the channel feature, is the global average pooling feature, is the global average pooling, is the spatial weight matrix, is the spatial attention mechanism, is element-wise addition, is the channel-space joint feature, It is a dilated convolution; S33: Extract the barcode bounding box and tilt angle based on the channel-space joint feature. The calculation method is: in, is the barcode bounding box, is the tilt angle, It is the bounding box decoding function based on the preset anchor box; S34: Generate an angle-normalized barcode image based on the barcode bounding box and tilt angle through the spatial transformation network, and recognize the barcode content. The calculation method is: in, is the affine transformation matrix, is the spatial transformation network, is the angle-normalized barcode image, For the barcode content, To connect the temporal classification decoder, It is a convolutional recurrent neural network.

[0014] It should be further explained that in step S3 of the present invention, the core of the enhanced channel-space dual attention mechanism is to achieve dual-domain constraints for feature enhancement through collaborative modeling of the channel weight matrix and the spatial weight matrix.

[0015] Specifically, after generating a channel weight matrix through the channel attention mechanism, this mechanism further uses the global average pooling feature as an auxiliary input and performs an element-by-element addition operation with the output of the spatial attention mechanism. This makes the spatial weight matrix not only rely on the spatial distribution of the local feature map (such as the gradient direction of the barcode edge), but also integrates the global texture statistical characteristics represented by the global average pooling feature. For example, when there is complex background interference (such as printed text or patterns) on the package surface, the channel weight matrix will prioritize enhancing the channel features related to the barcode texture, while the spatial weight matrix uses global features to guide attention to the overall outline of the barcode area, avoiding false responses in local areas. Compared with the limitation of the traditional channel-spatial attention mechanism that generates spatial weights based only on local features, the present invention strengthens the global consistency of the barcode area while retaining local details. In particular, its anti-interference ability is significantly improved for working conditions where there is a large area of ​​interference label on the package surface. In addition, the present invention expands the receptive field through void convolution, allowing the feature fusion process to cover a wider range of contextual information, further suppressing the interference of artifacts similar to barcodes in complex backgrounds.

[0016] To address the problem of barcode rotation, the present invention determines the spatial position of the barcode area through bounding box coordinates, and then calculates the affine transformation matrix based on the tilt angle, so that the barcode characters in the corrected image are always in the horizontal direction; the parameter learning process of the affine transformation network is jointly trained with the subsequent barcode content recognition module, so that the angle correction process can directly respond to the needs of the recognition module; compared with traditional methods that only rely on bounding box regression for angle estimation, the present invention can directly model the arrangement pattern of barcode characters through the collaborative design of the spatial transformation network and the connection temporal classification decoder; for example, when the barcode on the package surface is tilted at any angle, the affine transformation matrix generated by the affine transformation network can project the barcode area into a unified reference coordinate system, so that the recognition module always decodes based on the forward-arranged character sequence, thereby breaking through the template matching method's dependence on fixed angles, which is particularly critical when dealing with angle changes caused by package rotation in high-speed sorting scenarios.

[0017] Furthermore, the step S4 further includes: S41: Based on the barcode content and the sorting window mapping table, a rule matching algorithm is used to perform key-value matching on the address information in the barcode content and the sorting window mapping table to obtain a target delivery window identifier; S42: According to the target delivery window identifier, the swing arm timing parameters and motor drive strength parameters bound to the target delivery window are retrieved through a table lookup matching algorithm to obtain a swing arm control instruction including a pulse width and a drive duration; S43: According to the swing arm control instruction and the package arrival signal detected by the photoelectric sensor, the swing arm rod is triggered to complete the package sorting.

[0018] The present invention also discloses a deep learning-based intelligent parcel sorting system, comprising: Image acquisition module: uses a single camera to capture and preprocess the package image to obtain the preprocessed package image; Illumination and deformation joint correction module: Based on the preprocessed package image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map. The illumination-corrected image and the wrinkle distribution heat map are combined to calculate the geometrically regularized image and the standardized barcode image. Barcode recognition module: Based on the standardized barcode image, it extracts standardized barcode features, designs an enhanced channel-spatial dual attention mechanism, extracts channel-spatial joint features, extracts the barcode bounding box and tilt angle, and recognizes the barcode content; Swing arm control module: Generates swing arm control instructions based on the barcode content and the sorting window mapping table to trigger the swing arm lever to complete package sorting.

[0019] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention addresses the problems of insufficient barcode recognition rate and sorting errors caused by wrinkles, reflections, and shadows on the package surface in traditional single-view vision solutions in swing-arm sorting machines. By jointly correcting illumination and deformation, the quality of standardized images is significantly improved. Then, the channel-space dual attention mechanism and the affine transformation network are combined to achieve high-precision barcode positioning and rotation correction under complex backgrounds. Finally, a swing-arm control signal is generated based on the recognition results to complete accurate sorting actions, effectively improving the barcode recognition rate and system robustness.

[0020] (2) The present invention designs a frequency domain enhanced multi-scale feature extraction branch and a topology-aware deformable convolution branch to achieve joint correction of uneven illumination and surface deformation; the frequency domain enhanced multi-scale feature extraction branch uses two-dimensional fast Fourier transform to extract image frequency domain features, and uses the real part and imaginary part as the query vector and key vector of the attention mechanism respectively, and then outputs the illumination compensation mask through the three-scale attention mechanism fusion, which more accurately models the global trend and local anomalies of the illumination distribution, significantly outperforming the traditional Retinex algorithm; the topology-aware deformable convolution branch generates a gradient field through deformable convolution and partial derivative calculation, combines the graph construction algorithm to generate a wrinkle distribution heat map, and realizes pixel-level non-rigid correction through the deformation field generation function and the local stability constraint term, breaking through the limitation of the traditional STN affine transformation that can only handle rigid deformation. It is particularly suitable for mixed sorting scenarios of flexible packaging and rigid cartons, significantly improving the quality of geometrically regular images and the clarity of standardized barcode images.

[0021] (3) The present invention improves the stability and adaptability of barcode recognition through the collaborative design of the enhanced channel-space dual attention mechanism and the affine transformation network; the enhanced channel-space dual attention mechanism strengthens the global consistency of the barcode area while retaining local details by fusing the channel weight matrix with the spatial weight matrix, so that the system can still focus on the target barcode in a complex background and avoid false detection of interfering labels or patterns; to address the problem of barcode rotation, the affine transformation network jointly estimates the bounding box and the tilt angle to generate an affine transformation matrix to achieve angle-normalized image output; this process is jointly trained with the recognition module, so that the angle correction directly responds to the recognition requirements. Compared with traditional methods, it is more suitable for barcode recognition at any angle and shows stronger adaptability and generalization ability in high-speed sorting scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a process flow of a deep learning-based intelligent parcel sorting method provided by the present invention; Figure 2 This is a schematic diagram of the algorithm flow of the frequency domain enhanced multi-scale feature extraction branch provided by the present invention; Figure 3 Schematic diagram of the algorithm flow of the topology-aware deformable convolution branch provided by the present invention; Figure 4 The present invention provides a comparison example of a barcode with wrinkles on a package before and after correction, wherein: Figure 4 (a) is a sample image of a barcode with wrinkles before correction; Figure 4 (b) is a sample image of a barcode with wrinkles after correction. DETAILED DESCRIPTION

[0023] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0024] Example 1: A method for intelligent parcel sorting based on deep learning, such as Figure 1 As shown, the following steps are included: S1: Use a single camera to capture and preprocess the package image to obtain the preprocessed package image, including: S11: A single camera fixed on the sorting transmission line captures the package image to obtain an original RGB image; S12: Convert the original RGB image into a grayscale image using a three-channel weighted average algorithm; S13: performing median filtering on the grayscale image by sorting the local neighborhood pixel values ​​and taking the median value to remove image noise, thereby obtaining a denoised package image; S14: performing histogram equalization processing on the denoised package image to obtain a preprocessed package image.

[0025] S2: Based on the preprocessed package image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map. The illumination-corrected image and the wrinkle distribution heat map are combined to calculate the geometrically regularized image and the standardized barcode image, including: S21: Based on the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask, such as Figure 2 As shown, the calculation method is: in, is the frequency domain feature, is the two-dimensional fast Fourier transform, is the preprocessed package image, is the query vector of the i-th scale, i is the scale index, , is the linear layer of the i-th scale, For downsampling, To extract the real part of the tensor operation, is the key vector of the i-th scale, To extract the imaginary part of the tensor operation, is the multi-scale attention feature, are the query vectors of the first, second and third scales respectively, are the key vectors of the first, second and third scales respectively, is the Softmax function, is the feature dimension, is the lighting compensation mask, is the Sigmoid function, is the convolutional layer, is complex tensor concatenation, Feature splicing; S22: Generate a lighting correction image using the Retinex algorithm based on the lighting compensation mask and the preprocessed package image. The calculation method is: in, Correct the image for lighting, It is the Retinex algorithm; S23: Based on the preprocessed package image, a topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain the wrinkle distribution heat map, such as Figure 3 As shown, the calculation method is: in, is the deformable feature map, is a deformable convolution, is the offset parameter, is a topological structure, Building algorithms for graphs, is the heat map of wrinkle distribution, is the convolution operation, is the partial derivative of the deformable feature map in the horizontal direction, is the partial derivative of the deformable feature map in the ordinate direction; S24: Based on the wrinkle distribution thermal map and the illumination correction image, a geometrically regular image is obtained through deformation field generation function, Jacobian matrix and interpolation calculation. The calculation method is: in, is a spatially deformable grid, is the deformation field generating function, is the Jacobian matrix, is the Jacobian matrix calculation, is a geometrically regular image, is bilinear interpolation sampling, is the Lagrange multiplier, Calculate the determinant; S25: Generate a standardized barcode image by fusing the illumination correction image and the geometric regularization image. The calculation method is: in, is the edge mask, is the Sobel operator, is the channel attention weight, is the channel attention mechanism, To standardize the barcode image, For Hadamard, is the Laplacian sharpening operator.

[0026] For example, Figure 4 This is a comparison diagram of a barcode image with wrinkles before and after correction in the training sample of the present invention; Figure 4 As shown in (a), before correction, the lower right corner area has obvious distortion and local occlusion, which not only affects the geometric integrity of the barcode, but also may cause traditional recognition algorithms to be unable to accurately extract barcode information, thereby reducing sorting efficiency and accuracy; the present invention uses the frequency domain enhanced multi-scale feature extraction branch to generate an illumination compensation mask, combined with the topology-aware deformable convolution branch to predict the wrinkle distribution heat map, locate the wrinkle area, and realize pixel-level non-rigid correction through the deformation field generation function; as shown in Figure 4 As shown in (b), after correction, the complete structure of the barcode is clearly displayed, providing high-quality input for subsequent recognition.

[0027] S3: Based on the standardized barcode image, extract standardized barcode features, design an enhanced channel-space dual attention mechanism, extract channel-space joint features, extract the barcode bounding box and tilt angle, and recognize the barcode content, including: S31: Based on the standardized barcode image, the standardized barcode features are extracted through the YOLOv7 network and the path aggregation network. The calculation method is: in, To standardize barcode features, is a path aggregation network, It is the YOLOv7 network; S32: Based on the standardized barcode features, an enhanced channel-space dual attention mechanism is designed to enhance the key area features and obtain the channel-space joint features. The calculation method is: in, is the channel weight matrix, is the channel feature, is the global average pooling feature, is the global average pooling, is the spatial weight matrix, is the spatial attention mechanism, is element-wise addition, is the channel-space joint feature, It is a dilated convolution; S33: Extract the barcode bounding box and tilt angle based on the channel-space joint feature. The calculation method is: in, is the barcode bounding box, is the tilt angle, It is the bounding box decoding function based on the preset anchor box; S34: Generate an angle-normalized barcode image based on the barcode bounding box and tilt angle through the spatial transformation network, and recognize the barcode content. The calculation method is: in, is the affine transformation matrix, is the spatial transformation network, is the angle-normalized barcode image, For the barcode content, To connect the temporal classification decoder, It is a convolutional recurrent neural network.

[0028] In particular, for extreme scenarios where the tilt angle is greater than 45°, the present invention also provides an enhanced angle normalization calculation method to replace the step S34. The calculation method is: in, For image rotation operation, is a symbolic function.

[0029] S4: Generate a swing arm control instruction to trigger the swing arm lever according to the barcode content and the sorting window mapping table, including: S41: Based on the barcode content and the sorting window mapping table, a rule matching algorithm is used to perform key-value matching on the address information in the barcode content and the sorting window mapping table to obtain a target delivery window identifier; S42: According to the target delivery window identifier, the swing arm timing parameters and motor drive strength parameters bound to the target delivery window are retrieved through a table lookup matching algorithm to obtain a swing arm control instruction including a pulse width and a drive duration; S43: According to the swing arm control instruction and the package arrival signal detected by the photoelectric sensor, the swing arm rod is triggered to complete the package sorting.

[0030] Example 2: The present invention also discloses a deep learning-based intelligent parcel sorting system, comprising: Image acquisition module: uses a single camera to capture and preprocess the package image to obtain the preprocessed package image; Illumination and deformation joint correction module: Based on the preprocessed package image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map. The illumination-corrected image and the wrinkle distribution heat map are combined to calculate the geometrically regularized image and the standardized barcode image. Barcode recognition module: Based on the standardized barcode image, it extracts standardized barcode features, designs an enhanced channel-spatial dual attention mechanism, extracts channel-spatial joint features, extracts the barcode bounding box and tilt angle, and recognizes the barcode content; Swing arm control module: Generates swing arm control instructions based on the barcode content and the sorting window mapping table to trigger the swing arm lever to complete package sorting.

[0031] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are for descriptive purposes only and do not represent the advantages or disadvantages of the embodiments. In addition, the terms "including", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method comprising the element.

[0032] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in multiple embodiments of the present invention.

[0033] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for intelligent parcel sorting based on deep learning, characterized in that: The following steps are involved: S1: Use a single camera to capture and preprocess the package image to obtain a preprocessed package image; S2: Based on the preprocessed parcel image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate the illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain the wrinkle distribution heat map. Combine the illumination correction image with the wrinkle distribution heat map to calculate the geometric regular image and the standardized barcode image; The frequency-domain enhanced multi-scale feature extraction branch includes: extracting frequency-domain features through two-dimensional fast Fourier transform, using downsampling and linear layers to extract real and imaginary tensors respectively, combining three-scale attention mechanisms to generate multi-scale attention features, and generating illumination compensation masks through complex tensor splicing and convolution layers; the topology-aware deformable convolution branch includes: generating a deformable feature map through deformable convolution, generating a topological map structure through a graph construction algorithm, and generating a wrinkle distribution heat map through convolution operations and partial derivatives; S3: Based on the standardized barcode image, extract the standardized barcode features, design an enhanced channel-space dual attention mechanism, extract the channel-space joint features, extract the barcode bounding box and tilt angle, and recognize the barcode content; The enhanced channel-space dual attention mechanism includes: generating a channel weight matrix through the channel attention mechanism, combining the global average pooling feature with the spatial attention mechanism to generate a spatial weight matrix, and finally generating a channel-space joint feature through weighted fusion and void convolution of the spatial weight matrix; S4: Generate a swing arm control instruction based on the barcode content and the sorting window mapping table to trigger the swing arm lever to complete the package sorting.

2. The deep learning-based intelligent parcel sorting method according to claim 1 is characterized in that: The S1 step includes: S11: A single camera fixed on the sorting transmission line captures the package image to obtain an original RGB image; S12: Convert the original RGB image into a grayscale image using a three-channel weighted average algorithm; S13: performing median filtering on the grayscale image by sorting the local neighborhood pixel values ​​and taking the median value to remove image noise, thereby obtaining a denoised package image; S14: performing histogram equalization processing on the denoised package image to obtain a preprocessed package image.

3. The deep learning-based intelligent parcel sorting method according to claim 1, characterized in that: The S2 step includes: According to the wrinkle distribution thermal map, a spatial deformation grid is obtained by the deformation field generation function. The local deformation stability constraint term is generated by combining the Jacobian matrix calculation and the determinant calculation, and a geometrically regular image is generated by bilinear interpolation sampling.

4. The deep learning-based intelligent parcel sorting method according to claim 1, characterized in that: The S2 step includes: According to the illumination-corrected image and the geometrically regular image, the edge mask is extracted by the Sobel operator. The channel attention weight is generated by combining the channel attention mechanism and the feature splicing operation. The channel attention weight is used for weighting, and the Laplacian sharpening operator is combined to enhance the details to generate a standardized barcode image.

5. The deep learning-based intelligent parcel sorting method according to claim 2, characterized in that: The S2 step includes: S21: Based on the preprocessed package image, a frequency domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask. The calculation method is: in, is the frequency domain feature, is the two-dimensional fast Fourier transform, is the preprocessed package image, is the query vector of the i-th scale, i is the scale index, , is the linear layer of the i-th scale, For downsampling, To extract the real part of the tensor operation, is the key vector of the i-th scale, To extract the imaginary part of the tensor operation, is the multi-scale attention feature, are the query vectors of the first, second and third scales respectively, are the key vectors of the first, second and third scales respectively, is the Softmax function, is the feature dimension, is the lighting compensation mask, is the Sigmoid function, is the convolutional layer, is complex tensor concatenation, Feature splicing; S22: Generate a lighting correction image using the Retinex algorithm based on the lighting compensation mask and the preprocessed package image. The calculation method is: in, Correct the image for lighting, It is the Retinex algorithm; S23: Based on the preprocessed package image, a topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain the wrinkle distribution heat map. The calculation method is: in, is the deformable feature map, is a deformable convolution, is the offset parameter, is a topological structure, Building algorithms for graphs, is the heat map of wrinkle distribution, is the convolution operation, is the partial derivative of the deformable feature map in the horizontal direction, is the partial derivative of the deformable feature map in the ordinate direction; S24: Based on the wrinkle distribution thermal map and the illumination correction image, a geometrically regular image is obtained through deformation field generation function, Jacobian matrix and interpolation calculation. The calculation method is: in, is a spatially deformable grid, is the deformation field generating function, is the Jacobian matrix, is the Jacobian matrix calculation, is a geometrically regular image, is bilinear interpolation sampling, is the Lagrange multiplier, Calculate the determinant; S25: Generate a standardized barcode image by fusing the illumination correction image and the geometric regularization image. The calculation method is: in, is the edge mask, is the Sobel operator, is the channel attention weight, is the channel attention mechanism, To standardize the barcode image, For Hadamard, is the Laplacian sharpening operator.

6. The deep learning-based intelligent parcel sorting method according to claim 5, characterized in that: The S3 step includes: S31: Based on the standardized barcode image, the standardized barcode features are extracted through the YOLOv7 network and the path aggregation network. The calculation method is: in, To standardize barcode features, is a path aggregation network, It is the YOLOv7 network; S32: Based on the standardized barcode features, an enhanced channel-space dual attention mechanism is designed to enhance the key area features and obtain the channel-space joint features. The calculation method is: in, is the channel weight matrix, is the channel feature, is the global average pooling feature, is the global average pooling, is the spatial weight matrix, is the spatial attention mechanism, is element-wise addition, is the channel-space joint feature, It is a dilated convolution; S33: Extract the barcode bounding box and tilt angle based on the channel-space joint feature. The calculation method is: in, is the barcode bounding box, is the tilt angle, It is the bounding box decoding function based on the preset anchor box; S34: Generate an angle-normalized barcode image based on the barcode bounding box and tilt angle through the spatial transformation network, and recognize the barcode content. The calculation method is: in, is the affine transformation matrix, is the spatial transformation network, is the angle-normalized barcode image, For the barcode content, To connect the temporal classification decoder, It is a convolutional recurrent neural network.

7. The deep learning-based intelligent parcel sorting method according to claim 6, characterized in that: The S4 step comprises: S41: Based on the barcode content and the sorting window mapping table, a rule matching algorithm is used to perform key-value matching on the address information in the barcode content and the sorting window mapping table to obtain a target delivery window identifier; S42: According to the target delivery window identifier, the swing arm timing parameters and motor drive strength parameters bound to the target delivery window are retrieved through a table lookup matching algorithm to obtain a swing arm control instruction including a pulse width and a drive duration; S43: According to the swing arm control instruction and the package arrival signal detected by the photoelectric sensor, the swing arm rod is triggered to complete the package sorting.

8. A deep learning-based intelligent parcel sorting system, characterized by: include: Image acquisition module: uses a single camera to capture and preprocess the package image to obtain the preprocessed package image; Illumination and deformation joint correction module: Based on the preprocessed package image, a frequency-domain enhanced multi-scale feature extraction branch is designed to extract the illumination compensation mask and generate an illumination-corrected image. A topology-aware deformable convolution branch is designed to predict the local wrinkle distribution and obtain a wrinkle distribution heat map. The illumination-corrected image and the wrinkle distribution heat map are combined to calculate the geometrically regularized image and the standardized barcode image. Barcode recognition module: Based on the standardized barcode image, it extracts standardized barcode features, designs an enhanced channel-spatial dual attention mechanism, extracts channel-spatial joint features, extracts the barcode bounding box and tilt angle, and recognizes the barcode content; Swing arm control module: Generates swing arm control instructions based on the barcode content and the sorting window mapping table to trigger the swing arm lever to complete package sorting; To implement a deep learning-based intelligent parcel sorting method as described in any one of claims 1-8.

Images