Parcel multi-target identification method and device based on improved YOLOv8

Abstract

The invention discloses a parcel multi-target identification method and device based on improved YOLOv8, relates to the technical field of target identification, and solves the problems that in the prior art, when YOLOv8 carries out multi-target detection, different convolutional layers have different feature attention degrees due to different receptive fields, the detection precision is low, the efficiency is low, and the robustness is poor. The method comprises the steps that a multi-target detection model is constructed based on an improved YOLO, the multi-target detection model comprises a convolutional layer network based on dynamic convolution, a cross-layer target interaction module based on a self-attention mechanism and a rotating frame detection head based on shared convolution, a data set is prepared, a loss function is constructed, and model training is performed; according to the method, the parcel image is detected by using the trained model to output the position information, the type and the correction angle of the target, and then the horizontal correction of the target is performed by using the correction function, so that the identification and the horizontal correction of the multiple targets in the parcel image are realized, and the generalization ability and the robustness of the model and the feature extraction efficiency are enhanced at the same time.

Description

Technical Field

[0001] This application relates to the technical field of target recognition, and in particular to a method and device for multi-target recognition of packages based on improved YOLOv8. Background Art

[0002] Express packages contain one-dimensional codes and two-dimensional codes, which are a machine-readable coding method. Among them, one-dimensional codes are usually composed of black and white stripes of different widths, and two-dimensional codes are composed of nested black and white squares. Both can be used as unique identifiers to identify goods, assets or other items. The three-segment code in the express waybill is mainly composed of numbers and English letters, and the number is divided into three segments by two blanks. The first segment contains the destination sorting center of the express package and the type of the package, the second segment represents the destination branch company and its sub-branches, and the third segment represents the delivery route or the delivery person. By decoding this information, relevant information such as the order information, sorting information and inventory of the package can be quickly obtained. YOLOv8 is a deep learning-based object detection algorithm, and its full name is "You Only Look Once". This algorithm can detect and identify multiple targets in real time in images or videos and is famous for its high speed and accuracy. YOLOv8 is a new version of this series of algorithms, with higher detection accuracy and faster inference speed.

[0003] Detecting and identifying multiple information of packages is crucial in many applications, such as inventory management, logistics tracking and automated settlement. By quickly and accurately detecting and reading package information, the efficiency of the sorting work can be improved, manual errors can be reduced, and the accuracy of data can be ensured. This not only optimizes the operation process, but also provides enterprises with the ability of real-time data tracking and analysis.

[0004] However, in the existing package information recognition technology, due to the certain randomness during package placement, the waybill captured by the camera may not be horizontal, and the waybill contains multiple regions storing information, such as one-dimensional codes, two-dimensional codes, three-segment codes, etc. Therefore, it is necessary to achieve the detection of multiple targets at one time and perform horizontal correction on non-horizontal targets to facilitate subsequent information recognition. However, in the existing YOLOv8 technology, there are problems such as different convolutional layers having different attentions to features due to different receptive fields during multi-target detection, low detection accuracy, low efficiency and poor robustness. Summary of the Invention

[0005] The purpose of this application is to overcome the problems in the existing YOLOv8 technology during multi-target detection, such as different convolutional layers having different attentions to features due to different receptive fields, low detection accuracy, low efficiency and poor robustness, and to provide a method and device for multi-target recognition of packages based on improved YOLOv8.

[0006] In a first aspect, a multi-object recognition method for packages based on improved YOLOv8 is provided, including:

[0007] Construct a correction function for correcting the target area;

[0008] Construct a multi-object detection model, where the multi-object detection model adopts the YOLOV8 structure, including a convolutional layer network based on dynamic convolution, a cross-layer object interaction module based on self-attention mechanism, and a rotated box detection head based on shared convolution. The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer object interaction module based on self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction for multiple targets;

[0009] Obtain the image of the package and perform annotation of the target position, target category, and correction angle to make a data set;

[0010] Construct a combined loss function of the DFL loss function and the Focaler-SIoU loss function;

[0011] Use the data set to train the multi-object detection model until the combined loss function is less than a preset combined loss function value;

[0012] Use the trained multi-object detection model to predict the package image to output the position information, category, and correction angle of the target;

[0013] Input the position information and correction angle of the target into the correction function to obtain a corrected image with a prediction box and target category.

[0014] In some possible implementation manners, the targets include face sheets, one-dimensional codes, two-dimensional codes, and three-segment codes, and the correction function is:

[0015]

[0016] where (x, y) are the initial coordinates of the pixel point in the input image, (x', y') are the coordinates of the pixel point after correction, and θ is the correction angle of the target area.

[0017] In some possible implementation manners, in the multi-scale selection box detection head based on shared convolution, the angle prediction branches of the output layers of different scales are independent of each other.

[0018] In some possible implementation manners, group normalization is adopted in the multi-scale selection box detection head based on shared convolution. The group normalization divides the channels into several groups, calculates the mean and variance of all features within each group, and normalizes each feature within the group.

[0019] In some possible implementation manners, the convolution layers in the backbone network and the neck network are improved by using Kernel Warehouse.

[0020] In some possible implementation manners, in the fusion of features at different levels in the backbone network and the neck network, the initial CGA structure adjusts the input features by using learnable spatial weights, adaptively fuses the low-level features and the corresponding high-level features in the network, and the skip connections in the CGA structure increase the input features.

[0021] In some possible implementation manners, during the process of training a multi-object detection model, the position of the first prediction box is calculated by using the DFL loss function, and then the position of the second prediction box is calculated by using the Focaler-SIoU loss function.

[0022] In a second aspect, a package multi-object recognition device based on an improved YOLOv8 is provided, including:

[0023] A correction function construction module, configured to construct a correction function for correcting a target area;

[0024] A model construction module, configured to construct a multi-object detection model, where the multi-object detection model adopts a YOLOV8 structure, includes a convolutional layer network based on dynamic convolution, a cross-layer object interaction module based on a self-attention mechanism, and a rotated box detection head based on shared convolution. The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer object interaction module based on the self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction on multiple targets;

[0025] A data set acquisition module, configured to acquire an image of a package and perform annotation of the target position, target type, and correction angle to form a data set;

[0026] A loss function construction module, configured to construct a combined loss function of the DFL loss function and the Focaler-SIoU loss function;

[0027] A model training module, configured to train the multi-object detection model by using the data set until the combined loss function is less than a preset combined loss function value;

[0028] A prediction module, configured to perform prediction on a package image by using the trained multi-object detection model to output the position information, type, and correction angle of the target;

[0029] A correction module is used to input the position information and correction angle of the target into a correction function to obtain a corrected image with a prediction box and target category.

[0030] In a third aspect, a computer-readable storage medium is provided. The computer-readable medium stores program code for a device to execute, and the program code includes steps for executing the method in any one of the implementation manners in the first aspect described above.

[0031] In a fourth aspect, an electronic device is provided. The electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor. When the program or instruction is executed by the processor, the method in any one of the implementation manners in the first aspect described above is implemented.

[0032] The present application has the following beneficial effects:

[0033] 1. By improving the convolutional layer network and detection head module of YOLOv8 in the present application, specifically by introducing a cross-layer target interaction module based on the self-attention mechanism at the ends of the backbone network and neck network, the problem that different convolutional layers have different attentions to features due to different receptive fields is solved. Moreover, the hybrid structure of the convolutional neural network and the self-attention mechanism can achieve higher accuracy in single-region detection of the package surface with fewer parameters and computational amounts, enhancing the generalization ability and robustness of the model. At the same time, using the total number of operations of information aggregation of the convolutional layer network based on dynamic convolution and the kernel dense scale of the convolution can further improve the efficiency of the model in extracting face-sheet features;

[0034] 2. In the detection head module of YOLOv8 in the present application, the prediction of the horizontal rotation angle of the target is added, which improves the accuracy of the model in predicting the single region of the package surface. In addition, considering that the true distribution of the target face-sheet region usually does not deviate too far from its labeled position, the model should focus on the values near the labeled position during training. Therefore, by selecting the DFL loss function and the Focaler-SIoU loss function in combination in the present application, the training effect of the model can be effectively improved, and the accuracy of target detection can be further enhanced;

[0035] 3. In the detection head module of YOLOv8 in the present application, group normalization and shared convolution are introduced. The design method of the multi-scale selection box detection head pair can learn the position information and category information of the target through different branches respectively and fuse them, so as to effectively improve the utilization rate of parameters and accelerate the model training speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application.

[0037] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 It is a flowchart of the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0039] Figure 2 It is a schematic diagram of the detection results output by the multi-object detection model in the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0040] Figure 3 It is a schematic diagram of the target position information in the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0041] Figure 4 It is a schematic diagram of the fusion of different-level features in the backbone network and the neck network in the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0042] Figure 5 It is a schematic diagram of the rotated bounding box detection head in the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0043] Figure 6 It is a schematic diagram of the convolution operation process of KernelWarehouse in the multi-object recognition method for packages based on improved YOLOv8 in Embodiment 1 of the present application;

[0044] Figure 7 It is a structural block diagram of the multi-object recognition device for packages based on improved YOLOv8 in Embodiment 2 of the present application;

[0045] Figure 8 It is a schematic diagram of the internal structure of the electronic device in Embodiment 4 of the present application.

[0046] Reference numerals:

[0047] 100, correction function construction module; 200, model construction module; 300, dataset acquisition module; 400, loss function construction module; 500, model training module; 600, prediction module; 700, correction module. Detailed implementation manners

[0048] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0049] Embodiment 1

[0050] As Figure 1 shown, a multi-object recognition method for packages based on improved YOLOv8 involved in Embodiment 1 of this application includes:

[0051] S100. Construct a correction function for correcting the target area;

[0052] Since there is a certain randomness in the placement of packages, there are various angles of deviation between the express package label and the horizontal direction in the pictures taken by the camera. General OCR decoders and barcode decoders are sensitive to such rotated box targets with angles, and are prone to parsing incorrect content or decoding failure. Therefore, for such a deformed label area, a correction module based on rotation transformation is used for correction, and the correction function is:

[0053]

[0054] where (x, y) is the initial coordinate of the pixel point in the input image, (x', y') is the coordinate of the pixel point after correction, and θ is the correction angle of the target area.

[0055] S200. Construct a multi-object detection model, where, as Figure 2 shown, the multi-object detection model adopts the YOLOV8 structure, including a convolutional layer network based on dynamic convolution, a cross-layer target interaction module based on self-attention mechanism, and a rotated box detection head based on shared convolution. The convolutional layer network based on dynamic convolution is used for feature extraction and feature fusion of the targets to be recognized (such as labels, one-dimensional codes, two-dimensional codes, and three-segment codes, etc.). The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer target interaction module based on self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction for multiple targets, and process multi-scale feature maps, etc., to improve the accuracy and efficiency of target detection;

[0056] As Figure 4As shown, in order to improve the effective utilization of features and the accuracy of the face bill detection model, in this embodiment, the fusion of features at different levels in the backbone network and the neck network is designed. The initial CGA structure adjusts the input features using learnable spatial weights and adaptively fuses the low-level features and the corresponding high-level features in the network. Moreover, the skip connections in this structure increase the input features, alleviate the problem of gradient disappearance during model training, and simplify the learning process. Specifically, given the low-level input features and high-level input features of the network, the fused features are calculated as follows:

[0057] F out = Conv 1×1 (F low ·W + F high ·(1 - W)+ F low + F high )

[0058] W = F o ' ut = F global + F local

[0059] F local = Conv 3×3×3 (DSConv(Conv 1×1 (F in )))

[0060]

[0061] Among them, F out is the output feature, F global is the output of the global branch, F local is the output of the local branch, Conv(·) is the convolution operation, DSConv(·) is the channel shuffle operation, F in is the input feature, α is a learnable scaling parameter, and α is used to control and 's multiplication size.

[0062] In the local branch, first use a 1×1 convolution to adjust the channel dimension, then perform a channel transformation operation to mix the channel information, and use depthwise separable convolution to induce channel shuffling. Finally, the output tensors are concatenated along the channel dimension and a 3×3×3 convolution is used to extract features.

[0063] In the global branch, first generate query Q, key value K, and value V through a 1×1 convolution and a 3×3 depthwise separable convolution, obtaining three tensors of shape H×W×C. Then reshape Q into reshape K into and calculate the attention map through the two

[0064] As shown Figure 5 in the rotating box detection head, multiple output layers at the end of the neck network of the multi-object detection model are subjected to the same shared convolution calculation, and then their respective prediction and regression tasks are performed. And the dual-branch of a single scale is modified. After the shared convolution calculation, it is divided into two branches to perform the prediction and regression tasks respectively, which further improves the parameter utilization rate while reducing the number of parameters of the detection head. Since the barcodes of packages in reality are distributed at different angles, the prediction of the horizontal rotation angle of the target is added in the detection head. Since the rotation angle information of different targets in different output layers has no correlation, the angle prediction branches of the output layers of different scales in the rotating box detection head are independent of each other.

[0065] To reduce the influence of different batch sizes on model training, group normalization is used in the detection head. Group normalization divides the channels into several groups, calculates the mean and variance of all features within each group, and normalizes each feature within the group. Compared with batch normalization, group normalization does not depend on the batch size. Therefore, the calculation process of group normalization is independent of the number of samples in the batch, and group normalization can improve the localization and classification performance of the detection head and is suitable for the waybill detection task in this application.

[0066] In the waybill detection task, the waybill area may appear in any direction, and the shapes and sizes are different. Compared with ordinary convolution that uses a fixed convolution kernel to process all input data, this limits its adaptability and flexibility. Although the adaptability and flexibility of dynamic convolution help to better capture these changes, the existing dynamic convolution ignores the parameter dependencies between adjacent convolution layers by allocating a different set of n convolution kernels in each individual convolution layer. To improve the feature expression ability of the waybill detection model and the parameter efficiency in the convolution layer, as Figure 6 shown in this embodiment, KernelWarehouse (KW) is used to improve the convolution layer in the network.

[0067] Compared with the single overall scale paradigm of the existing dynamic convolution, KW applies a hybrid learning paradigm to the dense local scales of static convolution kernels by using Kernel Partition (KP) and Warehouse Sharing (WS). Specifically, for a convolution layer, given an input x ∈ R h×w×c , KP divides the static convolution kernel W ∈ R k×k×c×f into m non-overlapping kernel units w1, w2,... w m one by one in the conventional convolution layer. These kernel units have the same dimension, and the definition formula of KP is as follows:

[0068]

[0069] After KP, w1, w2, …, w m are regarded as local kernels, and a warehouse E = {e1, e2, …, e n} containing n kernel units is defined, where e1, e2, …, e n have the same dimension as w1, w2, …, w m In the same convolutional layer, the warehouse can be shared, and w1, w2, …, w m can be expressed as:

[0070] w i = α i1 e1 + … + α in e n , and i ∈ {1, …, m}

[0071] where α i1 … α in are scalar attentions generated by the attention module based on the input x.

[0072] Since the dimension of the kernel unit can be much smaller than that of the static convolutional kernel W, and its number of parameters is only 1 / m of that of the static convolutional kernel, so under the same parameter budget, a warehouse can have a larger n setting (for example: n > 100). Existing dynamic convolutions involve n global kernels (i.e., each kernel unit has the same dimension as the static convolutional kernel) when defining the linear mixture, and due to its low parameter efficiency problem, usually a smaller n setting is adopted (for example: n < 10). In addition, a warehouse can also be shared to represent each kernel unit in other convolutional layers with the same stage. For the i-th kernel unit in the static convolutional kernel W, let z i1 … z in be the feature logits generated by the second fully connected layer, then the attention function can be defined as the formula:

[0073]

[0074] where τ is the temperature parameter that linearly decreases from 1 to 0 during the model training phase. is a linear normalization function that can produce negative attention outputs. τ is a binary value (0 or 1) used to initialize the attention.

[0075] In the barcode detection task of this embodiment, KW is mainly applied to the convolutional layers in the backbone network and the neck network in the network structure to improve the utilization efficiency of parameters in the network for barcode feature extraction and fusion, and adaptively process complex and diverse input information.

[0076] S300. Obtain images of packages and perform annotations on the target positions, target types, and correction angles to create a dataset;

[0077] Among them, the images of packages are obtained by actually photographing different packages with a camera. Different annotation tools are used for different image formats. Briefly speaking, the target area in the original image of the package is divided and the area is labeled as the target category to create a dataset that can be used for training and validating a multi-object detection model.

[0078] In order to reduce the negative impact caused by sample imbalance in the model training stage, in this embodiment, different preprocessings are performed on simple samples and difficult samples to improve the training quality of the model.

[0079] S400. Construct a combined loss function of the DFL loss function and the Focaler-SIoU loss function;

[0080] It should be noted that the DFL loss function is a loss function for processing bounding box regression. The DFL loss function models the coordinates of the bounding box as a probability distribution, quickly focuses the predicted box position of the model near the target position, and then calculates the coordinate information of the predicted box by weighted calculation.

[0081] The Focaler-SIoU loss function is the loss function used when calculating IOU. It is used to calculate the overlap degree, center point distance, aspect ratio, etc. between the predicted box and the bounding box.

[0082] During the training of the multi-object detection model, DFL first calculates a predicted box position closer to the target position, and then uses Focal-SioU to further refine the predicted box position. The multi-object detection model uses the DFL loss function and the Focaler-SIoU loss function as loss functions. Among them, assuming that the predicted distribution output by the network is P and the true label distribution is Q, the calculation formula of the DFL loss function is as follows:

[0083] DFL(S i ,S i +1)=((y i+1 -y)+(y-y i )log(S i+1 ));

[0084]

[0085] Among them, y i and y i+1 are the two labels closest to the label y.

[0086] Given [u, d] ∈ [0, 1], by adjusting the values of u and d, make IoU FocalerFocusing on different samples, the calculation formula of the Focaler-SIoU loss function is as follows:

[0087] L Focaler-SIoU = L SIoU + IoU - IoU Focaler ;

[0088]

[0089] where, is the coordinate of the center point of the ground truth box, and (b cx , b cy ) is the coordinate of the center point of the predicted box. (c w , c h ) are the width and height of the minimum bounding rectangle of the ground truth box and the predicted box, (w, h) are the width and height of the predicted box, (w gt , h gt ) are the width and height of the ground truth box, θ ∈ [2, 6] is a parameter used to control the shape cost, and << represents much greater than.

[0090] S500. Use the dataset to train the multi-object detection model until the combined loss function is less than a preset combined loss function value;

[0091] In the model training stage, divide the dataset in step S300 into a test set and a validation set according to a certain ratio, for example: 8:2, 7:3, etc. When inputting the image with target annotation information into the multi-object detection model, the convolutional layer network in the multi-object detection model will extract the features of the targets in the model, and continuously learn the feature information of the targets to optimize the internal parameters and related weights.

[0092] S600. Use the trained multi-object detection model to predict the package image to output the position information, type, and correction angle of the target;

[0093] In step S500, after a large number of training and validation stages of the multi-object detection model, the relevant parameters and weights inside the multi-object detection model are determined. At this time, the combined loss function is less than the preset combined loss function value, and the multi-object detection model has the ability to identify the target areas in the image, that is, the trained multi-object detection model is obtained. When inputting the package image into the trained multi-object detection model, the multi-object detection model will perform relevant image region division and calculation inside. Finally, the multi-object detection model will count the calculation results of all regions, and classify the regions with higher scores as target regions and output them. The output results include the target regions framed by the detection boxes, the types of the targets, the correction angles, and the coordinate values of the four corner points of the detection boxes (i.e., the position information of the targets).

[0094] Such as Figure 3As shown, in this embodiment, the two types of coordinate corner points output by the model are uniformly defined as (x1, y1, x2, y2, x3, y3, x4, y4). First, calculate the four values of x2 - x1, y2 - y1, x3 - x2, and y3 - y2. Combining x2 - x1 and y2 - y1 gives an angle θ, and comparing (x2 - x1 2 + y2 - y1 2 ) with (x3 - x2 2 + y3 - y2 2 ) can determine whether the line segment l1 formed by the first two corner points is the long side of the rotated bounding box. If l1 is the long side, the correction angle of the rotated bounding box is Otherwise, it is θ. For each target area (i.e., the face sheet, one-dimensional code, two-dimensional code, and three-segment code), clockwise or counterclockwise rotation is performed.

[0095] S700. Input the position information and correction angle of the target into the correction function to obtain the corrected image with the prediction bounding box and target category.

[0096] In step S600, the correction angle required to correct the target area can be obtained. Inputting this correction angle and the initial coordinates of the pixel points in the image into the correction function in step S100 can calculate the coordinates of the corrected pixel points, and then an image of each target corrected to horizontal can be output. In this image, each target is corrected to horizontal and framed by a detection bounding box, and the category of the target (e.g., face sheet, one-dimensional code, two-dimensional code, or three-segment code) is marked next to the detection bounding box to facilitate the decoding of the target area by the OCR decoder and barcode decoder, thereby improving the decoding accuracy and efficiency.

[0097] In this embodiment, the convolutional layer network and the detection head module of the traditional YOLOv8 are improved. Specifically, a cross-layer target interaction module based on the self-attention mechanism is introduced at the end of the backbone convolutional layer network and the neck convolutional layer network to solve the problem that different convolutional layers have different attentions to features due to different receptive fields. Moreover, the hybrid structure of the convolutional neural network and the self-attention mechanism achieves higher accuracy in detecting single regions of the package surface with fewer parameters and computational costs, enhancing the generalization ability and robustness of the model. At the same time, dynamic convolution is used to adjust the total number of information aggregation operations of the convolutional layers in the network and the kernel density scale of the convolution, which can further improve the efficiency of the model in extracting features of the face sheet. Group normalization and shared convolutional units are introduced into the traditional detection head module. The design method of multiple detection heads can learn and fuse the position and category information of the target through different branches respectively, which improves the utilization rate of parameters and speeds up the model training speed. The prediction of the horizontal rotation angle of the target is added to the detection head to improve the accuracy of the model in predicting single regions of the package surface. Considering that the true distribution of the target face sheet region usually does not deviate too far from its labeled position, the model should focus on the values near the labeled position during training. Therefore, the DFL loss function is selected in combination with the Focaler-SIoU loss function to improve the training effect of the model and further improve the accuracy of object detection. Moreover, the simultaneous detection of face sheets, one-dimensional codes, two-dimensional codes, and three-segment codes is realized, and the inclination angle of the detection frame can also be predicted, so as to correct the detection frame together with the target according to the inclination angle to improve the accuracy and efficiency of subsequent target decoding.

[0098] Embodiment 2

[0099] As Figure 7 shown, a multi-object recognition device for packages based on improved YOLOv8 according to Embodiment 2 of the present application includes:

[0100] A correction function construction module 100 for constructing a correction function for correcting the target area;

[0101] A model construction module 200 for constructing a multi-object detection model, where the multi-object detection model adopts a YOLOV8 structure, including a convolutional layer network based on dynamic convolution, a cross-layer target interaction module based on the self-attention mechanism, and a rotation box detection head based on shared convolution. The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer target interaction module based on the self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction on multiple targets;

[0102] A dataset acquisition module 300 for acquiring images of packages and annotating the target positions, target types, and correction angles to make a dataset;

[0103] A loss function construction module 400 for constructing a combined loss function of a DFL loss function and a Focaler-SIoU loss function;

[0104] A model training module 500 for training the multi-object detection model using the dataset until the combined loss function is less than a preset combined loss function value;

[0105] A prediction module 600 for predicting a package image using the trained multi-object detection model to output the position information, type, and correction angle of the target;

[0106] A correction module 700 for inputting the position information and correction angle of the target into a correction function to obtain a corrected image with a prediction box and the target type.

[0107] It should be noted that for other specific implementation manners of the package multi-object recognition device based on the improved YOLOv8 in this embodiment, reference can be made to the specific implementation manners of the package multi-object recognition method based on the improved YOLOv8 above. To avoid redundancy, it will not be elaborated here.

[0108] Embodiment 3

[0109] A computer-readable storage medium according to Embodiment 3 of the present application, where the computer-readable medium stores program code for device execution, and the program code includes steps for executing the method in any one of the implementation manners in Embodiment 1 of the present application;

[0110] Among them, the computer-readable storage medium may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by a processor, the processor is used to execute the steps of the method in any one of the implementation manners in Embodiment 1 of the present application.

[0111] Embodiment 4

[0112] As Figure 8 shown, an electronic device according to Embodiment 4 of the present application, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, and when the program or instruction is executed by the processor, it implements the method in any one of the implementation manners in Embodiment 1 of the present application;

[0113] Among them, the processor may adopt a general - purpose central processing unit (CPU), a microprocessor, an application - specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation manners in Embodiment 1 of the present application.

[0114] The processor may also be an integrated - circuit electronic device with signal - processing capabilities. In the implementation process, each step of the method in any one of the implementation manners in Embodiment 1 of the present application can be completed by the integrated logic circuit in the hardware of the processor or the instructions in the form of software.

[0115] The above - mentioned processor may also be a general - purpose processor, a digital signal processor, an application - specific integrated circuit (ASIC), a field - programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general - purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random - access memory, a flash memory, a read - only memory, a programmable read - only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the target in the memory and combines its hardware to complete the functions required to be executed by the units included in the data - processing device of the present application, or execute the method in any one of the implementation manners in Embodiment 1 of the present application.

[0116] The above is only the preferred specific implementation manner of the present application; however, the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application, according to the technical solution of the present application and its improved concept, makes equivalent replacements or changes, and should be covered by the protection scope of the present application.

Claims

1. A package multi-target recognition method based on improved YOLOv8, characterized in that: include: Constructing a correction function for correcting the target area; Constructing a multi-target detection model, wherein the multi-target detection model includes a YOLOV8 structure, including a convolutional layer network based on dynamic convolution, a cross-layer target interaction module based on a self-attention mechanism, and a rotation box detection head based on a shared convolution. The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer target interaction module based on a self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction on multiple targets. Obtain images of packages and annotate target locations, target types, and correction angles to create a dataset; Construct a joint loss function of DFL loss function and Focaler-SIoU loss function; Using the data set to train the multi-target detection model until the joint loss function is less than a preset joint loss function value; Use the trained multi-target detection model to predict the package image to output the location information, type and correction angle of the target; The target's position information and correction angle are input into the correction function to obtain a corrected image with a prediction box and target type.

2. The package multi-target recognition method based on improved YOLOv8 according to claim 1, characterized in that: The targets include labels, one-dimensional codes, two-dimensional codes and three-segment codes. The correction function is: Among them, (x, y) is the initial coordinate of the pixel in the input image, (x', y') is the coordinate of the pixel after correction, and θ is the correction angle of the target area.

3. The package multi-target recognition method based on improved YOLOv8 according to claim 1, characterized in that: In the multi-scale selected box detection head based on shared convolution, the angle prediction branches of output layers of different scales are independent of each other.

4. The package multi-target recognition method based on improved YOLOv8 according to claim 3, characterized in that: Group normalization is used in the shared convolution-based multi-scale selection box detection head, which divides the channels into several groups, calculates the mean and variance of all features in each group, and normalizes each feature in the group.

5. The package multi-target recognition method based on improved YOLOv8 according to claim 1, characterized in that: Kernel Warehouse is used to improve the convolutional layers in the backbone network and the neck network.

6. The package multi-target recognition method based on improved YOLOv8 according to claim 1, characterized in that: In the fusion of different levels of features in the backbone network and the neck network, the initial CGA structure uses learnable spatial weights to adjust the input features, adaptively fuses the low-level features in the network and the corresponding high-level features, and the jump connections in the CGA structure increase the input features.

7. The package multi-target recognition method based on improved YOLOv8 according to claim 1, characterized in that: In the process of training the multi-target detection model, the DFL loss function is used to calculate the position of the first prediction box, and then the Focaler-SIoU loss function is used to calculate the position of the second prediction box.

8. A package multi-target recognition device based on improved YOLOv8, characterized in that: include: A correction function building module, used to build a correction function for correcting a target area; A model building module, used to build a multi-target detection model, wherein the multi-target detection model includes a YOLOV8 structure, including a convolutional layer network based on dynamic convolution, a cross-layer target interaction module based on a self-attention mechanism, and a rotation box detection head based on a shared convolution. The convolutional layer network based on dynamic convolution includes a backbone network and a neck network. The cross-layer target interaction module based on a self-attention mechanism is used to connect the output layers at the ends of the backbone network and the neck network. The multi-scale selection box detection head based on shared convolution is used to independently perform position prediction, category prediction, and correction angle prediction on multiple targets. A data set acquisition module is used to acquire images of packages and annotate target positions, target types, and correction angles to create a data set; Loss function construction module, used to construct the joint loss function of DFL loss function and Focaler-SIoU loss function; A model training module, used to train the multi-target detection model using the data set until the joint loss function is less than a preset joint loss function value; The prediction module is used to predict the package image using the trained multi-target detection model to output the location information, type and correction angle of the target; The correction module is used to input the position information and correction angle of the target into the correction function to obtain a corrected image with a prediction box and target type.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method as claimed in any one of claims 1 to 7 when executed by the processor.

Images