A Deep Learning-Based Real-Time Accurate Visual Detection Method for Dense Packages

Abstract

This invention discloses a real-time and accurate visual detection method for dense packages based on deep learning, belonging to the field of computer vision technology. The method includes: acquiring dense package images using a 2D industrial camera and preprocessing the images; using a GhostConv convolutional layer based on the YOLOv5n network, factoring the convolution into one-dimensional horizontal and one-dimensional vertical convolution operations to extract first-level features, and generating multi-scale feature maps using all first-level features; performing position regression, iteratively calculating the regression loss function of the rotated bounding box until the maximum number of iterations or function convergence is reached, completing the training of the YOLOv5n network; and using the trained YOLOv5n network to perform real-time and accurate visual detection of dense packages in the dense package images. By optimizing the YOLOv5n network structure, efficient and accurate detection of dense, stacked, and rotated packages is achieved on low-performance devices, balancing real-time performance and high accuracy, improving detection robustness and overall efficiency, while reducing hardware costs, making it suitable for complex logistics scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method for real-time and accurate visual detection of dense packages based on deep learning. Background Technology

[0002] In modern logistics and warehouse automation systems, single-item separation systems have extremely high requirements for real-time performance and accuracy in visual inspection and precise positioning of dense, diverse, and stacked packages. One of the key tasks of such systems is to quickly and accurately detect and separate packages that are constantly moving on conveyor belts, thereby ensuring smooth and efficient logistics. However, current visual inspection methods, especially traditional 3D camera-based RGBV inspection methods, face many technical bottlenecks and struggle to cope with the diversity, density, and stacking of packages in complex application scenarios.

[0003] Traditional 3D camera detection methods typically employ image binarization, thresholding, and morphological segmentation algorithms to detect packages. While these methods perform well in simple scenarios, they suffer from limitations in complex logistics environments (such as densely stacked packages, ultra-thin packages, and packages with diverse shapes): Poor generalization ability: Traditional binarization and morphological algorithms are insensitive to changes in package shape, especially when dealing with packages of different shapes and materials, making it difficult to maintain consistent detection performance. This hinders the efficient detection of packages of different types and sizes. High computational complexity: Morphological processing and thresholding often require significant computational resources, particularly when processing high-resolution images, resulting in low detection efficiency. This makes it difficult for these methods to achieve real-time detection in high-throughput scenarios, failing to meet the processing speed requirement of 10,000 packages / hour for single-item separation systems. High-precision real-time detection and localization algorithms based on deep learning require deployment on high-performance GPUs, but high-performance GPUs are expensive and cost-effective, making them unsuitable for large-scale industrial applications. Existing detection algorithms that can run on low-performance CPUs struggle to balance real-time performance and accuracy. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a real-time and accurate visual detection method for dense packages based on deep learning, so as to solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a real-time and accurate visual detection method for dense packages based on deep learning, comprising:

[0006] Dense package images are acquired using a 2D industrial camera, and the dense package images are then preprocessed.

[0007] The preprocessed densely wrapped image is subjected to factorization convolution through the GhostConv convolutional layer of the YOLOv5n network. The convolution is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations to extract first-level features. Multi-scale feature maps are generated through all the first-level features.

[0008] Position regression is performed on the multi-scale feature map, and the regression loss function of the rotated box is iteratively calculated until the maximum number of iterations or function convergence is reached, thus completing the training of the YOLOv5n network.

[0009] The trained YOLOv5n network is used to perform real-time and accurate visual detection of densely packed images.

[0010] The present invention is further configured to preprocess the densely packed image, including:

[0011] The densely packed image is enhanced with HSV color space, perspective transformation, random scale transformation, and 180-degree image flipping.

[0012] Edge detection is performed on the densely packed image, the horizontal and vertical gradients are calculated, and the edge features of the package are extracted.

[0013] Based on the horizontal and vertical gradients, a region search is performed on the densely packed image to obtain a region search image.

[0014] The present invention is further configured such that, in obtaining a region search image by performing a region search on the densely packed image based on the horizontal gradient and the vertical gradient, the calculation logic for the region search is as follows: Where R(x,y) is the pixel value at position (x,y) in the region search image, and I(x,y) is the pixel value at position (x,y) in the dense wrapper image. and These are the horizontal and vertical gradients at position (x,y) in the densely wrapped image.

[0015] The present invention is further configured to extract first-level features by factoring convolution, decomposing the convolution into one-dimensional horizontal and one-dimensional vertical convolution operations. The calculation logic is as follows: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) represent the feature maps after convolution in the horizontal and vertical directions, respectively. W hor (i) and W ver(j) represents the convolution kernel weights in the horizontal and vertical directions, R(x,y) represents the preprocessed densely wrapped image, and F base (x,y) represents the first-level feature image, and k and l represent the convolution kernel radii in the horizontal and vertical directions, respectively.

[0016] The present invention is further configured such that, in generating a multi-scale feature map using all the first-level features, the generation logic of the multi-scale feature map is as follows: Among them, F multi (x,y) represents the multi-scale feature map, S represents the number of convolutional layers, and W represents the number of layers. s For convolution kernels of different scales, σ(·) is the activation function.

[0017] The present invention is further configured to perform position regression on the multi-scale feature map and iteratively calculate the regression loss function of the rotated box. The calculation logic of the regression loss function of the rotated box is as follows: Among them, L reg To iteratively calculate the regression loss function of the rotated box, x c and y c Let x be the predicted center point coordinates of the bounding box on the multi-scale feature map, θ be the predicted rotation angle of the bounding box on the multi-scale feature map, and x be the predicted center point coordinates of the bounding box. t and y t Let θ be the actual center point coordinates of the rotating bounding box on the multi-scale feature map. t α represents the actual rotation angle of the bounding box on the multi-scale feature map, and α and β are weight parameters, both of which are greater than zero and sum to one.

[0018] The present invention is further configured to perform real-time and accurate visual detection of densely packed images using a trained YOLOv5n network, and after obtaining the rotated bounding box, it also includes:

[0019] Based on the region search image, calculate the degree of overlap between the region search image and the rotating frame;

[0020] When the overlap is less than the preset overlap threshold, the rotating frame is marked and manually confirmed.

[0021] The present invention is further configured such that the calculation logic for the overlap between the region search image and the rotating frame is as follows: Where Area(E∩B) is the pixel area of ​​the overlapping portion of the region search image and the rotating frame, and Area(B) is the pixel area of ​​the rotating frame.

[0022] The present invention is further configured such that the method further includes: using the loss function of multi-objective decoupled regression, setting independent loss functions to optimize position regression and rotation angle regression respectively.

[0023] The present invention further specifies that the method includes: processing rotation angle regression through circular smooth labels, and treating rotation angle regression as a periodic problem to avoid angle ambiguity and reduce angle error; the calculation logic of the regression loss function for rotation angle is as follows: Among them, L θ Let θ be the regression loss function for the rotation angle, where N is the number of rotation angles, and θ is the value of θ. c For the predicted rotation angle, θ t This represents the actual rotation angle.

[0024] This invention provides a real-time and accurate visual detection method for dense packages based on deep learning. The method acquires dense package images using a 2D industrial camera and preprocesses these images. Then, it applies factorial convolutions to the GhostConv convolutional layer of the YOLOv5n network, decomposing the convolutions into one-dimensional horizontal and one-dimensional vertical convolutions to extract primary features. A multi-scale feature map is generated from these primary features. Position regression is performed on the multi-scale feature map, iteratively calculating the regression loss function of the rotated bounding box until the maximum number of iterations or function convergence is reached, completing the training of the YOLOv5n network. The trained YOLOv5n network enables real-time and accurate visual detection of dense packages, producing the following beneficial effects:

[0025] 1. Balancing Real-Time Performance and High Accuracy: This invention optimizes the YOLOv5n network structure by introducing GhostConv convolutional layers and factorial convolutions, significantly reducing the computational cost of convolution operations and enabling the algorithm to run efficiently on low-performance CPUs. Simultaneously, through multi-scale feature extraction technology, it effectively improves the detection accuracy of packages of different sizes and shapes, enhancing package detection precision while ensuring real-time performance.

[0026] 2. Enhanced Rotating Package Detection Capability: This invention employs a rotating frame detection mechanism. By calculating the center point, length, width, and rotation angle of the package, it can accurately detect the position and orientation of rotating packages. Combined with a smooth circular label design, it solves the problem that traditional detection methods cannot effectively handle rotating packages, significantly reducing detection errors caused by rotation angles and improving the detection accuracy of densely packed and stacked packages.

[0027] 3. Optimized resource utilization, adaptable to low-performance devices: This invention focuses on efficient implementation on low-performance devices (such as CPUs). By using a lightweight network structure and efficient convolutional operations, the computational complexity of the model is significantly reduced. Compared to traditional deep learning models, this invention can achieve efficient real-time detection without relying on high-performance GPUs, making it more cost-effective and suitable for large-scale industrial applications.

[0028] 4. Reduce detection errors and improve system efficiency: By calculating the overlap between the rotating frame and the edge enhancement results, this invention can dynamically adjust the detection confidence level, effectively reducing false detections and missed detections. Combined with independent classification and position regression loss function optimization, the detection accuracy is further improved, thereby enhancing the overall detection accuracy and stability of the system and improving the overall efficiency of the logistics sorting and single-item separation system.

[0029] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0031] Figure 1 A flowchart illustrating a real-time and accurate visual detection method for dense packages based on deep learning, as an exemplary embodiment of the present invention;

[0032] Figure 2 This is a real-time effect diagram of a deep learning-based dense package visual real-time accurate detection method, which is an exemplary embodiment of the present invention. Detailed Implementation

[0033] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0034] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0035] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0036] A real-time and accurate visual detection method for dense packages based on deep learning, such as Figure 1 As shown, it includes:

[0037] Dense package images are acquired using a 2D industrial camera, and the dense package images are then preprocessed.

[0038] The preprocessed densely wrapped image is subjected to factorization convolution through the GhostConv convolutional layer of the YOLOv5n network. The convolution is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations to extract first-level features. Multi-scale feature maps are generated through all the first-level features.

[0039] Position regression is performed on the multi-scale feature map, and the regression loss function of the rotated box is iteratively calculated until the maximum number of iterations or function convergence is reached, thus completing the training of the YOLOv5n network.

[0040] The trained YOLOv5n network is used to perform real-time and accurate visual detection of densely packed images.

[0041] Specifically, in logistics scenarios, packages move at high speed via conveyor belts, so industrial cameras are used to photograph these packages to obtain real-time images. The present invention further includes preprocessing the densely packed package images, comprising:

[0042] The dense package images are enhanced using HSV color space, perspective transformation, random scale transformation, and 180-degree image flipping. Specifically, HSV color space enhancement: HSV (Hue, Saturation, Value) color space is a color representation model, representing hue, saturation, and brightness, respectively. By enhancing the image in the HSV space, the brightness, saturation, and hue of the image can be adjusted to simulate image changes under different lighting conditions. Specific enhancement methods include: adjusting brightness (Value) to simulate changes in light; adjusting hue (Hue) to simulate the appearance of the package under different colors; and adjusting saturation (Saturation) to increase or decrease the color intensity. Perspective transformation: Perspective transformation is used to simulate changes in the camera's shooting angle. In logistics scenarios, packages may be at different shooting angles. By performing perspective transformation on the image, the network can learn the features of the package from different perspectives, enhancing the generalization ability of the detection model. Random scale transformation enhancement: Package sizes vary in logistics scenarios. By performing random scale transformation on the image, the network can adapt to packages of various sizes. This allows the detection model to better handle the detection tasks of large and small packages. 180-degree image flip enhancement: This operation simulates situations where packages may appear at different angles on a conveyor belt, especially inverted packages. By flipping the image, the network can learn how packages behave at different rotation angles, thereby improving detection robustness.

[0043] Edge detection is performed on the densely packed image, calculating the horizontal and vertical gradients to extract the edge features of the packages. Specifically, the horizontal and vertical gradients of the image are calculated through gradient changes in the image. The horizontal gradient reflects edge changes in the horizontal direction, and the vertical gradient reflects edge changes in the vertical direction.

[0044] A region search is performed on the densely packed image based on the horizontal and vertical gradients to obtain a region search image. The present invention further specifies that the region search calculation logic in obtaining the region search image by performing a region search on the densely packed image based on the horizontal and vertical gradients is as follows: Where R(x,y) is the pixel value at position (x,y) in the region search image, and I(x,y) is the pixel value at position (x,y) in the dense wrapper image. and This represents the horizontal and vertical gradients at position (x, y) in the densely packed image. Specifically, the purpose of the above calculation logic is to perform region search using gradient information from edge detection. Specifically, this logic generates a region search image based on the horizontal and vertical gradients of the image, combined with the pixel values ​​of the original image. In this way, the features of the packed edges can be significantly enhanced, and background noise and invalid regions can be effectively removed, highlighting the position and shape of the packed area. By multiplying the image pixel values ​​with the gradient information, edge information can be effectively enhanced while suppressing background noise. In particular, areas with more obvious edges will receive a stronger response, while areas with indistinct edges or background will be suppressed.

[0045] The present invention is further configured to extract first-level features by factoring convolution, decomposing the convolution into one-dimensional horizontal and one-dimensional vertical convolution operations. The calculation logic is as follows: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) represent the feature maps after convolution in the horizontal and vertical directions, respectively. W hor (i) and W ver (j) represents the convolution kernel weights in the horizontal and vertical directions, R(x,y) represents the preprocessed densely wrapped image, and F base (x, y) represents the first-level feature image, where k and l are the kernel radii in the horizontal and vertical directions, respectively. Specifically, the above calculation logic decomposes the two-dimensional convolution operation into factorial convolution, specifically into one-dimensional convolutions in the horizontal and vertical directions. The goal of this decomposition is to optimize the feature extraction process by reducing computational cost while maintaining feature perception. Factorial convolution reduces overall computational complexity and improves processing speed by separately calculating features in the horizontal and vertical directions, while still maintaining strong perception capability for detecting large-scale wrappers.

[0046] The present invention is further configured such that, in generating a multi-scale feature map using all the first-level features, the generation logic of the multi-scale feature map is as follows: Among them, F multi (x,y) represents the multi-scale feature map, S represents the number of convolutional layers, and W represents the number of layers. s For convolutional kernels of different scales, σ(·) is the activation function. Specifically, the above logic improves the robustness and accuracy of object detection by generating multi-scale feature maps. Specifically, the multi-scale feature map is generated by weighting and combining the first-level features extracted from each convolutional layer, and then processing them through an activation function to obtain the final feature map F. multi(x,y) is used for accurate package detection; the core idea of ​​multi-scale feature maps is to fuse feature information from different scales, enabling the detection model to simultaneously capture features of both large and small objects in the image. Therefore, regardless of the package size, the model can extract effective information from the multi-scale feature map, thereby improving detection accuracy.

[0047] The present invention is further configured to perform position regression on the multi-scale feature map and iteratively calculate the regression loss function of the rotated box. The calculation logic of the regression loss function of the rotated box is as follows: Among them, L reg To iteratively calculate the regression loss function of the rotated box, x c and y c Let x be the predicted center point coordinates of the bounding box on the multi-scale feature map, θ be the predicted rotation angle of the bounding box on the multi-scale feature map, and x be the predicted center point coordinates of the bounding box. t and y t Let θ be the actual center point coordinates of the rotating bounding box on the multi-scale feature map. t The actual rotation angle of the bounding box on the multi-scale feature map is α, and β are weight parameters, both greater than zero and summing to one. Specifically, a bounding box is generated on the multi-scale feature map, and each detection unit needs to set a bounding box on the multi-scale feature map. The bounding box differs from the traditional horizontal bounding box; it represents not only the location of the package (determined by the center point coordinates) but also the package's rotation angle (described by the angle parameter). In deep learning detection tasks, the network processes the data by predicting potential target objects (such as packages) through each detection unit (e.g., grid cells or anchor boxes in the multi-scale feature map). Each detection unit attempts to predict whether a target object exists within the region and sets a bounding box for each potential target object (i.e., the package). After setting the bounding box, position regression is required, iteratively calculating the regression loss function of the bounding box. The core of the above calculation logic is the regression loss function of the bounding box, used to optimize the network's localization and rotation angle prediction of the target object (such as the package). In target detection, it is typically necessary to predict the center point position, size, and rotation angle of the package. To ensure detection accuracy, a loss function is used to measure the difference between the predicted and actual values. The network gradually improves the accuracy of package detection by minimizing this loss function. This formula combines the package's center position error and rotation angle error to ensure that the network can not only accurately predict the package's position but also accurately predict its rotation state.

[0048] The effects of implementing the method according to the present invention are as follows: Figure 2 As shown, the present invention is further configured to perform real-time and accurate visual detection of densely packed images using a trained YOLOv5n network, and after obtaining the rotated bounding box, it also includes:

[0049] Based on the region search image, the overlap degree between the region search image and the rotating frame is calculated; the present invention is further configured such that the calculation logic for the overlap degree between the region search image and the rotating frame is as follows: Where Area(E∩B) represents the pixel area of ​​the overlapping portion between the region search image and the rotated bounding box, and Area(B) represents the pixel area of ​​the rotated bounding box. Specifically, the core of the above calculation logic is to evaluate the degree of matching between the model-predicted rotated bounding box and the region search image by calculating the overlap degree, that is, calculating the overlapping area between the region search image and the predicted rotated bounding box. The magnitude of the overlap degree reflects whether the model-predicted rotated bounding box highly coincides with the actual location of the package. By calculating the overlap degree, the confidence of the rotated bounding box can be further adjusted to ensure the reliability of the detection results.

[0050] When the overlap is less than the preset overlap threshold, the rotating frame is marked and manually confirmed.

[0051] The invention further specifies that the method includes: optimizing position regression and rotation angle regression separately by setting independent loss functions through the loss function of multi-objective decoupled regression. The invention further specifies that the method includes: processing rotation angle regression with circular smooth labels, treating rotation angle regression as a periodic problem to avoid angle ambiguity and reduce angle errors; the calculation logic of the rotation angle regression loss function is as follows: Among them, L θ Let θ be the regression loss function for the rotation angle, where N is the number of rotation angles, and θ is the value of θ. c For the predicted rotation angle, θ tThis refers to the actual rotation angle. Specifically, the core of the above logic is to optimize the angle prediction of the rotating bounding box through a regression loss function of the rotation angle. In object detection, the rotating bounding box not only needs to accurately predict the position of the object, but also needs to accurately describe its rotation angle. Since the rotation angle is a periodic variable (i.e., a rotation of 360 degrees is equivalent to 0 degrees), traditional angle regression may lead to unnecessary errors due to the periodicity of the angle. This invention optimizes the regression of the rotation angle by introducing a circular smooth label and periodic processing of the angle, reducing the ambiguity in angle prediction and ultimately improving the detection accuracy of rotating packages. To optimize the position and rotation angle in package detection, this invention uses a decoupled regression loss function, optimizing the position regression and rotation angle regression independently. This means that each loss function is calculated and minimized separately to avoid mutual interference when optimizing multiple parameters simultaneously. For the angle prediction of the rotating bounding box, a dedicated rotation angle regression loss function is used. Since the rotation angle is a periodic variable, traditional linear regression methods may introduce unnecessary errors when dealing with angle problems. For example, although 0 degrees and 360 degrees are physically the same, they are considered to be extremely different in regression calculations. To address this issue, this invention performs periodic processing on the angle regression.

[0052] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0053] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0054] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0055] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0058] In the several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0059] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0060] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0061] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A real-time and accurate visual detection method for dense packages based on deep learning, characterized in that, include: Images of densely packed packages are acquired using a 2D industrial camera, and these images are then preprocessed. The preprocessed densely wrapped image is subjected to factorization convolution through the GhostConv convolutional layer of the YOLOv5n network. The convolution is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations to extract first-level features. Multi-scale feature maps are generated through all the first-level features. Position regression is performed on the multi-scale feature map, and the regression loss function of the rotated box is iteratively calculated until the maximum number of iterations or function convergence is reached, thus completing the training of the YOLOv5n network. The trained YOLOv5n network is used to perform real-time and accurate visual detection of densely packed images.

2. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 1, characterized in that, Preprocessing the densely packed image includes: The densely packed image is enhanced with HSV color space, perspective transformation, random scale transformation, and 180-degree image flipping. Edge detection is performed on the densely packed image, the horizontal and vertical gradients are calculated, and the edge features of the package are extracted. Based on the horizontal and vertical gradients, a region search is performed on the densely packed image to obtain a region search image.

3. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 2, characterized in that, In the process of performing a region search on the densely packed image based on the horizontal and vertical gradients to obtain a region search image, the calculation logic for the region search is as follows: Where R(x,y) is the pixel value at position (x,y) in the region search image, and I(x,y) is the pixel value at position (x,y) in the dense wrapper image. and These are the horizontal and vertical gradients at position (x,y) in the densely wrapped image.

4. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 1, characterized in that, By using factorial convolution, the convolution is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations to extract first-level features. The calculation logic is as follows: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) represent the feature maps after convolution in the horizontal and vertical directions, respectively. W hor (i) and W ver (j) represents the convolution kernel weights in the horizontal and vertical directions, R(x,y) represents the preprocessed densely wrapped image, and F base (x,y) represents the first-level feature image, and k and l represent the convolution kernel radii in the horizontal and vertical directions, respectively.

5. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 4, characterized in that, The logic for generating the multi-scale feature map using all the first-level features is as follows: Among them, F multi (x,y) represents the multi-scale feature map, S represents the number of convolutional layers, and W represents the number of layers. s For convolution kernels of different scales, σ(·) is the activation function.

6. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 1, characterized in that, Position regression is performed on the multi-scale feature map, and the regression loss function of the rotated bounding box is calculated iteratively. The calculation logic of the regression loss function of the rotated bounding box is as follows: Among them, L reg To iteratively calculate the regression loss function of the rotated box, x c and y c Let x be the predicted center point coordinates of the bounding box on the multi-scale feature map, θ be the predicted rotation angle of the bounding box on the multi-scale feature map, and x be the predicted center point coordinates of the bounding box. t and y t Let θ be the actual center point coordinates of the rotating bounding box on the multi-scale feature map. t α represents the actual rotation angle of the bounding box on the multi-scale feature map, and α and β are weight parameters, both of which are greater than zero and sum to one.

7. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 1, characterized in that, The trained YOLOv5n network is used to perform real-time and accurate visual detection of densely packed images. After obtaining the rotated bounding box, the following steps are also taken: Based on the region search image, calculate the degree of overlap between the region search image and the rotating frame; When the overlap is less than the preset overlap threshold, the rotating frame is marked and manually confirmed.

8. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 7, characterized in that, The calculation logic for the overlap between the region search image and the rotated bounding box is as follows: Where Area(E∩B) is the pixel area of ​​the overlapping portion of the region search image and the rotating frame, and Area(B) is the pixel area of ​​the rotating frame.

9. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 6, characterized in that, The method further includes: using the loss function of multi-objective decoupled regression, setting independent loss functions to optimize position regression and rotation angle regression respectively.

10. The method for real-time and accurate visual detection of dense packages based on deep learning according to claim 9, characterized in that, The method further includes: processing rotation angle regression using circular smooth labels, and treating rotation angle regression as a periodic problem to avoid angle ambiguity and reduce angle errors; the calculation logic of the regression loss function for rotation angle is as follows: Among them, L θ Let θ be the regression loss function for the rotation angle, where N is the number of rotation angles, and θ is the value of θ. c For the predicted rotation angle, θ t This represents the actual rotation angle.

Images