Deep learning-based dense parcel visual real-time accurate detection method

Abstract

The invention discloses a dense parcel visual real-time accurate detection method based on deep learning, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a dense parcel image through a 2D industrial camera, and carrying out the preprocessing of the image; based on a GhostConv convolution layer of the YOLOv5n network, convolution is dealt with into convolution operation in a one-dimensional horizontal direction and a one-dimensional vertical direction through factor convolution, first-level features are extracted, and a multi-scale feature map is generated through all the first-level features; position regression is carried out, a regression loss function of a rotating frame is iteratively calculated, the maximum number of iterations or function convergence is achieved, and training of the YOLOv5n network is completed; and dense parcel visual real-time accurate detection is carried out on the dense parcel image through the trained YOLOv5n network. By optimizing a YOLOv5n network structure, efficient and accurate detection of dense, stacked and rotating packages on low-performance equipment is realized, real-time performance and high precision are both considered, detection robustness and overall efficiency are improved, hardware cost is reduced, and the method is suitable for complex logistics scenes.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for real-time and accurate detection of densely packed visual data based on deep learning. Background Art

[0002] In modern logistics and warehouse automation systems, single-piece separation systems have extremely high real-time and precision requirements for visual inspection and precise positioning of dense, multi-variety, and stacked packages. One of the key tasks of such systems is to be able to quickly and accurately detect and separate packages that are constantly moving on conveyor belts, thereby ensuring smooth and efficient logistics. However, current visual inspection methods, especially traditional methods based on 3D camera RGBV detection, face many technical bottlenecks and are difficult to cope with the diversity, density, and stacking of packages in complex application scenarios.

[0003] Traditional 3D camera detection methods usually use algorithms such as image binarization, threshold segmentation, and morphological segmentation to detect packages. These methods can achieve good results in simple scenarios, but in complex logistics environments (such as densely stacked packages, ultra-thin packages, and packages of various shapes), they have the following limitations: Poor generalization ability: Traditional binarization and morphological algorithms are insensitive to changes in package shape, especially when facing packages of different shapes and materials, it is difficult to maintain consistent detection performance. This makes it difficult for these algorithms to maintain efficient detection on packages of different types and sizes; High computational complexity: Methods such as morphological processing and threshold segmentation often require a large amount of computing resources, especially when processing high-resolution images, the amount of calculation is huge, resulting in low detection efficiency. This makes it difficult for these methods to achieve real-time detection in high-throughput scenarios and cannot meet the processing speed of 10,000 pieces / hour required by the single-piece separation system. High-precision real-time detection and positioning algorithms based on deep learning need to be deployed on high-performance GPUs, but high-performance GPU hardware costs are high and cost-effective, and are not suitable for large-scale applications in industrial scenarios. Existing detection algorithms that can run on low-performance CPUs are difficult to balance real-time performance and accuracy. Summary of the invention

[0004] Based on the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a real-time and accurate detection method of dense package vision based on deep learning to solve the above-mentioned technical problems.

[0005] To achieve the above object, the present invention provides the following technical solution: a real-time and accurate detection method of dense package vision based on deep learning, comprising:

[0006] Acquire a dense package image by using a 2D industrial camera, and preprocess the dense package image;

[0007] The preprocessed densely packed image is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations through factored convolution based on the GhostConv convolution layer of the YOLOv5n network, and primary features are extracted, and a multi-scale feature map is generated through all the primary features;

[0008] Performing position regression on the multi-scale feature map, iteratively calculating the regression loss function of the rotation box, reaching a maximum number of iterations or function convergence, and completing the training of the YOLOv5n network;

[0009] The trained YOLOv5n network is used to perform real-time and accurate dense package visual detection on dense package images.

[0010] The present invention is further configured to pre-process the densely packed image, including:

[0011] Performing HSV color space enhancement, perspective transformation, random scale transformation enhancement, and 180-degree image flip enhancement on the densely packed image;

[0012] Performing edge detection on the densely packed image, calculating the transverse gradient and the longitudinal gradient, and extracting the edge features of the package;

[0013] Performing a region search on the densely packed image according to the transverse gradient and the longitudinal gradient to obtain a region search image.

[0014] The present invention is further configured such that, in performing a region search on the densely packed image according to the transverse gradient and the longitudinal gradient to obtain a region search image, the calculation logic of the region search is: Where R(x,y) is the pixel value at the position (x,y) in the region search image, and I(x,y) is the pixel value at the position (x,y) in the densely packed image. and are the lateral and longitudinal gradients at the (x, y) position in the densely packed image.

[0015] The present invention is further configured to decompose the convolution into one-dimensional horizontal and one-dimensional vertical convolution operations through factored convolution to extract primary features, and the calculation logic is: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) are the feature maps after convolution operation in the horizontal and vertical directions respectively, W hor (i) and W ver(j) is the convolution kernel weight in the horizontal and vertical directions, R(x, y) is the densely packed image after preprocessing, and F base (x, y) is the first-level feature image, k and l are the convolution kernel radii in the horizontal and vertical directions respectively.

[0016] The present invention is further configured such that, in generating a multi-scale feature map through all the primary features, the generation logic of the multi-scale feature map is: Among them, F multi (x, y) is a multi-scale feature map, S is the number of convolutional layers, W s are convolution kernels of different scales, and σ(·) is the activation function.

[0017] The present invention is further configured to perform position regression on the multi-scale feature map, and iteratively calculate the regression loss function of the rotation frame, and the calculation logic of the regression loss function of the rotation frame is: Among them, L reg Iteratively calculate the regression loss function of the rotation box, x c and c is the predicted center coordinate of the rotation box on the multi-scale feature map, θ is the predicted rotation angle of the rotation box on the multi-scale feature map, x t and t is the actual center coordinate of the rotation box on the multi-scale feature map, θ t is the actual rotation angle of the rotation box on the multi-scale feature map, α and β are weight parameters, both are greater than zero, and their sum is one.

[0018] The present invention is further configured to perform real-time and accurate dense packing visual detection on the dense packing image through the trained YOLOv5n network, and after obtaining the rotation frame, it also includes:

[0019] Calculating the overlap between the area search image and the rotation frame according to the area search image;

[0020] When the overlap is less than a preset overlap threshold, the rotation frame is marked for manual confirmation.

[0021] The present invention is further configured such that the calculation logic of the overlap between the area search image and the rotation frame is: Wherein, Area(E∩B) is the pixel area of ​​the overlapping part between the area search image and the rotation frame, and Area(B) is the pixel area of ​​the rotation frame.

[0022] The present invention is further configured such that the method further includes: setting independent loss functions to optimize position regression and rotation angle regression respectively through a loss function of multi-objective decoupling regression.

[0023] The present invention is further configured that the method further includes: processing the rotation angle regression through a circular smooth label, setting the rotation angle regression as a periodic problem processing to avoid angle ambiguity and reduce angle error; the calculation logic of the regression loss function of the rotation angle is: Among them, L θ is the regression loss function of the rotation angle, N is the number of rotation angles, θ c is the predicted rotation angle, θ t is the actual rotation angle.

[0024] The present invention provides a method for real-time and accurate detection of dense package vision based on deep learning. The method obtains a dense package image through a 2D industrial camera and preprocesses the dense package image; the preprocessed dense package image is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations based on the GhostConv convolution layer of the YOLOv5n network through factored convolution, and primary features are extracted, and a multi-scale feature map is generated through all the primary features; position regression is performed on the multi-scale feature map, and the regression loss function of the rotation box is iteratively calculated to reach the maximum number of iterations or function convergence, and the training of the YOLOv5n network is completed; the dense package image is subjected to real-time and accurate detection of dense package vision through the trained YOLOv5n network, and the beneficial effects produced include:

[0025] 1. Real-time and high-precision: This paper optimizes the YOLOv5n network structure, introduces GhostConv convolution layer and factored convolution, significantly reduces the amount of convolution operation, and enables the algorithm to run efficiently on low-performance CPUs. At the same time, through multi-scale feature extraction technology, the detection accuracy of packages of different sizes and shapes is effectively improved, while ensuring real-time performance, the accuracy of package detection is improved;

[0026] 2. Enhanced detection capability of rotating packages: The present invention adopts a rotating frame detection mechanism, which can accurately detect the position and posture of rotating packages by calculating the center point, length, width and rotation angle of the package. Combined with the circular smooth label design, it solves the problem that traditional detection methods cannot effectively handle rotating packages, greatly reduces the detection error caused by the rotation angle, and improves the detection accuracy of dense packages and stacked packages;

[0027] 3. Optimize resource utilization and adapt to low-performance devices: This invention focuses on efficient implementation on low-performance devices (such as CPUs). By using lightweight network structures and efficient convolution operations, the computational complexity of the model is greatly reduced. Compared with traditional deep learning models, this invention can achieve efficient real-time detection without relying on high-performance GPUs, making it more cost-effective and suitable for large-scale industrial application scenarios;

[0028] 4. Reduce detection errors and improve system efficiency: Through the overlap calculation of the rotating frame and the edge enhancement results, the present invention can dynamically adjust the detection confidence and effectively reduce false detection and missed detection. Combined with independent classification and position regression loss function optimization, the detection accuracy is further improved, thereby improving the detection accuracy and stability of the entire system, and improving the overall efficiency of the logistics sorting and single-piece separation system.

[0029] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0031] Figure 1 A flowchart of a method for real-time accurate detection of densely packed visual data based on deep learning is shown as an exemplary embodiment of the present invention;

[0032] Figure 2 A real-time effect diagram of a method for real-time and precise detection of dense packaging vision based on deep learning according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0034] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0035] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0036] A real-time and accurate detection method of dense package vision based on deep learning, such as Figure 1 As shown, including:

[0037] Acquire a dense package image by using a 2D industrial camera, and preprocess the dense package image;

[0038] The preprocessed densely packed image is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations through factored convolution based on the GhostConv convolution layer of the YOLOv5n network, and primary features are extracted, and a multi-scale feature map is generated through all the primary features;

[0039] Performing position regression on the multi-scale feature map, iteratively calculating the regression loss function of the rotation box, reaching a maximum number of iterations or function convergence, and completing the training of the YOLOv5n network;

[0040] The trained YOLOv5n network is used to perform real-time and accurate dense package visual detection on dense package images.

[0041] Specifically, in a logistics scenario, packages are moved at high speed via a conveyor belt, so an industrial camera is used to photograph these packages to obtain real-time images of the packages. The present invention is further configured to pre-process the dense package images, including:

[0042] The dense package image is enhanced by HSV color space, perspective transformation, random scale transformation and 180-degree image flipping; specifically, HSV color space enhancement: HSV (Hue, Saturation, Value) color space is a color representation model, representing hue, saturation and brightness respectively. By enhancing the image in HSV space, the brightness, saturation and hue of the image can be adjusted to simulate image changes under different lighting conditions. Specific enhancement methods include: adjusting brightness (Value) to simulate light changes; adjusting hue (Hue) to simulate the appearance of packages under different colors; adjusting saturation (Saturation) to increase or decrease the concentration of the color. Perspective transformation: Perspective transformation is to simulate changes in camera shooting angles. In logistics scenarios, packages may be at different shooting angles. By performing perspective transformation on the image, the network can learn the features of the package at different viewing angles and enhance the generalization ability of the detection model. Random scale transformation enhancement: The size of the package varies in logistics scenarios. By performing random scale changes on the image, the network can adapt to packages of various sizes. This allows the detection model to better handle the detection tasks of large and small packages. 180-degree image flip enhancement: This operation is used to simulate the situation where packages may appear at different angles on the conveyor belt, especially inverted packages. By flipping the image, the network can learn the performance of the package at different rotation angles, thereby improving the robustness of detection.

[0043] Perform edge detection on the densely packed image, calculate the horizontal gradient and the vertical gradient, and extract the edge features of the package; specifically, the horizontal gradient and the vertical gradient of the image are calculated by the gradient change of the image. The horizontal gradient reflects the edge change in the horizontal direction of the image, and the vertical gradient reflects the edge change in the vertical direction;

[0044] The densely packed image is subjected to a region search according to the transverse gradient and the longitudinal gradient to obtain a region search image. The present invention is further configured such that, in the region search according to the transverse gradient and the longitudinal gradient to obtain a region search image, the calculation logic of the region search is: Where R(x,y) is the pixel value at the position (x,y) in the region search image, and I(x,y) is the pixel value at the position (x,y) in the densely packed image. and is the lateral gradient and longitudinal gradient at the (x, y) position in the densely packed image. Specifically, the purpose of the above calculation logic is to perform a region search through the gradient information of edge detection. Specifically, this logic generates a region search image based on the lateral gradient and longitudinal gradient of the image, combined with the pixel value of the original image. In this way, the features of the package edge can be significantly enhanced, and the background noise and invalid areas can be effectively removed, highlighting the location and shape of the package; by multiplying the image pixel value with the gradient information, the edge information can be effectively enhanced while suppressing the background noise. In particular, parts with more obvious edges will get a stronger response, while parts with unclear edges or background will be suppressed;

[0045] The present invention is further configured to decompose the convolution into one-dimensional horizontal and one-dimensional vertical convolution operations through factored convolution to extract primary features, and the calculation logic is: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) are the feature maps after convolution operation in the horizontal and vertical directions respectively, W hor (i) and W ver (j) is the convolution kernel weight in the horizontal and vertical directions, R(x, y) is the densely packed image after preprocessing, and F base (x, y) is the first-level feature image, k and l are the convolution kernel radii in the horizontal and vertical directions respectively; Specifically, the above calculation logic decomposes the two-dimensional convolution operation through factored convolution, specifically decomposing it into one-dimensional convolution in the horizontal and vertical directions. The goal of the decomposition is to optimize the feature extraction process by reducing the amount of calculation while maintaining the ability to perceive the features. Factored convolution reduces the overall calculation complexity and improves the processing speed by separately calculating the features in the horizontal and vertical directions, and still maintains a strong perception ability for the detection of large-scale packages;

[0046] The present invention is further configured such that, in generating a multi-scale feature map through all the primary features, the generation logic of the multi-scale feature map is: Among them, F multi (x, y) is a multi-scale feature map, S is the number of convolutional layers, W s are convolution kernels of different scales, and σ(·) is the activation function. Specifically, the above logic improves the robustness and accuracy of target detection by generating multi-scale feature maps. Specifically, the generation of multi-scale feature maps is achieved by weighted combination of the first-level features extracted by each convolution layer and processing them through the activation function to obtain the final feature map F multi(x,y) is used for accurate package detection; the core idea of ​​the multi-scale feature map is to enable the detection model to capture the features of large and small objects in the image at the same time by fusing feature information of different scales. Therefore, regardless of the size of the package, the model can extract effective information from the multi-scale feature map, thereby improving detection accuracy.

[0047] The present invention is further configured to perform position regression on the multi-scale feature map, and iteratively calculate the regression loss function of the rotation frame, and the calculation logic of the regression loss function of the rotation frame is: Among them, L reg Iteratively calculate the regression loss function of the rotation box, x c and c is the predicted center coordinate of the rotation box on the multi-scale feature map, θ is the predicted rotation angle of the rotation box on the multi-scale feature map, x t and t is the actual center coordinate of the rotation box on the multi-scale feature map, θ t is the actual rotation angle of the rotation box on the multi-scale feature map, α and β are weight parameters, both greater than zero, and the sum is one; specifically, the rotation box is performed on the multi-scale feature map, and each detection unit on the multi-scale feature map needs to set a rotation box. The rotation box is different from the traditional horizontal bounding box. It not only indicates the position of the package (determined by the center point coordinates), but also indicates the rotation angle of the package (described by the angle parameter); in the deep learning detection task, the network processing method is to predict the potential target object (such as a package) through each detection unit (such as a grid unit or anchor box in the multi-scale feature map). Each detection unit will try to predict whether there is a target object in the area, and set a rotation box for each potential target object (i.e., package); after setting the rotation box, position regression is required, and the regression loss function of the rotation box is iteratively calculated. The core of the above calculation logic is the regression loss function of the rotation box, which is used to optimize the network's positioning and rotation angle prediction of the target object (such as a package). In target detection, it is usually necessary to predict the center point position, size and rotation angle of the package. To ensure the accuracy of detection, the loss function is used to measure the gap between the predicted value and the true value. The network gradually improves the accuracy of package detection by minimizing the loss function. This formula combines the center position error and the rotation angle error of the package to ensure that the network can accurately predict not only the location of the package, but also the rotation state of the package;

[0048] The effects of the method according to the present invention are as follows: Figure 2 As shown, the present invention is further configured to perform real-time and accurate dense packaging visual detection on the dense packaging image through the trained YOLOv5n network, and after obtaining the rotation frame, it also includes:

[0049] According to the area search image, the overlap degree between the area search image and the rotation frame is calculated; the present invention is further configured such that the calculation logic of the overlap degree between the area search image and the rotation frame is: Among them, Area(E∩B) is the pixel area of ​​the overlapping part of the regional search image and the rotating box, and Area(B) is the pixel area of ​​the rotating box. Specifically, the core of the above calculation logic is to evaluate the matching degree between the rotating box predicted by the model and the regional search image by calculating the overlap, that is, calculating the overlapping area between the regional search image and the predicted rotating box. The size of the overlap reflects whether the rotating box predicted by the model is highly overlapped with the actual position of the package. By calculating the overlap, the confidence of the rotating box can be further adjusted to ensure the reliability of the detection results.

[0050] When the overlap is less than a preset overlap threshold, the rotation frame is marked for manual confirmation.

[0051] The present invention is further configured such that the method further includes: setting independent loss functions to optimize position regression and rotation angle regression respectively through the loss function of multi-objective decoupling regression. The present invention is further configured such that the method further includes: processing rotation angle regression through circular smooth labels, setting rotation angle regression as periodic problem processing, so as to avoid angle ambiguity and reduce angle error; the calculation logic of the regression loss function of the rotation angle is: Among them, L θ is the regression loss function of the rotation angle, N is the number of rotation angles, θ c is the predicted rotation angle, θ tis the actual rotation angle. Specifically, the core of the above logic is to optimize the angle prediction of the rotating box through the regression loss function of the rotation angle. In object detection, the rotating box not only needs to accurately predict the position of the object, but also needs to accurately describe its rotation angle. Since the rotation angle is a periodic variable (i.e., a rotation of 360 degrees is equivalent to 0 degrees), the traditional angle regression may cause unnecessary errors due to the periodic characteristics of the angle. The present invention optimizes the regression of the rotation angle by introducing a circular smooth label (Circular Smooth Label) and periodic processing of the angle, reduces the ambiguity in the angle prediction, and ultimately improves the detection accuracy of the rotating package; Target decoupling regression loss function In order to optimize the position and rotation angle in package detection, the present invention uses a decoupled regression loss function to independently optimize the position regression and the rotation angle regression. This means that each loss function is calculated and minimized separately to avoid mutual interference when optimizing multiple parameters at the same time. The regression loss function of the rotation angle For the angle prediction of the rotating box, a special rotation angle regression loss function is used. Since the rotation angle is a periodic variable, the traditional linear regression method may produce unnecessary errors when dealing with angle problems. For example, although 0 degrees and 360 degrees are physically the same, they are considered to be very different during regression calculation. In order to solve this problem, the present invention performs periodic processing on angle regression.

[0052] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0053] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0054] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0055] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0056] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0057] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0058] In the several embodiments provided in the present application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0059] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0060] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0061] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0062] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A real-time and accurate detection method of dense package vision based on deep learning, characterized in that: include: Acquire a dense package image by using a 2D industrial camera, and preprocess the dense package image; The preprocessed densely packed image is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations through factored convolution based on the GhostConv convolution layer of the YOLOv5n network, and primary features are extracted, and a multi-scale feature map is generated through all the primary features; Performing position regression on the multi-scale feature map, iteratively calculating the regression loss function of the rotation box, reaching a maximum number of iterations or function convergence, and completing the training of the YOLOv5n network; The trained YOLOv5n network is used to perform real-time and accurate dense package visual detection on dense package images.

2. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 1, characterized in that: Preprocessing the densely packed image includes: Performing HSV color space enhancement, perspective transformation, random scale transformation enhancement, and 180-degree image flip enhancement on the densely packed image; Performing edge detection on the densely packed image, calculating the transverse gradient and the longitudinal gradient, and extracting the edge features of the package; Performing a region search on the densely packed image according to the transverse gradient and the longitudinal gradient to obtain a region search image.

3. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 2, characterized in that: In performing a region search on the densely packed image according to the transverse gradient and the longitudinal gradient to obtain a region search image, the calculation logic of the region search is: Where R(x,y) is the pixel value at the position (x,y) in the region search image, and I(x,y) is the pixel value at the position (x,y) in the densely packed image. and are the lateral and longitudinal gradients at the (x, y) position in the densely packed image.

4. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 1, characterized in that: Through factored convolution, the convolution is decomposed into one-dimensional horizontal and one-dimensional vertical convolution operations to extract the first-level features. The calculation logic is: F base (x,y)=F hor (x,y)+F ver (x,y), where F hor (x,y) and F ver (x, y) are the feature maps after convolution operation in the horizontal and vertical directions respectively, W hor (i) and W ver (j) is the convolution kernel weight in the horizontal and vertical directions, R(x, y) is the densely packed image after preprocessing, and F base (x, y) is the first-level feature image, k and l are the convolution kernel radii in the horizontal and vertical directions respectively.

5. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 4, characterized in that: In generating a multi-scale feature map through all the first-level features, the generation logic of the multi-scale feature map is: Among them, F multi (x, y) is a multi-scale feature map, S is the number of convolutional layers, W s are convolution kernels of different scales, and σ(·) is the activation function.

6. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 1, characterized in that: Position regression is performed on the multi-scale feature map, and the regression loss function of the rotation frame is iteratively calculated. The calculation logic of the regression loss function of the rotation frame is: Among them, L reg Iteratively calculate the regression loss function of the rotation box, x c and c is the predicted center coordinate of the rotation box on the multi-scale feature map, θ is the predicted rotation angle of the rotation box on the multi-scale feature map, x t and t is the actual center coordinate of the rotation box on the multi-scale feature map, θ t is the actual rotation angle of the rotation box on the multi-scale feature map, α and β are weight parameters, both are greater than zero, and their sum is one.

7. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 1, characterized in that: The trained YOLOv5n network is used to perform real-time and accurate detection of densely packed images. After obtaining the rotation frame, it also includes: Calculating the overlap between the area search image and the rotation frame according to the area search image; When the overlap is less than a preset overlap threshold, the rotation frame is marked for manual confirmation.

8. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 7, characterized in that: The calculation logic of the overlap between the area search image and the rotation frame is: Wherein, Area(E∩B) is the pixel area of ​​the overlapping part of the area search image and the rotation frame, and Area(B) is the pixel area of ​​the rotation frame.

9. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 6, characterized in that: The method also includes: setting independent loss functions to optimize position regression and rotation angle regression respectively through the loss function of multi-objective decoupling regression.

10. The method for real-time accurate detection of densely packed visual data based on deep learning according to claim 9, characterized in that: The method further includes: processing the rotation angle regression through a circular smooth label, setting the rotation angle regression as a periodic problem processing, so as to avoid angle ambiguity and reduce angle error; the calculation logic of the regression loss function of the rotation angle is: Among them, L θ is the regression loss function of the rotation angle, N is the number of rotation angles, θ c is the predicted rotation angle, θ t is the actual rotation angle.

Images