Stacked package segmentation method, system, device and storage medium
By calculating the package segmentation candidate points and edge curves through the fitting algorithm, the problem of low segmentation efficiency in nonlinear scenarios in the prior art is solved, and high accuracy segmentation of continuous stacked packages is achieved, and security inspection efficiency is improved.
Patent Information
- Application Number
- CN202411144830.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The prior art cannot effectively segment continuous stacked packages in nonlinear scenarios, resulting in low security inspection efficiency and accuracy, especially in peak periods of time, facing a large amount of package segmentation pressure.
By obtaining the stacked package image to be segmented, the package segmentation candidate point queue and edge curve are calculated based on the preset vertical slice and fitting algorithm, and nonlinear model fitting is used to improve segmentation accuracy.
It significantly improves the segmentation accuracy and efficiency of continuous stacked packages, reduces calculation overhead, and improves security inspection efficiency.
Smart Images

Figure CN118657947B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to the segmentation of stacked packages, and in particular to a method, system, device and storage medium for segmenting stacked packages. Background Art
[0002] With the rapid development of civil aviation and rail transportation, more and more people choose to travel by plane, high-speed rail and subway. When passengers enter the station, their carry-on packages must be inspected for safety. However, due to the excessive concentration of passengers during rush hour, security inspectors are faced with the pressure of inspecting a large number of continuously stacked packages in a short period of time.
[0003] The current method is to first segment the package in the security inspection machine video stream image, and then submit the output package image to the AI image judgment module for contraband identification. Therefore, before AI image judgment, how to improve the efficiency and accuracy of segmenting continuously stacked packages in the security inspection machine video has become a very urgent issue at present. In related technologies, the motion model of the target is usually modeled through historical states and observation data, and the position of the target in the next frame is estimated. However, only linear process models and measurement models can be accurately estimated, and the optimal estimation effect cannot be achieved in nonlinear scenarios. Both process models and measurement models are nonlinear. Summary of the invention
[0004] The present invention aims to at least solve the technical problems existing in the prior art. To this end, the present invention provides a method, system, device and storage medium for segmenting stacked packages, which can improve the segmentation accuracy of continuously stacked packages.
[0005] A first aspect of the present invention provides a method for splitting a stacked package, comprising the following steps:
[0006] Obtain the stacked package image to be segmented;
[0007] Based on the preset vertical slices and the stacked package images to be segmented, a candidate point queue for package segmentation is calculated by a fitting algorithm;
[0008] Based on the parcel segmentation candidate point queue, preset segmentation parameters and preset ratio, the parcel edge curve is calculated by a fitting algorithm;
[0009] The stacked package image to be segmented is segmented according to the package edge curve to obtain a segmented image of the stacked package.
[0010] The control method according to the embodiment of the present invention has at least the following beneficial effects:
[0011] The method obtains an image of a stacked package to be segmented; based on preset vertical slices and the image of the stacked package to be segmented, a candidate point queue for package segmentation is calculated by a fitting algorithm; based on the candidate point queue for package segmentation, preset segmentation parameters and preset ratios, a package edge curve is calculated by a fitting algorithm; the image of the stacked package to be segmented is segmented according to the package edge curve to obtain a segmented image of the stacked package. The method improves the segmentation accuracy of continuously stacked packages by fitting and calculating a nonlinear model.
[0012] According to some embodiments of the present invention, obtaining the image of the stacked packages to be segmented includes:
[0013] Obtain the original stacked package image to be segmented;
[0014] Based on the original stacked package image to be segmented, denoising is performed by a Gaussian filtering method to obtain a denoised stacked package image to be segmented;
[0015] Based on the denoised stacked package image to be segmented, edge detection is performed using a Canny edge detection algorithm to obtain an edge-detected stacked package image to be segmented;
[0016] Based on a preset rectangular kernel, the stacked package image to be segmented after edge detection is expanded to obtain the expanded stacked package image to be segmented;
[0017] Performing contour detection on the expanded stacked package image to be segmented by using a findContours method to obtain a set of point coordinates of the expanded stacked package image to be segmented;
[0018] Based on the point coordinate set, the length and area of the minimum circumscribed rectangle of the expanded stacked wrapped image to be segmented are calculated by a boundingRect function;
[0019] When the length of the minimum circumscribed rectangle is greater than a first preset threshold or the area of the minimum circumscribed rectangle is greater than a second preset threshold, the expanded stacked package image to be segmented is used as the stacked package image to be segmented.
[0020] According to some embodiments of the present invention, the step of calculating a candidate parcel segmentation point queue by a fitting algorithm based on a preset vertical slice and the stacked parcel image to be segmented includes:
[0021] Acquire a first horizontal coordinate range corresponding to each of the preset vertical slices and a second horizontal coordinate range corresponding to the stacked package image to be segmented;
[0022] Based on the set of point coordinates, calculating a first fitting curve by a fitting algorithm;
[0023] Calculating the slice region area corresponding to each of the preset vertical slices based on the point coordinate set and the first horizontal coordinate range;
[0024] Based on the slice area corresponding to all preset vertical slices, a second fitting curve is calculated by a fitting algorithm;
[0025] Calculating a first-order derivative of the first fitting curve within the second horizontal coordinate range to obtain a first fitting derivative;
[0026] Calculating a first-order derivative of the second fitting curve within the second horizontal coordinate range to obtain a second fitting derivative;
[0027] The package segmentation candidate point queue is calculated according to the first horizontal coordinate range, the first fitting derivative, and the second fitting derivative.
[0028] According to some embodiments of the present invention, the calculating, based on the point coordinate set and the first horizontal coordinate range, the slice region area corresponding to each of the preset vertical slices includes:
[0029] Calculating the difference between the maximum value and the minimum value of the first horizontal axis range corresponding to each of the preset vertical slices; and using the difference as the width of the slice area corresponding to the preset vertical slice;
[0030] Calculate the sum of the maximum value and the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices; and use half of the sum as a reference position;
[0031] Searching for two upper edge contour points located closest to the reference position on the upper edge of the stacked package image to be segmented, wherein the upper edge contour points belong to the point coordinate set;
[0032] Calculate the first upper edge mean of the horizontal coordinates of the two nearest upper edge contour points, and when the first upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first upper edge mean as the first vertical coordinate average value;
[0033] When the first upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and a second upper edge mean of the horizontal coordinates of the three closest upper edge contour points is calculated; when the second upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second upper edge mean is used as the first vertical coordinate average value; when the second upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and a third upper edge mean of the horizontal coordinates of the four closest upper edge contour points is calculated, and so on, until the n-1th upper edge mean of the horizontal coordinates of the n upper edge contour points located closest to the reference position is found on the upper edge of the stacked package image to be segmented, and the first vertical coordinate average value is obtained, wherein n is the total number of upper edge contour points located closest to the reference position;
[0034] Searching for two lower edge contour points located closest to a reference position at the lower edge of the stacked package image to be segmented, wherein the lower edge contour points belong to the point coordinate set;
[0035] Calculate the first lower edge mean of the horizontal coordinates of the two nearest lower edge contour points, and when the first lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first lower edge mean as the second vertical coordinate average value;
[0036] When the first lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three lower edge contour points located closest to the reference position are searched on the lower edge of the stacked package image to be segmented, and a second lower edge mean of the horizontal coordinates of the three closest lower edge contour points is calculated; when the second lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second lower edge mean is used as the second vertical coordinate average value; when the second lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four lower edge contour points located closest to the reference position are searched on the lower edge of the stacked package image to be segmented, and a third lower edge mean of the horizontal coordinates of the four closest lower edge contour points is calculated, and so on, until the n-1th lower edge mean of the horizontal coordinates of the n lower edge contour points located closest to the reference position on the lower edge of the stacked package image to be segmented is found to be within the first horizontal coordinate range, and the second vertical coordinate average value is obtained;
[0037] Calculate the difference between the second vertical coordinate average value and the second vertical coordinate average value to obtain the height of the slice area corresponding to the preset vertical slice;
[0038] Based on the height of the slice area and the width of the slice area, the area of the slice area corresponding to the preset vertical slice is calculated.
[0039] According to some embodiments of the present invention, calculating the package segmentation candidate point queue according to the first horizontal coordinate range, the first fitting derivative, and the second fitting derivative includes:
[0040] In the upper edge of the stacked package image to be segmented, obtaining the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices, and using it as an initial package segmentation candidate point queue;
[0041] The contour feature candidate compliance flag is calculated according to the first fitting derivative and the initial parcel segmentation candidate point queue by the following formula:
[0042]
[0043] in, is the contour feature candidate compliance mark, The first candidate point in the initial parcel segmentation queue The horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the third preset threshold, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a fourth preset threshold;
[0044] The area feature candidate compliance flag is calculated according to the second fitting derivative and the initial package segmentation candidate point queue using the following formula:
[0045]
[0046] in, is the area feature candidate compliance mark, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the fifth preset threshold, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a sixth preset threshold;
[0047] When the contour feature candidate meets the flag and the area feature candidate meets the flag is true, the first The initial parcel segmentation candidate points are put into the parcel segmentation candidate point queue; and each initial parcel segmentation candidate point in the initial parcel segmentation candidate point queue is calculated in turn to obtain the parcel segmentation candidate point queue.
[0048] According to some embodiments of the present invention, based on the parcel segmentation candidate point queue, the preset segmentation parameter and the preset ratio, the parcel edge curve is calculated by a fitting algorithm, including:
[0049] Performing region division on each parcel segmentation candidate point in the parcel segmentation candidate point queue according to the preset segmentation parameter and the preset horizontal coordinate offset to obtain a first square sample block corresponding to each parcel segmentation candidate point;
[0050] Screening the first square sample block according to the preset ratio to obtain a second square sample block;
[0051] Based on the coordinates of the center point of the second square sample block, calculating a third fitting curve by a fitting algorithm;
[0052] Calculate a third horizontal coordinate range of each candidate parcel segmentation point in the candidate parcel segmentation point queue according to the preset horizontal coordinate offset;
[0053] Calculating the second-order derivative of the third fitting curve within the third horizontal coordinate range to obtain a third fitting derivative;
[0054] Based on the third fitting derivative, calculating the curvature corresponding to the coordinates of each center point;
[0055] Deleting the maximum and minimum values of all the curvatures to obtain a curvature set;
[0056] Calculating the standard deviation of the curvature set to obtain the curvature standard deviation;
[0057] Calculate the difference between the coordinates of every two adjacent center points, and use it as the spacing of the corresponding second square sample blocks;
[0058] Calculate the standard deviation of all the intervals to obtain the interval standard deviation;
[0059] In the parcel segmentation candidate point queue, the parcel segmentation candidate points whose curvature standard deviation is less than the seventh preset threshold and whose spacing standard deviation is less than the eighth preset threshold are screened to obtain screened parcel segmentation candidate points; and the third fitting curve corresponding to the screened parcel segmentation candidate points is used as the parcel edge curve.
[0060] According to some embodiments of the present invention, segmenting the stacked package image to be segmented according to the package edge curve to obtain a segmented image of the stacked package includes:
[0061] Segmenting the point coordinate set based on the wrapped edge curve to obtain a segmented set;
[0062] The center point coordinates of all the second square sample blocks corresponding to each of the wrapping edge curves are added to the corresponding segmented set to obtain the segmented image of the stacked wrapping.
[0063] A second aspect of the present invention provides a stacked package segmentation system, the stacked package segmentation system comprising:
[0064] A data acquisition module, used for acquiring images of stacked packages to be segmented;
[0065] A parcel segmentation candidate point queue calculation module, used to calculate the parcel segmentation candidate point queue through a fitting algorithm based on a preset vertical slice and a stacked parcel image to be segmented;
[0066] A package edge curve calculation module, used to calculate the package edge curve through a fitting algorithm based on the package segmentation candidate point queue, preset segmentation parameters and preset ratio;
[0067] The image segmentation module is used to segment the stacked package image to be segmented according to the package edge curve to obtain a segmented image of the stacked package.
[0068] The system obtains an image of stacked packages to be segmented; calculates a candidate point queue for package segmentation through a fitting algorithm based on preset vertical slices and the image of the stacked packages to be segmented; calculates a package edge curve through a fitting algorithm based on the candidate point queue for package segmentation, preset segmentation parameters and preset ratios; segments the image of the stacked packages to be segmented according to the package edge curve to obtain a segmented image of the stacked packages. The method improves the segmentation accuracy of continuously stacked packages by fitting and calculating a nonlinear model.
[0069] According to a third aspect of the present invention, there is provided an electronic device for splitting stacked packages, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the above-mentioned method for splitting stacked packages.
[0070] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned method for dividing stacked packages.
[0071] It should be noted that the beneficial effects between the second to fourth aspects of the present invention and the prior art are the same as the beneficial effects between the above-mentioned stacked package segmentation system and the prior art, and will not be described in detail here.
[0072] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0074] Figure 1 is a flow chart of a method for segmenting stacked packages according to an embodiment of the present invention;
[0075] Figure 2 is a schematic diagram of stacked package segmentation according to a stacked package segmentation method provided by an embodiment of the present invention;
[0076] Figure 3 is a schematic structural diagram of an embodiment of a stacked package segmentation system provided by the present application;
[0077] Figure 4 It is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION
[0078] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0079] In the description of the present invention, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0080] In the description of the present invention, it should be understood that descriptions involving orientation, such as orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0081] In the description of the present invention, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.
[0082] With the rapid development of civil aviation and rail transportation, more and more people choose to travel by plane, high-speed rail and subway. When passengers enter the station, their carry-on packages must be inspected for safety. However, due to the excessive concentration of passengers during rush hour, security inspectors are faced with the pressure of inspecting a large number of continuously stacked packages in a short period of time.
[0083] The current method is to first segment the package in the security inspection machine video stream image, and then submit the output package image to the AI image judgment module for contraband identification. Therefore, before AI image judgment, how to improve the efficiency and accuracy of segmenting continuously stacked packages in the security inspection machine video has become a very urgent issue at present. In related technologies, the motion model of the target is usually modeled through historical states and observation data, and the position of the target in the next frame is estimated. However, only linear process models and measurement models can be accurately estimated, and the optimal estimation effect cannot be achieved in nonlinear scenarios. Both process models and measurement models are nonlinear.
[0084] In order to solve the above technical defects, the embodiments of the present application provide a method, system, device and storage medium for dividing stacked packages.
[0085] See also Figure 1 , is a flow chart of a method for splitting a stacked package provided in an embodiment of the present application, the method is applied to an electronic device, and the electronic device may be a server, etc. Figure 1 As shown, the stacked package segmentation method includes:
[0086] Step S101, obtaining an image of a stacked package to be segmented;
[0087] Step S102, calculating a candidate point queue for parcel segmentation by a fitting algorithm based on a preset vertical slice and the image of the stacked parcels to be segmented;
[0088] Step S103, calculating the parcel edge curve by a fitting algorithm based on the parcel segmentation candidate point queue, preset segmentation parameters and preset ratios;
[0089] Step S104 , segmenting the image of the stacked packages to be segmented according to the package edge curve to obtain a segmented image of the stacked packages.
[0090] The method obtains an image of a stacked package to be segmented; based on preset vertical slices and the image of the stacked package to be segmented, a candidate point queue for package segmentation is calculated by a fitting algorithm; based on the candidate point queue for package segmentation, preset segmentation parameters and preset ratios, a package edge curve is calculated by a fitting algorithm; the image of the stacked package to be segmented is segmented according to the package edge curve to obtain a segmented image of the stacked package. The method improves the segmentation accuracy of continuously stacked packages by fitting and calculating a nonlinear model.
[0091] In some embodiments, obtaining the image of the stacked packages to be segmented in step S101 includes:
[0092] Step S201, obtaining an original image of stacked packages to be segmented;
[0093] Step S202: Based on the original image of the stacked packages to be segmented, denoising is performed by using a Gaussian filtering method to obtain a denoised image of the stacked packages to be segmented;
[0094] Step S203: Based on the denoised image of the stacked package to be segmented, edge detection is performed using a Canny edge detection algorithm to obtain an edge-detected image of the stacked package to be segmented;
[0095] Step S204, dilating the stacked package image to be segmented after edge detection based on a preset rectangular kernel to obtain an expanded stacked package image to be segmented;
[0096] Step S205, performing contour detection on the expanded stacked package image to be segmented by using the findContours method to obtain a point coordinate set of the expanded stacked package image to be segmented;
[0097] Step S206, based on the point coordinate set, using the boundingRect function to calculate the length and area of the minimum circumscribed rectangle of the stacked wrapped image to be segmented after expansion;
[0098] Step S207: when the length of the minimum bounding rectangle is greater than the first preset threshold or the area of the minimum bounding rectangle is greater than the second preset threshold, the expanded stacked package image to be segmented is used as the stacked package image to be segmented.
[0099] Specifically, Opencv related methods are used to detect all packages in the current pseudo-color image. The specific steps are as follows:
[0100] Use GaussianBlur Gaussian filter to reduce the noise of pseudo-color images;
[0101] Use the Canny method to perform edge detection on the denoised image;
[0102] Use a W*H rectangular kernel (pre-set W=3, H=3) to dilate the image after edge detection;
[0103] Use the findContours method to detect the point coordinates of all the parcel contours in the expanded image;
[0104] Use the boundingRect method to get the width and length of the minimum enclosing rectangle of each wrapping outline;
[0105] Calculate the approximate size S of each package based on the width and length of the minimum enclosing rectangle;
[0106] If the approximate size of the package is smaller than the threshold T (for example, T=100 is preset) or the horizontal length is smaller than the threshold H (for example, H=30 is preset), the package is directly determined to be an independent package without further segmentation, and the process ends; otherwise, the subsequent steps are continued.
[0107] The present application reduces the images to be segmented and improves the efficiency of image segmentation by screening the original stacked package images to be segmented.
[0108] In some embodiments, in step S102, based on the preset vertical slice and the image of the stacked package to be segmented, a parcel segmentation candidate point queue is calculated by a fitting algorithm, including:
[0109] Step S301, obtaining a first horizontal coordinate range corresponding to each preset vertical slice and a second horizontal coordinate range corresponding to the stacked package image to be segmented;
[0110] Step S302: Calculate a first fitting curve by a fitting algorithm based on the point coordinate set;
[0111] Step S303: Calculate the slice area corresponding to each preset vertical slice based on the point coordinate set and the first horizontal coordinate range;
[0112] Step S304, calculating a second fitting curve by a fitting algorithm based on the slice region areas corresponding to all preset vertical slices;
[0113] Step S305, calculating the first-order derivative of the first fitting curve within the second horizontal coordinate range to obtain a first fitting derivative;
[0114] Step S306, calculating the first-order derivative of the second fitting curve within the second horizontal coordinate range to obtain a second fitting derivative;
[0115] Step S307: Calculate a candidate point queue for parcel segmentation according to the first horizontal coordinate range, the first fitting derivative, and the second fitting derivative.
[0116] Specifically, in some embodiments, according to the point set result of findContours, a fitting algorithm (such as Fourier series) is used to obtain the wrapped contour fitting curve F(x);
[0117] At every ∆x pixel interval in the horizontal direction (for example, ∆x=10 is preset), the stacked wrapped image to be segmented is sliced vertically;
[0118] Approximate each vertical slice area as a rectangle and calculate its area S1, S2, S3 respectively. Sn (n is the number of vertical slices);
[0119] According to the result set of the vertical slice rectangular areas, a fitting algorithm (such as Fourier series) is used to fit them to obtain a fitting curve S(x);
[0120] Assuming that the horizontal coordinate range of the stacked wrapped image to be segmented is [a, b], the value range of the x-coordinate can be obtained as [a, b-∆x];
[0121] Calculate the first derivative of F(x) in the interval [a, b-∆x] ;
[0122] Calculate the first-order derivative of S(x) in the interval [a, b-∆x] ;
[0123] according to and Calculate the possible segmentation candidate points on the upper edge of the package.
[0124] The present application improves the accuracy of package image segmentation by calculating segmentation candidate points as intermediate data for subsequent package image segmentation.
[0125] In some embodiments, in step S303, based on the point coordinate set and the first horizontal coordinate range, calculating the slice region area corresponding to each preset vertical slice includes:
[0126] Step S401, calculating the difference between the maximum value and the minimum value of the first horizontal coordinate range corresponding to each preset vertical slice; and using the difference as the width of the slice area corresponding to the preset vertical slice;
[0127] Step S402, calculating the sum of the maximum value and the minimum value of the first horizontal coordinate range corresponding to each preset vertical slice; and taking half of the sum as the reference position;
[0128] Step S403, searching for two upper edge contour points located closest to the reference position on the upper edge of the stacked package image to be segmented, wherein the upper edge contour points belong to a point coordinate set;
[0129] Step S404, calculating the first upper edge mean of the horizontal coordinates of the two nearest upper edge contour points, and when the first upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the first upper edge mean is used as the first vertical coordinate average value;
[0130] Step S405: when the first upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and a second upper edge mean of the horizontal coordinates of the three closest upper edge contour points is calculated; when the second upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second upper edge mean is used as the first vertical coordinate average value; when the second upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and a third upper edge mean of the horizontal coordinates of the four closest upper edge contour points is calculated, and so on, until the n-1th upper edge mean of the horizontal coordinates of the n upper edge contour points located closest to the reference position is searched on the upper edge of the stacked package image to be segmented, and the first vertical coordinate average value is obtained, wherein n is the total number of upper edge contour points located closest to the reference position;
[0131] Step S406, searching for two lower edge contour points located closest to the reference position at the lower edge of the stacked package image to be segmented, wherein the lower edge contour points belong to the point coordinate set;
[0132] Step S407, calculating the first lower edge mean of the horizontal coordinates of the two nearest lower edge contour points, and when the first lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the first lower edge mean is used as the second vertical coordinate average value;
[0133] Step S408: when the first lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and a second lower edge mean of the horizontal coordinates of the three closest lower edge contour points is calculated; when the second lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second lower edge mean is used as the second vertical coordinate average value; when the second lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and a third lower edge mean of the horizontal coordinates of the four closest lower edge contour points is calculated, and so on, until the n-1th lower edge mean of the horizontal coordinates of the n lower edge contour points located closest to the reference position is searched at the lower edge of the stacked package image to be segmented, and the second vertical coordinate average value is obtained;
[0134] Step S409, calculating the difference between the second vertical coordinate average value and the second vertical coordinate average value to obtain the height of the slice area corresponding to the preset vertical slice;
[0135] Step S410: Based on the height and width of the slice region, the slice region area corresponding to the preset vertical slice is calculated.
[0136] Specifically, assuming that the horizontal coordinate starting position of a slice area of the stacked wrapped image to be segmented is , the end position is The specific steps for calculating its area are as follows:
[0137] Calculate the width of the rectangle as follows:
[0138]
[0139] in, is the width of the rectangle;
[0140] Calculate the height H of the rectangle as follows:
[0141] Find the horizontal coordinate distance from the top edge of the package The two closest contour points , ;
[0142] like , The mean of the horizontal coordinates of is not If the range is within the range, continue to search for the distance The third nearest contour point ;
[0143] Recursively execute the above steps until you find , , .... The mean of the horizontal coordinates is within the scope;
[0144] calculate , , The vertical coordinate average .
[0145] Find the horizontal coordinate distance from the bottom edge of the package The two closest contour points , ;
[0146] like , The mean of the horizontal coordinates of is not If the The third nearest contour point ;
[0147] Recursively execute the above steps until , , The mean of the horizontal coordinates is within the scope;
[0148] calculate , , The vertical coordinate average .
[0149] calculate and The height H of the rectangle is obtained by taking the difference between .
[0150] Calculate the product of the height H of the rectangle and the width of the rectangle to get the area S of the rectangle.
[0151] In some embodiments, in step S307, calculating the candidate point queue for parcel segmentation according to the first horizontal coordinate range, the first fitting derivative and the second fitting derivative includes:
[0152] Step S501: in the upper edge of the stacked package image to be segmented, obtain the minimum value of the first horizontal coordinate range corresponding to each preset vertical slice, and use it as an initial package segmentation candidate point queue;
[0153] Step S502: Calculate the contour feature candidate compliance flag according to the first fitting derivative and the initial package segmentation candidate point queue using the following formula:
[0154]
[0155] in, is the contour feature candidate compliance mark, The first candidate point in the initial parcel segmentation queue The horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the third preset threshold, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a fourth preset threshold;
[0156] Step S503: Calculate the area feature candidate compliance flag according to the second fitting derivative and the initial package segmentation candidate point queue using the following formula:
[0157]
[0158] in, is the area feature candidate compliance mark, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the fifth preset threshold, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a sixth preset threshold;
[0159] Step S504: When the contour feature candidate meets the flag and the area feature candidate meets the flag, The initial parcel segmentation candidate points are put into the parcel segmentation candidate point queue; and each initial parcel segmentation candidate point in the initial parcel segmentation candidate point queue is calculated in turn to obtain the parcel segmentation candidate point queue.
[0160] This application improves the accuracy of input data, reduces computational overhead, and improves the accuracy of package image segmentation by screening candidate points for package segmentation.
[0161] In some embodiments, in step S103, based on the parcel segmentation candidate point queue, the preset segmentation parameters and the preset ratio, the parcel edge curve is calculated by a fitting algorithm, including:
[0162] Step S601, dividing each parcel segmentation candidate point in the parcel segmentation candidate point queue into regions according to preset segmentation parameters and preset horizontal coordinate offsets, to obtain a first square sample block corresponding to each parcel segmentation candidate point;
[0163] Step S602: Screen the first square sample block according to a preset ratio to obtain a second square sample block;
[0164] Step S603, calculating a third fitting curve by a fitting algorithm based on the coordinates of the center point of the second square sample block;
[0165] Step S604, calculating the third horizontal coordinate range of each candidate parcel segmentation point in the candidate parcel segmentation point queue according to the preset horizontal coordinate offset;
[0166] Step S605, calculating the second-order derivative of the third fitting curve within the third horizontal coordinate range to obtain a third fitting derivative;
[0167] Step S606: Calculate the curvature corresponding to each center point coordinate based on the third fitting derivative;
[0168] Step S607, deleting the maximum value and the minimum value of all curvatures to obtain a curvature set;
[0169] Step S608, calculating the standard deviation of the curvature set to obtain the curvature standard deviation;
[0170] Step S609, calculating the difference between the coordinates of every two adjacent center points, and using it as the spacing of the corresponding second square sample block;
[0171] Step S610, calculating the standard deviation of all spacings to obtain the spacing standard deviation;
[0172] Step S611: In the parcel segmentation candidate point queue, the parcel segmentation candidate points whose curvature standard deviation is less than the seventh preset threshold and whose spacing standard deviation is less than the eighth preset threshold are screened to obtain the screened parcel segmentation candidate points; and the third fitting curve corresponding to the screened parcel segmentation candidate points is used as the parcel edge curve.
[0173] Specifically, traverse the queue of candidate points for parcel segmentation, and for one of the candidate points Do the following;
[0174] The candidate points for segmentation Nearby area The package image is divided into A grid, where Candidate points for segmentation The horizontal coordinate offset on the left side, Candidate points for segmentation The horizontal coordinate offset on the right side, is the number of horizontal lines in the grid, is the number of vertical lines in the grid (for example, u1=10, u2=20, m=20, n=3 are preset).
[0175] Taking the intersection points of these grid lines as the starting point, we can divide them into A square sample block with a side length of d (for example, d=5 is preset);
[0176] Calculate the standard deviation and mean of these sample blocks and sort them in descending order. The specific steps are as follows:
[0177] Use OpenCV's meanStdDev interface to calculate the standard deviation and mean of each sample block;
[0178] First, sort the sample blocks in descending order by standard deviation, and take the blocks with the highest proportion A (for example, A=40%);
[0179] Then sort in descending order by the mean of the blocks with a top proportion of A (for example, A=40%), and take the blocks with a top proportion of B (for example, B=90%);
[0180] Use a fitting algorithm (such as Fourier series) to fit the coordinates of the center points (i.e., sample points) of the blocks with a front proportion of B (e.g., B=90%) to obtain a fitting curve E(x);
[0181] Determine whether the curve E(x) is the edge curve of the package stack. The specific method is as follows:
[0182] Calculate E(x) in the interval The second derivative in ;
[0183] according to Get the curvature of each sample point;
[0184] Remove the maximum and minimum values in the curvature;
[0185] Calculate the standard deviation Kstd of the curvature after screening;
[0186] Calculate the distance Di between each sample block and the next sample block (Di is the difference in coordinates of the center points between each sample block and the next sample block);
[0187] Calculate the standard deviation Dstd of the intervals Di, Di...D;
[0188] If the value of Kstd is less than the threshold value Kt (for example, Kt=10 is preset) and the value of Dstd is less than the threshold value Dt (for example, Dt=1 is preset), then it is considered that The edge curve of a point is a plausible stacked wrapping segmentation curve, and the stacked wrapping edge is approximated by the curve E(x).
[0189] In some embodiments, in step S104, segmenting the stacked package image to be segmented according to the package edge curve to obtain a segmented image of the stacked package includes:
[0190] Step S701, segmenting the point coordinate set based on the wrapping edge curve to obtain a segmented set;
[0191] Step S702: Add the center point coordinates of all the second square sample blocks corresponding to each wrapping edge curve to the corresponding segmented set to obtain a stacked wrapped segmentation image.
[0192] Specifically, all contour points of the package are divided into sets A and B using the curve E(x);
[0193] Add all sample points on the E(x) curve to sets A and B respectively again;
[0194] The points in sets A and B are the contour point sets of the two new packages after cutting.
[0195] While ensuring uniqueness, this application improves the overall efficiency of video package segmentation and tracking from 100ms to about 20ms, greatly improving recognition efficiency.
[0196] Specifically, refer to Figure 2 In order to facilitate understanding by those skilled in the art, a set of optimal embodiments are provided below:
[0197] 1. Data acquisition:
[0198] Obtain the original stacked package image to be segmented;
[0199] Based on the original stacked package image to be segmented, denoising is performed by using a Gaussian filtering method to obtain a denoised stacked package image to be segmented;
[0200] Based on the denoised image of the stacked package to be segmented, edge detection is performed using the Canny edge detection algorithm to obtain the image of the stacked package to be segmented after edge detection;
[0201] Based on a preset rectangular kernel, the stacked package image to be segmented after edge detection is expanded to obtain the expanded stacked package image to be segmented;
[0202] The findContours method is used to perform contour detection on the expanded stacked package image to be segmented, and a point coordinate set of the expanded stacked package image to be segmented is obtained;
[0203] Based on the point coordinate set, the boundingRect function is used to calculate the length and area of the minimum bounding rectangle of the stacked wrapped image to be segmented after expansion;
[0204] When the length of the minimum circumscribed rectangle is greater than the first preset threshold or the area of the minimum circumscribed rectangle is greater than the second preset threshold, the expanded stacked package image to be segmented is used as the stacked package image to be segmented.
[0205] 2. Package segmentation candidate point queue calculation:
[0206] Based on the preset vertical slices and the stacked package images to be segmented, the candidate point queue for package segmentation is calculated by the fitting algorithm, specifically:
[0207] Obtaining a first horizontal coordinate range corresponding to each preset vertical slice and a second horizontal coordinate range corresponding to the stacked package image to be segmented;
[0208] Based on the set of point coordinates, calculating a first fitting curve by a fitting algorithm;
[0209] Based on the point coordinate set and the first horizontal coordinate range, the slice area corresponding to each preset vertical slice is calculated, specifically:
[0210] Calculate the difference between the maximum value and the minimum value of the first horizontal axis range corresponding to each preset vertical slice; and use the difference as the width of the slice area corresponding to the preset vertical slice;
[0211] Calculate the sum of the maximum value and the minimum value of the first horizontal coordinate range corresponding to each preset vertical slice; and use half of the sum as the reference position;
[0212] Find two upper edge contour points located closest to the reference position on the upper edge of the stacked package image to be segmented, where the upper edge contour points belong to the point coordinate set;
[0213] Calculate the first upper edge mean of the horizontal coordinates of the two nearest upper edge contour points, and when the first upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first upper edge mean as the first vertical coordinate average value;
[0214] When the first upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and the second upper edge mean of the horizontal coordinates of the three closest upper edge contour points is calculated; when the second upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second upper edge mean is used as the first vertical coordinate average value; when the second upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and the third upper edge mean of the horizontal coordinates of the four closest upper edge contour points is calculated, and so on, until the n-1th upper edge mean of the horizontal coordinates of the n upper edge contour points located closest to the reference position is searched on the upper edge of the stacked package image to be segmented, and the first vertical coordinate average value is obtained, wherein n is the total number of upper edge contour points located closest to the reference position;
[0215] Find two lower edge contour points located closest to the reference position at the lower edge of the stacked package image to be segmented, where the lower edge contour points belong to the point coordinate set;
[0216] Calculate the first lower edge mean of the horizontal coordinates of the two nearest lower edge contour points, and when the first lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first lower edge mean as the second vertical coordinate average value;
[0217] When the first lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the second lower edge mean of the horizontal coordinates of the three closest lower edge contour points is calculated; when the second lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second lower edge mean is used as the second vertical coordinate average value; when the second lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the third lower edge mean of the horizontal coordinates of the four closest lower edge contour points is calculated, and so on, until the n-1th lower edge mean of the horizontal coordinates of the n lower edge contour points located closest to the reference position is found at the lower edge of the stacked package image to be segmented within the first horizontal coordinate range, and the second vertical coordinate average value is obtained;
[0218] Calculate the difference between the second vertical coordinate average value and the second vertical coordinate average value to obtain the height of the slice area corresponding to the preset vertical slice;
[0219] Based on the height and width of the slice area, the slice area corresponding to the preset vertical slice is calculated.
[0220] Based on the slice area areas corresponding to all the preset vertical slices, a second fitting curve is calculated by a fitting algorithm;
[0221] Calculate the first-order derivative of the first fitting curve within the second abscissa range to obtain a first fitting derivative;
[0222] Calculate the first-order derivative of the second fitting curve within the second abscissa range to obtain a second fitting derivative;
[0223] The candidate point queue for parcel segmentation is calculated according to the first horizontal coordinate range, the first fitting derivative and the second fitting derivative, specifically:
[0224] In the upper edge of the stacked package image to be segmented, the minimum value of the first horizontal coordinate range corresponding to each preset vertical slice is obtained, and the minimum value is used as the initial package segmentation candidate point queue;
[0225] According to the first fitting derivative and the initial parcel segmentation candidate point queue, the contour feature candidate compliance flag is calculated by the following formula:
[0226]
[0227] in, is the contour feature candidate compliance mark, The first candidate point in the initial parcel segmentation queue The horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the third preset threshold, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a fourth preset threshold;
[0228] According to the second fitting derivative and the initial parcel segmentation candidate point queue, the area feature candidate compliance flag is calculated by the following formula:
[0229]
[0230] in, is the area feature candidate compliance mark, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the fifth preset threshold, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a sixth preset threshold;
[0231] When the contour feature candidate meets the flag and the area feature candidate meets the flag, the first The initial parcel segmentation candidate points are put into the parcel segmentation candidate point queue; and each initial parcel segmentation candidate point in the initial parcel segmentation candidate point queue is calculated in turn to obtain the parcel segmentation candidate point queue.
[0232] 3. Wrap edge curve calculation
[0233] Performing region division on each parcel segmentation candidate point in the parcel segmentation candidate point queue according to preset segmentation parameters and preset horizontal coordinate offsets to obtain a first square sample block corresponding to each parcel segmentation candidate point;
[0234] Screening the first square sample block according to a preset ratio to obtain a second square sample block;
[0235] Based on the coordinates of the center point of the second square sample block, a third fitting curve is calculated by a fitting algorithm;
[0236] Calculate the third horizontal coordinate range of each parcel segmentation candidate point in the parcel segmentation candidate point queue according to the preset horizontal coordinate offset;
[0237] Calculating the second-order derivative of the third fitting curve within the third horizontal coordinate range to obtain a third fitting derivative;
[0238] Based on the third fitting derivative, the curvature corresponding to the coordinates of each center point is calculated;
[0239] Delete the maximum and minimum values of all curvatures to obtain the curvature set;
[0240] Calculate the standard deviation of the curvature set to obtain the curvature standard deviation;
[0241] Calculate the difference between the coordinates of every two adjacent center points and use it as the spacing of the corresponding second square sample block;
[0242] Calculate the standard deviation of all intervals to get the interval standard deviation;
[0243] In the parcel segmentation candidate point queue, the parcel segmentation candidate points whose curvature standard deviation is less than the seventh preset threshold and whose spacing standard deviation is less than the eighth preset threshold are screened to obtain the screened parcel segmentation candidate points; and the third fitting curve corresponding to the screened parcel segmentation candidate points is used as the parcel edge curve.
[0244] 4. Segmentation of stacked packages:
[0245] The point coordinate set is segmented based on the wrapped edge curve to obtain a segmented set;
[0246] The center point coordinates of all the second square sample blocks corresponding to each wrapped edge curve are added to the corresponding segmented set to obtain a stacked wrapped segmentation image.
[0247] In addition, refer to Figure 3 One embodiment of the present invention provides a stacked package segmentation system, including a data acquisition module 1100, a package segmentation candidate point queue calculation module 1200, a package edge curve calculation module 1300 and an image segmentation module 1400, wherein:
[0248] The data acquisition module 1100 is used to acquire the image of the stacked packages to be segmented;
[0249] The parcel segmentation candidate point queue calculation module 1200 is used to calculate the parcel segmentation candidate point queue by a fitting algorithm based on the preset vertical slice and the stacked parcel image to be segmented;
[0250] The package edge curve calculation module 1300 is used to calculate the package edge curve through a fitting algorithm based on the package segmentation candidate point queue, the preset segmentation parameters and the preset ratio;
[0251] The image segmentation module 1400 is used to segment the stacked package image to be segmented according to the package edge curve to obtain a segmented image of the stacked package.
[0252] The system obtains an image of stacked packages to be segmented; calculates a candidate point queue for package segmentation through a fitting algorithm based on preset vertical slices and the image of the stacked packages to be segmented; calculates a package edge curve through a fitting algorithm based on the candidate point queue for package segmentation, preset segmentation parameters and preset ratios; segments the image of the stacked packages to be segmented according to the package edge curve to obtain a segmented image of the stacked packages. The method improves the segmentation accuracy of continuously stacked packages by fitting and calculating a nonlinear model.
[0253] It should be noted that the present system embodiment and the above-mentioned method embodiment are based on the same inventive concept, so the relevant contents of the above-mentioned method embodiment are also applicable to the present system embodiment and will not be repeated here.
[0254] Figure 4 A schematic diagram of the hardware structure for segmenting stacked packages provided in an embodiment of the present application is shown.
[0255] The stacked package separation device may include a processor 401 and a memory 402 storing computer program instructions.
[0256] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0257] The memory 402 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 402 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 402 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 402 is a non-volatile solid-state memory.
[0258] In some embodiments, the memory 402 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0259] The processor 401 implements any one of the stacked package segmentation methods in the above embodiments by reading and executing computer program instructions stored in the memory 402 .
[0260] In one example, the stacked package separation device may further include a communication interface 403 and a bus 410. Figure 4 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.
[0261] The communication interface 403 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0262] Bus 410 includes hardware, software or both, and couples the components of the stacked and wrapped segmented devices to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, bus 410 may include one or more buses. Although the present application embodiment describes and illustrates a specific bus, the present application considers any suitable bus or interconnect.
[0263] The stacked package segmentation device can execute the stacked package segmentation method in the embodiment of the present application based on the three-dimensional design model, thereby realizing the combination of Figure 1 and Figure 3 A method and system for segmenting stacked packages is described.
[0264] In addition, in combination with the stacked package segmentation method in the above embodiment, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any stacked package segmentation method in the above embodiment is implemented.
[0265] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0266] The functional blocks shown in the structural block diagram described above can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0267] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0268] Aspects of the present disclosure are described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0269] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A method for dividing a stacked package, characterized in that: The stacked package segmentation method comprises: Acquire an image of stacked packages to be segmented; wherein the image of stacked packages to be segmented refers to an image obtained by collecting stacked packages on the security inspection machine, and the stacked packages are at least two packages stacked in the transmission direction of the security inspection machine, and the transmission direction is used as the horizontal axis. Acquire the image of stacked packages to be segmented, including: Acquire an original stacked package image to be segmented; based on the original stacked package image to be segmented, perform noise reduction by a Gaussian filtering method to obtain a denoised stacked package image to be segmented; based on the denoised stacked package image to be segmented, perform edge detection by a Canny edge detection algorithm to obtain a stacked package image to be segmented after edge detection; based on a preset rectangular kernel of W*H, dilate the stacked package image to be segmented after edge detection to obtain an expanded stacked package image, where W is width and H is height; perform contour detection on the expanded stacked package image by a findContours method to obtain a point coordinate set of the expanded stacked package image; based on the point coordinate set, calculate the length and area of the minimum circumscribed rectangle of the expanded stacked package image by a boundingRect function; when the length of the minimum circumscribed rectangle is greater than a first preset threshold or the area of the minimum circumscribed rectangle is greater than a second preset threshold, use the expanded stacked package image as the stacked package image to be segmented; Based on the preset vertical slice and the stacked package image to be segmented, a parcel segmentation candidate point queue is calculated by a fitting algorithm; Based on the preset vertical slice and the stacked package image to be segmented, a parcel segmentation candidate point queue is calculated by a fitting algorithm, including: Obtain a first horizontal coordinate range corresponding to each of the preset vertical slices and a second horizontal coordinate range corresponding to the stacked package image to be segmented; calculate a first fitting curve by a fitting algorithm based on the point coordinate set; calculate a slice area corresponding to each of the preset vertical slices based on the point coordinate set and the first horizontal coordinate range; calculate a second fitting curve by a fitting algorithm based on the slice area corresponding to all preset vertical slices; calculate a first-order derivative of the first fitting curve within the second horizontal coordinate range to obtain a first fitting derivative; calculate a first-order derivative of the second fitting curve within the second horizontal coordinate range to obtain a second fitting derivative; calculate the package segmentation candidate point queue according to the first horizontal coordinate range, the first fitting derivative, and the second fitting derivative; The step of calculating the slice region area corresponding to each of the preset vertical slices based on the point coordinate set and the first horizontal coordinate range includes: Calculating the difference between the maximum value and the minimum value of the first horizontal axis range corresponding to each of the preset vertical slices; and using the difference as the width of the slice area corresponding to the preset vertical slice; Calculate the sum of the maximum value and the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices; and use the horizontal coordinate corresponding to half of the sum as a reference position; Searching for two upper edge contour points located closest to the reference position on the upper edge of the stacked package image to be segmented, wherein the upper edge contour points belong to the point coordinate set; Calculate the first upper edge mean of the horizontal coordinates of the two nearest upper edge contour points. When the first upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first upper edge mean as the first vertical coordinate average value. The first upper edge mean is the sum of the corresponding values of the horizontal coordinates of the two nearest upper edge contour points, divided by 2. When the first upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and the second upper edge mean of the horizontal coordinates of the three closest upper edge contour points is calculated; when the second upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second upper edge mean is used as the first vertical coordinate average value; when the second upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four upper edges located closest to the reference position are searched on the upper edge of the stacked package image to be segmented. contour points, and calculate the third upper edge mean of the horizontal coordinates of the four upper edge contour points closest to the reference position, and so on, until the n-1th upper edge mean of the horizontal coordinates of the n upper edge contour points closest to the reference position is found on the upper edge of the stacked package image to be segmented, and is within the first horizontal coordinate range, to obtain the first vertical coordinate average value, wherein n is the total number of upper edge contour points closest to the reference position; the second upper edge mean is the sum of the corresponding values of the horizontal coordinates of the three upper edge contour points closest to the reference position, and then divided by 3 to obtain the value; the third upper edge mean is the sum of the corresponding values of the horizontal coordinates of the four upper edge contour points closest to the reference position, and then divided by 4 to obtain the value; Searching for two lower edge contour points located closest to a reference position at the lower edge of the stacked package image to be segmented, wherein the lower edge contour points belong to the point coordinate set; Calculate the first lower edge mean of the horizontal coordinates of the two nearest lower edge contour points. When the first lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first lower edge mean as the second vertical coordinate average value. The first lower edge mean is the sum of the corresponding values of the horizontal coordinates of the two nearest lower edge contour points, divided by 2. When the first lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the second lower edge mean of the horizontal coordinates of the three closest lower edge contour points is calculated; when the second lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second lower edge mean is used as the second vertical coordinate average value; when the second lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the third lower edge mean of the horizontal coordinates of the four closest lower edge contour points is calculated, and so on, until the n-1th lower edge mean of the horizontal coordinates of the n lower edge contour points located closest to the reference position is found at the lower edge of the stacked package image to be segmented, and the second vertical coordinate average value is obtained; the second lower edge mean is the sum of the corresponding values of the horizontal coordinates of the three closest lower edge contour points, and the value is obtained by dividing by 3; the third lower edge mean is the sum of the corresponding values of the horizontal coordinates of the four closest lower edge contour points, and the value is obtained by dividing by 4; Calculate the difference between the first vertical coordinate average value and the second vertical coordinate average value to obtain the height of the slice area corresponding to the preset vertical slice; Based on the height of the slice area and the width of the slice area, an area of the slice area corresponding to the preset vertical slice is calculated; Wherein, based on the slice region areas corresponding to all preset vertical slices, calculating the second fitting curve by a fitting algorithm comprises: Based on the area of each slice region corresponding to each preset vertical slice, a second fitting curve corresponding to the slice region area is calculated by Fourier series; The calculating the package segmentation candidate point queue according to the first horizontal coordinate range, the first fitting derivative and the second fitting derivative includes: In the upper edge of the stacked package image to be segmented, obtaining the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices, and using it as an initial package segmentation candidate point queue; The contour feature candidate compliance flag is calculated according to the first fitting derivative and the initial parcel segmentation candidate point queue by the following formula: ; in, is the contour feature candidate compliance mark, The first candidate point in the initial parcel segmentation queue The horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the third preset threshold, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a fourth preset threshold; The area feature candidate compliance flag is calculated according to the second fitting derivative and the initial package segmentation candidate point queue using the following formula: ; in, is the area feature candidate compliance mark, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the fifth preset threshold, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a sixth preset threshold; When the contour feature candidate meets the flag and the area feature candidate meets the flag is true, the first put the initial parcel segmentation candidate points into the parcel segmentation candidate point queue; and calculate each initial parcel segmentation candidate point in the initial parcel segmentation candidate point queue in turn to obtain the parcel segmentation candidate point queue; Based on the parcel segmentation candidate point queue, the preset segmentation parameters and the preset ratio, the parcel edge curve is calculated by a fitting algorithm; based on the parcel segmentation candidate point queue, the preset segmentation parameters and the preset ratio, the parcel edge curve is calculated by a fitting algorithm, including: Each package segmentation candidate point in the package segmentation candidate point queue is divided into regions according to the preset segmentation parameters and the preset horizontal coordinate offset, and a first square sample block corresponding to each package segmentation candidate point is obtained, and the preset horizontal coordinate offset is an offset set for each package segmentation candidate point in two directions of the horizontal coordinate; the first square sample blocks corresponding to each package segmentation candidate point are sorted, and the first square sample blocks are screened according to the preset ratio, and a second square sample block sorted within the preset ratio is obtained from the first square sample block corresponding to each package segmentation candidate point, and based on the center point coordinates of the second square sample block, a third fitting curve of the center point coordinates of the second square sample block is calculated by a fitting algorithm; the package segmentation candidate point queue is calculated according to the preset horizontal coordinate offset. The third horizontal coordinate range of each package segmentation candidate point is calculated; the second-order derivative of the third fitting curve within the third horizontal coordinate range is calculated to obtain the third fitting derivative; based on the third fitting derivative, the curvature corresponding to each center point coordinate is calculated; the maximum value and the minimum value of all the curvatures are deleted to obtain a curvature set; the standard deviation of the curvature set is calculated to obtain a curvature standard deviation; the difference between the coordinates of each two adjacent center points is calculated, and it is used as the spacing of the corresponding second square sample block; the standard deviation of all the spacings is calculated to obtain the spacing standard deviation; in the package segmentation candidate point queue, the package segmentation candidate points whose curvature standard deviation is less than the seventh preset threshold and whose spacing standard deviation is less than the eighth preset threshold are screened to obtain the screened package segmentation candidate points; and the third fitting curve corresponding to the screened package segmentation candidate points is used as the package edge curve; The stacked package image to be segmented is segmented according to the package edge curve to obtain a segmented image of the stacked package.
2. A method for dividing stacked packages according to claim 1, characterized in that: The stacked package image to be segmented is segmented according to the package edge curve to obtain a segmented image of the stacked package, including: Segmenting the point coordinate set based on the wrapped edge curve to obtain a segmented set; The center point coordinates of all the second square sample blocks corresponding to each of the wrapping edge curves are added to the corresponding segmented set to obtain the segmented image of the stacked wrapping.
3. A stacked package segmentation system, characterized in that: The stacked package segmentation system comprises: A data acquisition module, used for acquiring images of stacked packages to be segmented; The stacked package image to be segmented refers to an image obtained by collecting the stacked packages on the security inspection machine. The stacked packages are at least two packages stacked in the transmission direction of the security inspection machine, and the transmission direction is used as the horizontal coordinate. The step of obtaining the stacked package image to be segmented includes: Obtain the original stacked package image to be segmented; Based on the original stacked package image to be segmented, denoising is performed by a Gaussian filtering method to obtain a denoised stacked package image to be segmented; Based on the denoised stacked package image to be segmented, edge detection is performed using a Canny edge detection algorithm to obtain an edge-detected stacked package image to be segmented; Based on a preset rectangular kernel, the stacked package image to be segmented after edge detection is expanded to obtain the expanded stacked package image to be segmented; Performing contour detection on the expanded stacked package image to be segmented by using a findContours method to obtain a set of point coordinates of the expanded stacked package image to be segmented; Based on the point coordinate set, the length and area of the minimum circumscribed rectangle of the expanded stacked wrapped image to be segmented are calculated by a boundingRect function; When the length of the minimum circumscribed rectangle is greater than a first preset threshold or the area of the minimum circumscribed rectangle is greater than a second preset threshold, the expanded stacked package image to be segmented is used as the stacked package image to be segmented; A parcel segmentation candidate point queue calculation module, used to calculate the parcel segmentation candidate point queue through a fitting algorithm based on a preset vertical slice and a stacked parcel image to be segmented; Based on the preset vertical slice and the stacked package image to be segmented, a parcel segmentation candidate point queue is calculated by a fitting algorithm, including: Obtain a first horizontal coordinate range corresponding to each of the preset vertical slices and a second horizontal coordinate range corresponding to the stacked package image to be segmented; calculate a first fitting curve by a fitting algorithm based on the point coordinate set; calculate a slice area corresponding to each of the preset vertical slices based on the point coordinate set and the first horizontal coordinate range; calculate a second fitting curve by a fitting algorithm based on the slice area corresponding to all preset vertical slices; calculate a first-order derivative of the first fitting curve within the second horizontal coordinate range to obtain a first fitting derivative; calculate a first-order derivative of the second fitting curve within the second horizontal coordinate range to obtain a second fitting derivative; calculate the package segmentation candidate point queue according to the first horizontal coordinate range, the first fitting derivative, and the second fitting derivative; The step of calculating the slice region area corresponding to each of the preset vertical slices based on the point coordinate set and the first horizontal coordinate range includes: Calculating the difference between the maximum value and the minimum value of the first horizontal axis range corresponding to each of the preset vertical slices; and using the difference as the width of the slice area corresponding to the preset vertical slice; Calculate the sum of the maximum value and the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices; and use the horizontal coordinate corresponding to half of the sum as a reference position; Searching for two upper edge contour points located closest to the reference position on the upper edge of the stacked package image to be segmented, wherein the upper edge contour points belong to the point coordinate set; Calculate the first upper edge mean of the horizontal coordinates of the two nearest upper edge contour points. When the first upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first upper edge mean as the first vertical coordinate average value. The first upper edge mean is the sum of the corresponding values of the horizontal coordinates of the two nearest upper edge contour points, divided by 2. When the first upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three upper edge contour points located closest to the reference position are searched on the upper edge of the stacked package image to be segmented, and the second upper edge mean of the horizontal coordinates of the three closest upper edge contour points is calculated; when the second upper edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second upper edge mean is used as the first vertical coordinate average value; when the second upper edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four upper edges located closest to the reference position are searched on the upper edge of the stacked package image to be segmented. contour points, and calculate the third upper edge mean of the horizontal coordinates of the four upper edge contour points closest to the reference position, and so on, until the n-1th upper edge mean of the horizontal coordinates of the n upper edge contour points closest to the reference position is found on the upper edge of the stacked package image to be segmented, and is within the first horizontal coordinate range, to obtain the first vertical coordinate average value, wherein n is the total number of upper edge contour points closest to the reference position; the second upper edge mean is the sum of the corresponding values of the horizontal coordinates of the three upper edge contour points closest to the reference position, and then divided by 3 to obtain the value; the third upper edge mean is the sum of the corresponding values of the horizontal coordinates of the four upper edge contour points closest to the reference position, and then divided by 4 to obtain the value; Searching for two lower edge contour points located closest to a reference position at the lower edge of the stacked package image to be segmented, wherein the lower edge contour points belong to the point coordinate set; Calculate the first lower edge mean of the horizontal coordinates of the two nearest lower edge contour points. When the first lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, use the first lower edge mean as the second vertical coordinate average value. The first lower edge mean is the sum of the corresponding values of the horizontal coordinates of the two nearest lower edge contour points, divided by 2. When the first lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, three lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the second lower edge mean of the horizontal coordinates of the three closest lower edge contour points is calculated; when the second lower edge mean is within the first horizontal coordinate range corresponding to the preset vertical slice, the second lower edge mean is used as the second vertical coordinate average value; when the second lower edge mean is not within the first horizontal coordinate range corresponding to the preset vertical slice, four lower edge contour points located closest to the reference position are searched at the lower edge of the stacked package image to be segmented, and the third lower edge mean of the horizontal coordinates of the four closest lower edge contour points is calculated, and so on, until the n-1th lower edge mean of the horizontal coordinates of the n lower edge contour points located closest to the reference position is found at the lower edge of the stacked package image to be segmented, and the second vertical coordinate average value is obtained; the second lower edge mean is the sum of the corresponding values of the horizontal coordinates of the three closest lower edge contour points, and the value is obtained by dividing by 3; the third lower edge mean is the sum of the corresponding values of the horizontal coordinates of the four closest lower edge contour points, and the value is obtained by dividing by 4; Calculate the difference between the first vertical coordinate average value and the second vertical coordinate average value to obtain the height of the slice area corresponding to the preset vertical slice; Based on the height of the slice area and the width of the slice area, an area of the slice area corresponding to the preset vertical slice is calculated; Wherein, based on the slice region areas corresponding to all preset vertical slices, calculating the second fitting curve by a fitting algorithm comprises: Based on the area of each slice region corresponding to each preset vertical slice, a second fitting curve corresponding to the slice region area is calculated by Fourier series; The calculating the package segmentation candidate point queue according to the first horizontal coordinate range, the first fitting derivative and the second fitting derivative includes: In the upper edge of the stacked package image to be segmented, obtaining the minimum value of the first horizontal coordinate range corresponding to each of the preset vertical slices, and using it as an initial package segmentation candidate point queue; The contour feature candidate compliance flag is calculated according to the first fitting derivative and the initial parcel segmentation candidate point queue by the following formula: ; in, is the contour feature candidate compliance mark, The first candidate point in the initial parcel segmentation queue The horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the third preset threshold, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The first fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a fourth preset threshold; The area feature candidate compliance flag is calculated according to the second fitting derivative and the initial package segmentation candidate point queue using the following formula: ; in, is the area feature candidate compliance mark, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is the fifth preset threshold, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, For the The second fitting derivative value corresponding to the horizontal coordinate value of the initial parcel segmentation candidate point, is a sixth preset threshold; When the contour feature candidate meets the flag and the area feature candidate meets the flag is true, the first put the initial parcel segmentation candidate points into the parcel segmentation candidate point queue; and calculate each initial parcel segmentation candidate point in the initial parcel segmentation candidate point queue in turn to obtain the parcel segmentation candidate point queue; A package edge curve calculation module, used to calculate the package edge curve through a fitting algorithm based on the package segmentation candidate point queue, preset segmentation parameters and preset ratio; The method of calculating the parcel edge curve by a fitting algorithm based on the parcel segmentation candidate point queue, the preset segmentation parameters and the preset ratio includes: Each package segmentation candidate point in the package segmentation candidate point queue is divided into regions according to the preset segmentation parameters and the preset horizontal coordinate offset, and a first square sample block corresponding to each package segmentation candidate point is obtained, and the preset horizontal coordinate offset is an offset set for each package segmentation candidate point in two directions of the horizontal coordinate; the first square sample blocks corresponding to each package segmentation candidate point are sorted, and the first square sample blocks are screened according to the preset ratio, and a second square sample block sorted within the preset ratio is obtained from the first square sample block corresponding to each package segmentation candidate point, and based on the center point coordinates of the second square sample block, a third fitting curve of the center point coordinates of the second square sample block is calculated by a fitting algorithm; the package segmentation candidate point queue is calculated according to the preset horizontal coordinate offset. The third horizontal coordinate range of each package segmentation candidate point is calculated; the second-order derivative of the third fitting curve within the third horizontal coordinate range is calculated to obtain the third fitting derivative; based on the third fitting derivative, the curvature corresponding to each center point coordinate is calculated; the maximum value and the minimum value of all the curvatures are deleted to obtain a curvature set; the standard deviation of the curvature set is calculated to obtain a curvature standard deviation; the difference between the coordinates of each two adjacent center points is calculated, and it is used as the spacing of the corresponding second square sample block; the standard deviation of all the spacings is calculated to obtain the spacing standard deviation; in the package segmentation candidate point queue, the package segmentation candidate points whose curvature standard deviation is less than the seventh preset threshold and whose spacing standard deviation is less than the eighth preset threshold are screened to obtain the screened package segmentation candidate points; and the third fitting curve corresponding to the screened package segmentation candidate points is used as the package edge curve; The image segmentation module is used to segment the stacked package image to be segmented according to the package edge curve to obtain a segmented image of the stacked package.
4. A stacked package separation device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute a method for dividing stacked packages as described in any one of claims 1 to 2.
5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method for dividing stacked packages as described in any one of claims 1 to 2.
Citation Information
Patent Citations
Parcel stacking detection method and device based on neural network
CN113762190A
Stacked object identification method, device and equipment and computer storage medium
CN116171463A
Parcel stacking state visual detection method for stack separation
CN117612061A
Express separation method, device and equipment, storage medium and product
CN118212392A
Method and device for determining number of stacked articles, electronic equipment and medium
CN118351095A
