A method, apparatus, device, and storage medium for marking packages

Abstract

This invention discloses a method, apparatus, device, and storage medium for marking packages. The method includes: determining the location information of packages and / or waybills in an actual scene image captured by a camera, and the correspondence between packages and / or waybills, based on a preset instance segmentation algorithm; identifying a target package when a package is detected in the actual scene image, and determining whether a waybill corresponding to the target package exists; if so, determining the area excluding the waybill within the image region where the target package is located as the target area, and determining the optimal position for marking the target package within the target area; determining the actual position for marking the target package in the actual scene based on the optimal position and the mapping relationship between the actual scene image and the actual scene monitoring range; and controlling relevant equipment to mark the package based on the actual position, thereby improving the efficiency and accuracy of package marking.

Description

Technical Field

[0001] This invention relates to the field of computers, and more particularly to a method, apparatus, device, and storage medium for marking packages. Background Technology

[0002] The technology of automating the labeling of packages has appeared in many application scenarios. However, with the continuous development of the express delivery industry, manually labeling each package is extremely labor-intensive.

[0003] Therefore, how to automatically determine the optimal location for labeling packages and improve the efficiency and accuracy of package labeling is an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for marking packages, which can automatically determine the optimal position for marking packages, thereby improving the efficiency and accuracy of package marking.

[0005] According to one aspect of the present invention, a method for marking packages is provided, comprising:

[0006] Based on a preset instance segmentation algorithm, the location information of packages and / or express delivery slips in the actual scene images captured by the camera is determined, as well as the corresponding situation of packages and / or express delivery slips.

[0007] Based on the correspondence between the package and / or the waybill, when a package is detected in the actual scene image, the target package is identified, and it is determined whether there is a waybill corresponding to the target package.

[0008] If so, then in the image area where the target package is located, determine the area other than the waybill as the target area, and determine the optimal position for marking the target package in the target area;

[0009] Based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined, and the relevant equipment is controlled to mark the package according to the actual location.

[0010] According to another aspect of the present invention, a package marking device is provided, comprising:

[0011] The first determining module is used to determine the location information of packages and / or express delivery slips in the actual scene images captured by the camera, as well as the corresponding situation of packages and / or express delivery slips, based on a preset instance segmentation algorithm.

[0012] The second determining module is used to determine the target package and determine whether there is a corresponding express delivery slip when a package is detected in the actual scene image, based on the correspondence between the package and / or the express delivery slip.

[0013] The third determining module is used to, if so, determine the area outside the waybill in the image area where the target package is located as the target area, and determine the optimal position for marking the target package in the target area;

[0014] The marking module is used to determine the actual location for marking the target package in the actual scene based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, and to control the relevant equipment to mark the package based on the actual location.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the package marking method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the package marking method according to any embodiment of the present invention.

[0020] The technical solution of this invention, based on a preset instance segmentation algorithm, determines the location information of packages and / or waybills in the actual scene image captured by the camera, as well as the correspondence between packages and / or waybills. Based on the correspondence, when a package is detected in the actual scene image, a target package is identified, and it is determined whether a corresponding waybill exists. If so, the area excluding the waybill within the image region where the target package is located is identified as the target area. The optimal position for marking the target package is determined within the target area. Based on the optimal position and the mapping relationship between the actual scene image captured by the camera and the camera's monitoring range of the actual scene, the actual position for marking the target package in the actual scene is determined. Based on the actual position, relevant equipment is controlled to mark the package. This method automates the marking of packages, improving marking efficiency. Furthermore, it ensures that the marked labels do not obscure the waybill information or overlap with the package, improving the accuracy of package marking.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a package marking method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of a package marking method provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a structural block diagram of a package marking device provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," "target," "candidate," "alternative," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It should be noted that with the continuous development of the express delivery industry, it is necessary to distinguish between packages that have passed X-ray inspection and those that have not by marking them, so as to avoid confusion. In related technologies, marking is often done manually, which is extremely labor-intensive and limits the efficiency of package marking. Based on the above problems, the technical solution provided by this invention can improve the efficiency and accuracy of marking while realizing automated package marking, freeing up manpower. The specific implementation method will be described in detail in subsequent embodiments.

[0030] Example 1

[0031] Figure 1 This is a flowchart of a package marking method provided in Embodiment 1 of the present invention. This embodiment is applicable to marking packages placed in a real-world scenario. The method can be executed by a package marking device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method for marking the package includes:

[0032] S101. Based on a preset instance segmentation algorithm, determine the location information of packages and / or express delivery slips in the actual scene images captured by the camera, as well as the corresponding information of packages and / or express delivery slips.

[0033] Instance segmentation refers to algorithms used for object detection in images to obtain mask information for each target object. An example of an instance segmentation algorithm is the Mask R-CNN algorithm. The actual scene image refers to the image acquired by a camera capturing the actual scene where packages and / or delivery slips are placed. The positional information of the packages and / or delivery slips refers to the position of the center point of the packages and / or delivery slips in the actual scene image. This positional information can be represented, for example, by rectangular coordinates, i.e., by the distances from the center position of the packages and / or delivery slips to the four sides of the actual scene image.

[0034] It should be noted that in the actual scene image, what is visible may only be the package, or only the waybill (for example, the waybill is so large that it covers the entire package), or both the package and the waybill may be present. In the case where both the package and the waybill are present, each package may have a corresponding waybill, or some packages may not have a waybill attached. This invention does not impose any restrictions on this.

[0035] The correspondence between packages and / or waybills can be as follows: packages only, waybills only, or both. For example, if each package has a corresponding waybill, and the actual scene image includes package 1, package 2, waybill 1, and waybill 2, then the correspondence between packages and / or waybills can be that package 1 corresponds to waybill 1, and package 2 corresponds to waybill 2. If some packages do not have a corresponding waybill, and the actual scene image includes package 1, package 2, and waybill 1, then the correspondence between packages and / or waybills can be that package 1 corresponds to waybill 1, and package 2 has no corresponding waybill.

[0036] Optionally, based on a preset instance segmentation algorithm, the location information of packages and / or express delivery slips in the actual scene images captured by the camera, as well as the correspondence of packages and / or express delivery slips, are determined. This includes: segmenting the actual scene images captured by the camera based on the preset instance segmentation algorithm to determine the mask information of packages and / or express delivery slips in the actual scene images, and determining the location information of packages and / or express delivery slips in the actual scene images based on the mask information; dividing the packages and / or express delivery slips in the actual scene images into at least one group based on the location information, and determining the correspondence of packages and / or express delivery slips in the actual scene images based on the grouping results.

[0037] Optionally, after segmenting the actual scene image captured by the camera, the category of each target object and its corresponding mask information can be determined based on the processing results. The target objects include packages and / or express delivery slips, that is, the mask information of packages and / or express delivery slips in the actual scene image can be determined.

[0038] It should be noted that if the actual scene image contains Package 1, Package 2, and Express Delivery Slip 1, then after segmentation, four sets of mask information can be obtained. Two sets of mask information correspond to the category of Package, one set corresponds to the category of Express Delivery Slip, and the last set corresponds to the category of Image Background.

[0039] Optionally, after determining the mask information of the package and / or waybill, the mask coordinates of the package and / or waybill after noise removal can be extracted to determine the center position of the package and / or waybill, that is, to determine the position information of the package and / or waybill in the actual scene image.

[0040] Optionally, after determining the location information of packages and / or waybills in the actual scene image, the nearest distance between the waybill closest to the package location and the package can be determined based on the location information. Based on the relationship between the nearest distance and a preset distance threshold (denoted as k), it can be determined whether the waybill is inside the package. If the nearest distance between the package and the waybill is less than k, it means that the waybill is inside the package. If it is greater than k, it means that the package does not have a corresponding waybill. This way, the packages and / or waybills in the actual scene image are divided into at least one group.

[0041] For example, the preset distance threshold k can be set to the maximum value of the width and height of the package and / or express delivery slip in the actual scene image.

[0042] Optionally, after determining the location information of packages and / or waybills in the actual scene image, if the number of packages and waybills is the same, it is considered that the packages and waybills correspond one-to-one. For each package, the waybill closest to that package can be determined, and the waybill and package can be grouped together. If the number of packages is greater than the number of waybills, it indicates that there are packages that do not contain waybills. For each waybill, the waybill and the package closest to it can be grouped together. For unmatched packages, each package is considered as a group, thereby dividing the packages and / or waybills in the actual scene image into at least one group.

[0043] Optionally, after grouping the packages and / or waybills in the actual scene images, the packages and / or waybills in each group can be matched, that is, the matching of packages and / or waybills in the actual scene images can be determined based on the grouping results.

[0044] S102. Based on the correspondence between the package and / or the waybill, when a package is detected in the actual scene image, determine the target package and determine whether there is a waybill corresponding to the target package.

[0045] The target package refers to the package in the actual scene image.

[0046] Optionally, when a package is detected in the actual scene image, each package can be identified as a target package. Further analysis of the package and / or waybill correspondence can then determine whether a waybill corresponding to the target package exists. For example, if the actual scene image contains package 1, package 2, and waybill 1, and package 1 corresponds to waybill 1, then if the target package is package 1, it can be determined that a waybill corresponding to the target package exists; if the target package is package 2, it can be determined that no waybill corresponding to the target package exists.

[0047] Optionally, after determining whether there is a waybill corresponding to the target package, the method further includes: if it is determined that there is no waybill corresponding to the target package, then binarize the image area where the target package is located, generate a binarized image, and determine the pixel matrix corresponding to the binarized image; based on preset filtering rules, determine the optimal position for marking the target package in the image area where the target package is located according to the pixel matrix and the preset label matrix.

[0048] The label matrix refers to a preset matrix corresponding to the size of the label to be printed, which refers to the label to be printed or covered on the target package.

[0049] For example, based on the size of the label to be labeled, a label matrix with an element of 1 can be set, and the target can be wrapped with one-quarter of its width and height, which are respectively determined as the number of rows and columns of the label matrix, thereby generating the label matrix.

[0050] Optionally, if it is determined that there is no waybill corresponding to the target package, the package image can be extracted from the actual scene image based on the rectangular coordinates (i.e., location information) of the package, that is, the image area where the target package is located can be determined. Then, by assigning values ​​according to the category of the mask information, a binarized image can be generated. For example, the pixel of category package can be assigned a value of 255, and the pixel of category background can be assigned a value of 0, thereby generating a binarized image corresponding to the image area where the target package is located. And according to the pixel value of each pixel in the generated binarized image, the corresponding pixel matrix is ​​generated.

[0051] Optionally, based on preset filtering rules, the optimal position for marking the target package is determined in the image region where the target package is located, according to the pixel matrix and the preset label matrix. This includes: using the preset label matrix as a sliding window, and performing a traversal convolution operation on the pixel matrix of the target image and the preset label matrix based on a preset stride; based on the convolution result, identifying the sub-regions corresponding to the sub-matrices that satisfy the preset filtering rules in the determined pixel matrix; performing a connectivity operation on each sub-region to determine a connected region, and determining the center position of the connected region as the optimal position for marking the target package.

[0052] For example, a step size of 1 can be set, and the identifier matrix and the pixel matrix corresponding to the binarized image can be processed by a slider to determine the pixel submatrix after each slider processing. Then, a dot product operation can be performed on the identifier matrix after each slider processing and the corresponding pixel submatrix. In other words, the pixel matrix of the target image and the preset identifier matrix can be traversed and convolved.

[0053] It should be noted that if the width and height of the binarized image are w and h respectively, then the number of operations corresponding to the above traversal convolution operation of slider processing and dot product is 9 / 16 (w×h) times.

[0054] For example, the matrix product processed by each slider can be expressed by the following formula:

[0055]

[0056] Among them, Ω i This represents the matrix product of the i-th slider operation. U represents the pixel submatrix processed in the i-th slider operation. 矩阵 This represents a preset identifier matrix, with each element being 1.

[0057] Optionally, based on the matrix product corresponding to each slider processing in the convolution result, the pixel submatrix corresponding to the matrix product being less than the preset filtering threshold can be determined, and the pixel submatrix can be removed from the submatrix corresponding to the pixel matrix. The remaining pixel regions corresponding to each submatrix that meet the requirements can be determined, that is, the sub-regions corresponding to the submatrix that meet the preset filtering rules in the determined pixel matrix.

[0058] For example, the preset filtering threshold can be set to k×w 矩阵 ×h 矩阵 , where w 矩阵 and h 矩阵 These represent the number of rows and columns of the pixel matrix, respectively, and k is a preset constant value, such as 0.75.

[0059] It should be noted that by filtering the values ​​of the matrix product processed by each slider and removing values ​​less than a preset filtering threshold, the points of the mask that were originally deleted from the edge can be mapped and the binarized image of the mask can be compressed.

[0060] Optionally, a connectivity operation can be performed on the binarized image after mask compression, that is, the obtained sub-regions can be merged to generate connected regions that can be labeled, thereby realizing the connectivity operation of the sub-regions and determining the connected regions.

[0061] Optionally, after determining the connected region, the center point of the connected region can be determined as the center position to be marked, that is, the center position of the connected region can be determined as the optimal position to mark the target package.

[0062] S103. If so, then in the image area where the target package is located, determine the area other than the waybill as the target area, and determine the optimal position to mark the target package in the target area.

[0063] The image region containing the target package refers to the image region, after segmentation and extraction of the actual scene image, that includes at least one of the following: the target package, the delivery slip on the target package, and the background image. The target region represents the area on the target package, excluding the delivery slip, that is permitted to be marked.

[0064] Optionally, if it is determined that a waybill corresponding to the target package exists, the area excluding the waybill in the image region where the target package is located can be determined as the target region, and the optimal position for marking the target package can be determined within the target region. This includes: binarizing the image region where the target package is located based on the mask information of the target package and the corresponding waybill to generate a binarized image, and determining the area excluding the waybill in the binarized image as the target region; determining the center line corresponding to the target region, performing a traversal convolution operation between the pixel matrix corresponding to each center point on the center line and a preset label matrix, determining the center point position corresponding to the pixel matrix that meets the preset filtering conditions, and determining the center point position as the optimal position for marking the target package.

[0065] The identifier matrix for the case where a corresponding waybill exists for the target package can be the same as or different from the identifier matrix for the case where a corresponding waybill does not exist; this invention does not impose any restrictions on this. For example, the width and height of the identifier matrix can be set to 3×3, and the elements of the identifier matrix can be set to 1, thereby generating a preset identifier matrix.

[0066] Optionally, the image of the package can be extracted based on the rectangular coordinates of the package to determine the image region where the target package is located. Further, the extracted image region can be binarized based on the mask information, with pixels representing the package set to 255 and pixels representing the background and the waybill set to 0, thus generating a binarized image. Specifically, one possible implementation for binarizing the image region where the target package is located based on the mask information of the target package and the corresponding waybill is as follows: Values ​​are assigned to the image region where the target package is located based on the categories of the mask information of the target package and the corresponding waybill, thus generating a binarized image.

[0067] For example, the steps to generate a binarized image can be: 1) Generate a binarized image with the same width and height as the image area containing the target package, and set the pixel value of the binarized image to 0; 2) Based on the category information corresponding to the mask information, map the pixel position of the mask category "package" onto the binarized image and update its value to 255; 3) Based on the category information corresponding to the mask information, map the pixel position of the mask category "express delivery slip" onto the binarized image and update its value to 0. The blank area with a pixel value of 255 in the binarized image obtained through the above steps is the target area.

[0068] Optionally, after determining the target region, a traversal convolution operation can be performed on the preset identifier matrix and the pixel matrix corresponding to the target region. That is, based on a preset stride (e.g., stride of 1), the identifier matrix and the pixel matrix corresponding to the target region are subjected to slider processing and dot product operations. The number of operations required is (w-3)×(h-3), where w and h are the width and height of the pixel matrix corresponding to the target region, respectively.

[0069] Optionally, after performing a traversal convolution operation on the preset identifier matrix and the pixel matrix corresponding to the target region, image regions whose dot product is less than a preset product threshold (the preset product threshold can be, for example, 9) can be removed. The pixel values ​​of the target region after the removal operation are set to 0, and the Euclidean distance between each pixel in the target region after the removal operation is calculated. The points that are farthest from 0 pixels and have a variance of 0 within a range of 255 pixels are merged to obtain the center line of the target region.

[0070] It should be noted that if the courier unit is in the middle of the package, the determined target area is generally a ring-shaped area, and the center line of the target area is the center line of the ring. If the courier unit is on the edge of the package, the determined target area is generally a U-shaped area, and the center line of the target area is the center line of the U-shaped area.

[0071] Optionally, after determining the center line corresponding to the target area, a method similar to the traversal convolution operation described in the above embodiments of the present invention can be used. This involves performing a traversal convolution operation between the pixel matrix corresponding to each center point on the center line and a preset label matrix. For example, the preset label matrix with an element value of 1 can be processed by sliding block processing and dot product operation based on the principle that the center point is a point on the center line and a preset step size. That is, the pixel matrix corresponding to each center point on the center line is traversed and convolved with the preset label matrix. After performing the traversal convolution operation, the position of the center point corresponding to the maximum dot product can be determined as the optimal position for labeling the target package.

[0072] Optionally, after determining the target area, the system can analyze whether the target area meets the preset conditions for marking. If not, it is assumed that marking may cover the express delivery information, and the subsequent S104 operation will not be executed. If yes, it indicates that the target area can be marked, and the subsequent S104 operation will continue to be executed.

[0073] Specifically, one possible implementation method for analyzing whether the target area meets the preset conditions for marking is as follows: Set a matrix with width and height corresponding to the width and height of the target package, and set the value of the matrix to 0. Set the area covered by the mark corresponding to the target package to 255, and denote it as matrix 1; extract the rectangle width and height of the target package segmented by the instance, set the area of ​​the package to 255, and set the background and waybill areas to 0, and denote it as matrix 2; compare the positions with the value of 255 in matrix 1 and matrix 2, and obtain the number of coordinates with the same value of 255, and the percentage of the area of ​​the marked area. If it is less than 0.75, it means that the marked area does not meet the conditions for marking and may cover the waybill information. If it is greater than or equal to 0.75, it means that the marked area can be marked.

[0074] S104. Based on the optimal position and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, determine the actual position in the actual scene for marking the target package, and control the relevant equipment to mark the package according to the actual position.

[0075] The optimal location refers to the best position in the actual scene image for marking the target package. The actual location refers to the best position in the actual scene for marking the target package.

[0076] Optionally, the mapping relationship between the actual scene image captured by the camera and the monitoring range of the actual scene by the camera is determined, including: based on the principles of pinhole imaging and similar triangles, determining the ratio between the width and height of the actual scene image captured by the camera and the width and height of the monitoring range of the actual scene by the camera; and determining the mapping relationship between the size of the actual scene image captured by the camera and the monitoring range of the actual scene by the camera based on the ratio.

[0077] For example, the ratio of the width of the actual scene image captured by the camera to the width of the camera's monitoring range of the actual scene is rate. width This can be expressed by the following formula:

[0078]

[0079] Among them, width_ori fact This refers to the width of the monitored area, width_obj im，g The width_ori refers to the width of the target object in the preset actual scene image. imgThis refers to the width of the actual scene image.

[0080] For example, the ratio of the height of the actual scene image captured by the camera to the height of the camera's monitoring range of the actual scene is rate. height This can be expressed by the following formula:

[0081]

[0082] Among them, height_ori fact This refers to the height of the monitoring range, height_obj img The height_ori refers to the height of the target object in the preset actual scene image. img This refers to the height of the actual scene image.

[0083] Optionally, based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined, including: determining the width and height values ​​between the optimal location and the image edge in the actual scene image; and determining the actual location for marking the target package in the actual scene based on the width and height values ​​and the mapping relationship between the actual scene image captured by the camera and the camera's monitoring range of the actual scene.

[0084] Optionally, based on the width and height values, and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the width and height values ​​of the actual location for marking the target package in the actual scene and the boundary line of the monitoring range of the actual scene can be determined, thereby determining the actual location for marking the target package in the actual scene.

[0085] For example, the mapping relationship between the width and height values ​​of the optimal position in the actual scene image and the image edge, and the width and height values ​​of the actual position and the boundary line of the monitoring range, can be expressed by the following formula:

[0086] top_x fact =top_x img ×rate width

[0087] left_y fact =left_y img ×rate heigth

[0088] bot_x fact =bot_x img ×rate width

[0089] right_yfact =right_y img ×rate heigth

[0090] Specifically, the optimal position in the actual scene image has width and height values ​​relative to the image edge, which are: the width relative to the top edge of the image is top_x. img The height value of the left edge of the image is left_y img The width of the bottom edge of the image is bot_x img The height value of the right edge of the image is right_y img Then, the ratio of the width of the actual scene image to the width of the monitoring range, rate, is used. width And the ratio of the height of the actual scene image to the height of the monitoring range (rate) height This allows us to calculate the width and height values ​​of the actual location where the target package is marked in the real-world scenario relative to the boundary line of the monitoring range. Specifically, the width value relative to the upper edge of the monitoring range is top_x. fact The height value of the left edge of the monitored area is left_y fact The width of the lower edge of the monitored area is bot_x fact The height value of the right edge of the monitored area is right_y fact .

[0091] It should be noted that, by using pinhole imaging and the principle of similar triangles, this invention expresses the correlation between the image and the actual scene through a formula, thus realizing the mapping between the two.

[0092] The technical solution provided by this invention can determine the location of packages and / or waybills in an image through an instance segmentation algorithm, further determine the optimal marking position for each package, and determine the marking position for the package in the actual scene based on the mapping relationship between the image and the actual scene. It can automatically select the optimal marking position, thereby freeing up manpower, improving the efficiency of express delivery work, and realizing an automated, efficient and accurate package marking solution.

[0093] Example 2

[0094] Figure 2 This is a flowchart of a package marking method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment provides a preferred example.

[0095] like Figure 2 As shown, the method includes the following specific steps:

[0096] S201. Based on the principles of pinhole imaging and similar triangles, determine the ratio between the width and height of the actual scene image captured by the camera and the width and height of the camera's monitoring range of the actual scene.

[0097] S202. Based on the ratio, determine the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene.

[0098] S203. Based on the preset instance segmentation algorithm, the actual scene image captured by the camera is segmented to determine the mask information of the package and / or express delivery slip in the actual scene image, and the position information of the package and / or express delivery slip in the actual scene image is determined according to the mask information.

[0099] S204. Based on the location information, divide the packages and / or express delivery slips in the actual scene images into at least one group, and determine the corresponding situation of the packages and / or express delivery slips in the actual scene images based on the grouping results.

[0100] S205. Based on the correspondence between the package and / or the waybill, when a package is detected in the actual scene image, determine the target package and determine whether there is a waybill corresponding to the target package. If yes, execute S206; otherwise, execute S207.

[0101] S206. If so, then in the image area where the target package is located, determine the area other than the waybill as the target area, and determine the optimal position to mark the target package in the target area.

[0102] Optionally, the image region where the target package is located can be binarized based on the mask information of the target package and the corresponding waybill, generating a binarized image, and the region in the binarized image other than the waybill can be determined as the target region; the center line corresponding to the target region can be determined, and the pixel matrix corresponding to each center point on the center line can be traversed and convolved with a preset label matrix to determine the center point position corresponding to the pixel matrix that meets the preset filtering conditions, and the center point position can be determined as the optimal position for labeling the target package.

[0103] S207. If it is determined that there is no waybill corresponding to the target package, then the image area where the target package is located is binarized to generate a binarized image, and the pixel matrix corresponding to the binarized image is determined.

[0104] S208. Based on preset filtering rules, determine the optimal position for marking the target package in the image area where the target package is located, according to the pixel matrix and the preset identification matrix.

[0105] Optionally, a preset label matrix can be used as a sliding window, and based on a preset stride, the pixel matrix of the target image and the preset label matrix can be traversed and convolved. According to the result of the convolution, the sub-regions corresponding to the sub-matrices that satisfy the preset filtering rules in the determined pixel matrix are selected. Each sub-region is connected to determine the connected region, and the center position of the connected region is determined as the optimal position for labeling the target package.

[0106] S209. Based on the optimal position and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, determine the actual position in the actual scene for marking the target package, and control the relevant equipment to mark the package according to the actual position.

[0107] Optionally, the width and height values ​​between the optimal location and the edge of the actual scene image can be determined; based on the width and height values, and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene can be determined.

[0108] Example 3

[0109] Figure 3 This is a structural block diagram of a package marking device provided in Embodiment 3 of the present invention. The package marking device provided in this embodiment is applicable to marking packages in real-world scenarios. This package marking device can be implemented in hardware and / or software, such as... Figure 3 As shown, the device specifically includes: a first determining module 301, a second determining module 302, a third determining module 303, and a marking module 304. Among them,

[0110] The first determining module 301 is used to determine the location information of packages and / or express delivery slips in the actual scene images captured by the camera, as well as the corresponding situation of packages and / or express delivery slips, based on a preset instance segmentation algorithm.

[0111] The second determining module 302 is used to determine the target package and determine whether there is a corresponding express delivery slip when a package is detected in the actual scene image, based on the correspondence between the package and / or the express delivery slip.

[0112] The third determining module 303 is used to determine, if so, the area outside the waybill in the image area where the target package is located as the target area, and determine the optimal position for marking the target package in the target area;

[0113] The marking module 304 is used to determine the actual location for marking the target package in the actual scene based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, and to control the relevant equipment to mark the package based on the actual location.

[0114] The technical solution of this invention, based on a preset instance segmentation algorithm, determines the location information of packages and / or waybills in the actual scene image captured by the camera, as well as the correspondence between packages and / or waybills. Based on the correspondence, when a package is detected in the actual scene image, a target package is identified, and it is determined whether a corresponding waybill exists. If so, the area excluding the waybill within the image region where the target package is located is identified as the target area. The optimal position for marking the target package is determined within the target area. Based on the optimal position and the mapping relationship between the actual scene image captured by the camera and the camera's monitoring range of the actual scene, the actual position for marking the target package in the actual scene is determined. Based on the actual position, relevant equipment is controlled to mark the package. This method automates the marking of packages, improving marking efficiency. Furthermore, it ensures that the marked labels do not obscure the waybill information or overlap with the package, thus improving the accuracy of package marking.

[0115] Furthermore, the first determining module 301 is specifically used for:

[0116] Based on a preset instance segmentation algorithm, the actual scene image captured by the camera is segmented to determine the mask information of the package and / or express delivery slip in the actual scene image, and the position information of the package and / or express delivery slip in the actual scene image is determined according to the mask information.

[0117] Based on the location information, the packages and / or express delivery slips in the actual scene image are divided into at least one group, and the corresponding situation of the packages and / or express delivery slips in the actual scene image is determined according to the grouping results.

[0118] Furthermore, the third determining module 303 is specifically used for:

[0119] Based on the mask information of the target package and the corresponding waybill, the image region where the target package is located is binarized to generate a binarized image, and the region in the binarized image other than the waybill is determined as the target region.

[0120] Determine the center line corresponding to the target area, perform a traversal convolution operation on the pixel matrix corresponding to each center point on the center line and the preset label matrix, determine the center point position corresponding to the pixel matrix that meets the preset filtering conditions, and determine the center point position as the optimal position for labeling the target package.

[0121] Furthermore, the aforementioned device also includes:

[0122] The fourth determining module is used to binarize the image region where the target package is located if it is determined that there is no express delivery slip corresponding to the target package, generate a binarized image, and determine the pixel matrix corresponding to the binarized image.

[0123] The location determination module is used to determine the optimal location for marking the target package in the image area where the target package is located, based on preset filtering rules, the pixel matrix, and the preset identification matrix.

[0124] Furthermore, the location determination module is specifically used for:

[0125] Using a preset identifier matrix as a sliding window, and based on a preset stride, performing a traversal convolution operation between the pixel matrix of the target image and the preset identifier matrix.

[0126] Based on the result of convolution, select the sub-regions corresponding to the sub-matrices that satisfy the preset filtering rules from the determined pixel matrix;

[0127] Perform a connectivity operation on each sub-region to determine the connected regions, and determine the center position of the connected regions as the optimal position for marking the target package.

[0128] Furthermore, the above-mentioned device is also used for:

[0129] Based on the principles of pinhole imaging and similar triangles, the ratio between the width and height of the actual scene image captured by the camera and the width and height of the camera's monitoring range of the actual scene is determined.

[0130] Based on the ratio, a mapping relationship is determined between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene.

[0131] Furthermore, the marking module 304 is specifically used for:

[0132] Determine the optimal position in the actual scene image and the width and height values ​​between it and the image edge;

[0133] Based on the width and height values, and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined.

[0134] Example 4

[0135] Figure 4 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention. Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0136] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the package marking method.

[0139] In some embodiments, the package marking method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the package marking method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the package marking method by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0145] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0146] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for marking packages, characterized in that, include: Based on a preset instance segmentation algorithm, the location information of packages and / or express delivery slips in the actual scene images captured by the camera is determined, as well as the corresponding situation of packages and / or express delivery slips. Based on the correspondence between the package and / or the waybill, when a package is detected in the actual scene image, the target package is identified, and it is determined whether there is a waybill corresponding to the target package. If so, then in the image area where the target package is located, determine the area other than the waybill as the target area, and determine the optimal position for marking the target package in the target area; Based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined, and the relevant equipment is controlled to mark the package based on the actual location. The method involves determining the location information of packages and / or express delivery slips in the actual scene image captured by the camera, and the corresponding information of the packages and / or express delivery slips, based on a preset instance segmentation algorithm. This includes: segmenting the actual scene image captured by the camera based on the preset instance segmentation algorithm to determine the mask information of packages and / or express delivery slips in the actual scene image; determining the location information of packages and / or express delivery slips in the actual scene image based on the mask information; dividing the packages and / or express delivery slips in the actual scene image into at least one group based on the location information; and determining the corresponding information of packages and / or express delivery slips in the actual scene image based on the grouping results. Specifically, based on the location information, the packages and / or waybills in the actual scene image are divided into at least one group, including: Based on location information, for each package, the nearest distance between the nearest tracking number and the package is determined. Then, based on the relationship between this nearest distance and a preset distance threshold, it is determined whether the tracking number is within the package's location area. This allows the packages and / or tracking numbers in the actual scene image to be grouped into at least one group; or If the number of packages and waybills are the same, it is assumed that there is a one-to-one correspondence between packages and waybills. For each package, the nearest waybill is identified, and the waybill and package are grouped together. If the number of packages is greater than the number of waybills, it indicates that there are packages that do not contain waybills. For each waybill, the waybill and the nearest package are grouped together. For packages that are not matched, each package is treated as a group, thereby dividing the packages and / or waybills in the actual scene image into at least one group.

2. The method according to claim 1, characterized in that, Within the image area containing the target package, the area excluding the waybill is determined as the target area, and the optimal location within the target area for marking the target package is determined, including: Based on the mask information of the target package and the corresponding waybill, the image region where the target package is located is binarized to generate a binarized image, and the region in the binarized image other than the waybill is determined as the target region. Determine the center line corresponding to the target area, perform a traversal convolution operation on the pixel matrix corresponding to each center point on the center line and the preset label matrix, determine the center point position corresponding to the pixel matrix that meets the preset filtering conditions, and determine the center point position as the optimal position for labeling the target package.

3. The method according to claim 1, characterized in that, After confirming the existence of a tracking number corresponding to the target package, the process also includes: If it is determined that there is no waybill corresponding to the target package, the image region where the target package is located is binarized to generate a binarized image, and the pixel matrix corresponding to the binarized image is determined. Based on preset filtering rules, and according to the pixel matrix and preset identifier matrix, the optimal position for marking the target package is determined in the image area where the target package is located.

4. The method according to claim 3, characterized in that, Based on preset filtering rules, and according to the pixel matrix and preset identifier matrix, the optimal position for marking the target package is determined in the image region where the target package is located, including: Using a preset identifier matrix as a sliding window, and based on a preset stride, performing a traversal convolution operation between the pixel matrix of the target image and the preset identifier matrix. Based on the convolution result, the sub-regions corresponding to the sub-matrices that satisfy the preset filtering rules are determined from the pixel matrix; Perform a connectivity operation on each sub-region to determine the connected regions, and determine the center position of the connected regions as the optimal position for marking the target package.

5. The method according to claim 1, characterized in that, Also includes: Based on the principles of pinhole imaging and similar triangles, the ratio between the width and height of the actual scene image captured by the camera and the width and height of the camera's monitoring range of the actual scene is determined. Based on the ratio, a mapping relationship is determined between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene.

6. The method according to claim 5, characterized in that, Based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined, including: Determine the optimal position in the actual scene image and the width and height values ​​between it and the image edge; Based on the width and height values, and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, the actual location for marking the target package in the actual scene is determined.

7. A package marking device, characterized in that, include: The first determining module is used to determine the location information of packages and / or express delivery slips in the actual scene images captured by the camera, as well as the corresponding situation of packages and / or express delivery slips, based on a preset instance segmentation algorithm. The second determining module is used to determine the target package and determine whether there is a corresponding express delivery slip when a package is detected in the actual scene image, based on the correspondence between the package and / or the express delivery slip. The third determining module is used to, if so, determine the area outside the waybill in the image area where the target package is located as the target area, and determine the optimal position for marking the target package in the target area; The marking module is used to determine the actual location for marking the target package in the actual scene based on the optimal location and the mapping relationship between the actual scene image captured by the camera and the size of the camera's monitoring range of the actual scene, and to control the relevant equipment to mark the package based on the actual location. Specifically, the first determining module is used to: segment the actual scene image captured by the camera based on a preset instance segmentation algorithm, determine the mask information of packages and / or express delivery slips in the actual scene image, and determine the position information of packages and / or express delivery slips in the actual scene image according to the mask information; divide the packages and / or express delivery slips in the actual scene image into at least one group according to the position information, and determine the corresponding situation of packages and / or express delivery slips in the actual scene image according to the grouping results; The first determining module is further configured to: based on location information, for each package, determine the closest distance between the nearest delivery slip and the package, and based on the relationship between the closest distance and a preset distance threshold, determine whether the delivery slip is in the area where the package is located, thereby dividing the packages and / or delivery slips in the actual scene image into at least one group; or If the number of packages and waybills are the same, it is assumed that there is a one-to-one correspondence between packages and waybills. For each package, the nearest waybill is identified, and the waybill and package are grouped together. If the number of packages is greater than the number of waybills, it indicates that there are packages that do not contain waybills. For each waybill, the waybill and the nearest package are grouped together. For packages that are not matched, each package is treated as a group, thereby dividing the packages and / or waybills in the actual scene image into at least one group.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the package marking method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the package marking method according to any one of claims 1-6.

Images