Methods, apparatus, computer equipment and storage media for identifying non-standard parts
By combining computer vision methods with two-dimensional images and three-dimensional point clouds, the detection box and type of express parcels are identified, solving the problem of inaccurate identification of non-standard parcels in logistics transit areas, achieving efficient identification of non-standard parcels and protecting the conveying equipment.
Patent Information
- Application Number
- CN202211142165.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-20
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-09-20
AI Technical Summary
In existing technologies, when express parcels arrive at logistics transit points, the identification of non-standard parcels by personnel is inaccurate and inefficient, especially for overweight or oversized parcels, which may lead to damage to conveyor belts.
Using computer vision methods, combining initial two-dimensional images and initial three-dimensional point clouds, the detection box and package type are determined by a preset package recognition and positioning model. The two-dimensional and three-dimensional coordinate systems are unified by the checkerboard calibration method to determine the package outline and size, thereby identifying non-standard packages.
It improves the accuracy and efficiency of identifying non-standard items, ensuring that overweight and oversized packages can be accurately identified and avoiding damage to the conveyor system.
Smart Images

Figure CN117788742B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of express mail identification technology, specifically to a method, apparatus, computer equipment, and storage medium for identifying unconventional mail. Background Technology
[0002] After packages arrive at the logistics transit center, they are transported on conveyor belts. However, some packages may be too heavy or too large, causing damage to the conveyor belts. Currently, the main way to determine whether a package is an irregular item is through the observation and experience of staff. Due to individual differences, it cannot be guaranteed that every staff member can accurately identify overweight or oversized irregular items. Summary of the Invention
[0003] This application provides a method, apparatus, computer device, and storage medium for identifying non-standard parts, aiming to solve the problems of inaccurate identification and low identification efficiency of non-standard parts in the prior art.
[0004] On the one hand, this application provides a method for identifying unconventional parts, the method comprising:
[0005] Acquire the initial two-dimensional image and initial three-dimensional point cloud of the package to be identified;
[0006] The initial two-dimensional image is used to identify and locate the package by means of a preset package identification and location model, thereby determining the detection box of the package to be identified in the preset package identification and location model and the package type of the package to be identified.
[0007] The outline of the package to be identified is determined based on the detection frame, the initial two-dimensional image, and the initial three-dimensional point cloud.
[0008] Based on the outline of the package and the type of package, determine whether the package to be identified is an unconventional package.
[0009] In some embodiments of this application, before obtaining the initial two-dimensional image and initial three-dimensional point cloud corresponding to the package to be identified, the method further includes:
[0010] Obtain the two-dimensional image and three-dimensional point cloud of the chessboard of the preset chessboard calibration board;
[0011] Establish a two-dimensional coordinate system corresponding to the two-dimensional image, and a three-dimensional coordinate system corresponding to the three-dimensional point cloud;
[0012] Determine the first coordinates of the corner points of the chessboard in the two-dimensional image of the chessboard in the two-dimensional coordinate system;
[0013] Determine the second coordinates of the corner points of the chessboard grid in the three-dimensional point cloud of the chessboard grid in the three-dimensional coordinate system;
[0014] The mapping matrix between the two-dimensional image and the three-dimensional point cloud is determined based on the first coordinate and the second coordinate.
[0015] In some embodiments of this application, the initial two-dimensional image includes multiple detection boxes;
[0016] The step of determining the outline of the package to be identified based on the detection frame, the initial two-dimensional image, and the initial three-dimensional point cloud includes:
[0017] Determine whether multiple detection boxes in the initial two-dimensional image overlap.
[0018] If the multiple detection boxes do not overlap, the initial two-dimensional image is clustered using a preset clustering method to obtain the outline of the package to be identified.
[0019] In some embodiments of this application, determining whether the package to be identified is an unconventional package based on the package outline and the package type includes:
[0020] If the multiple detection boxes do not overlap, the outline of the package to be identified is mapped onto the three-dimensional coordinate system to determine the package size corresponding to the package to be identified;
[0021] Based on the package size and package type, determine whether the package to be identified is an unconventional package.
[0022] In some embodiments of this application, determining the outline of the express package based on the detection frame, the initial two-dimensional image, and the initial three-dimensional point cloud further includes:
[0023] If multiple detection boxes overlap, then the noise in the initial 3D point cloud is removed to obtain the 3D point cloud of the package to be identified.
[0024] The pixels in the three-dimensional point cloud are fitted to obtain the initial outline of the package to be identified.
[0025] The initial package outline is segmented to obtain the package outline of the package to be identified.
[0026] In some embodiments of this application, determining whether the package to be identified is an unconventional package based on the package outline and the package type includes:
[0027] If the multiple detection boxes overlap, the size of the package to be identified is determined based on the package outline and the initial 3D point cloud.
[0028] Based on the package size and package type, determine whether the package to be identified is a regular package.
[0029] In some embodiments of this application, determining whether the package to be identified is a regular package based on the package size and package type includes:
[0030] Obtain multiple preset size ranges for different express delivery types;
[0031] Based on the type of express delivery, determine the target size range of the express delivery to be identified from the plurality of express delivery size ranges;
[0032] Based on the dimensions of the package and the target package size range, determine whether the package to be identified is a standard package.
[0033] On the other hand, this application provides a non-standard part identification device, the non-standard part identification device comprising:
[0034] The image acquisition unit is used to acquire the initial two-dimensional image and initial three-dimensional point cloud of the package to be identified;
[0035] The express delivery identification and positioning unit is used to identify and locate the express delivery in the initial two-dimensional image through a preset express delivery identification and positioning model, and to determine the detection box of the express delivery to be identified in the preset express delivery identification and positioning model and the express delivery type of the express delivery to be identified;
[0036] The parcel outline determination unit is used to determine the parcel outline of the parcel to be identified based on the detection frame, the initial two-dimensional image and the initial three-dimensional point cloud;
[0037] The non-standard item identification unit is used to determine whether the item to be identified is a non-standard item based on the item outline and the item type.
[0038] On the other hand, this application also provides a computer device, the computer device comprising:
[0039] One or more processors;
[0040] Memory; and
[0041] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the non-routine component identification method as described in any of the first aspects.
[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the non-standard component identification method described in any of the first aspects.
[0043] This application provides a method, apparatus, computer device, and storage medium for identifying unconventional items. First, an initial two-dimensional image and an initial three-dimensional point cloud corresponding to the item to be identified are acquired. The item is then identified and located in the initial two-dimensional image to determine the corresponding detection box and item type. The item's outline is determined by combining the initial two-dimensional image, initial three-dimensional point cloud, and detection box. Finally, the item's outline and type are combined to determine whether the item is an unconventional item that is not excessively heavy or large. This application combines the two-dimensional image and three-dimensional point cloud corresponding to the item to be identified, along with the item's outline and type, to identify excessively heavy or large unconventional items. Furthermore, it utilizes computer vision methods for identification, which can effectively improve the accuracy and efficiency of unconventional item identification. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 This is a schematic diagram of a scenario for the non-standard component identification system provided in an embodiment of this application;
[0046] Figure 2 This is a schematic flowchart of an embodiment of the non-standard component identification method provided in this application.
[0047] Figure 3 A schematic flowchart illustrating an embodiment of determining the outline of a package provided in this application;
[0048] Figure 4 This is a flowchart illustrating an embodiment of determining whether a package is an unconventional package in this application.
[0049] Figure 5 A flowchart illustrating an embodiment of determining the type of regular shipment based on the size and type of the shipment, provided for the purposes of this application;
[0050] Figure 6 This is a schematic diagram of an embodiment of the non-standard component identification device provided in this application.
[0051] Figure 7 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation
[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0053] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0054] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0055] It should be noted that since the method in this application embodiment is executed in an electronic device, the processing objects of each electronic device exist in the form of data or information, such as time, which is essentially time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the electronic device can process them. Specific details will not be elaborated here.
[0056] This application provides a method, apparatus, computer device, and storage medium for identifying unconventional components, which will be described in detail below.
[0057] Please see Figure 1 , Figure 1 This is a schematic diagram of a scenario for an unconventional part identification system provided in an embodiment of this application. The unconventional part identification system may include a computer device 100, which integrates an unconventional part identification device, such as... Figure 1 The computer device 100 in this embodiment is mainly used for: acquiring an initial two-dimensional image and an initial three-dimensional point cloud of the parcel to be identified; performing parcel identification and positioning on the initial two-dimensional image using a preset parcel identification and positioning model, determining the detection box of the parcel to be identified in the preset parcel identification and positioning model and the parcel type of the parcel to be identified; determining the parcel outline of the parcel to be identified based on the detection box, the initial two-dimensional image, and the initial three-dimensional point cloud; and determining whether the parcel to be identified is an unconventional parcel based on the parcel outline and the parcel type.
[0058] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.
[0059] It is understood that the computer device 100 used in the embodiments of this application can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal computer device may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, tablet computer, laptop computer, etc.
[0060] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the document. It is understood that the non-standard device identification system may also include one or more other services, which are not limited here.
[0061] In addition, such as Figure 1As shown, the unconventional package identification system may also include a storage module 200 for storing data, such as storing the initial two-dimensional image and initial three-dimensional point cloud corresponding to the package to be identified.
[0062] It should be noted that, Figure 1 The schematic diagram of the unconventional part identification system shown is merely an example. The unconventional part identification system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of unconventional part identification systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.
[0063] First, this application provides a method for identifying unconventional parcels. The execution subject of this method is an unconventional parcel identification device applied to a computer device. The method includes: acquiring an initial two-dimensional image and an initial three-dimensional point cloud of the parcel to be identified; performing parcel identification and positioning on the initial two-dimensional image using a preset parcel identification and positioning model to determine the detection box of the parcel to be identified in the preset parcel identification and positioning model and the parcel type; determining the parcel outline of the parcel to be identified based on the detection box, the initial two-dimensional image, and the initial three-dimensional point cloud; and determining whether the parcel to be identified is an unconventional parcel based on the parcel outline and the parcel type.
[0064] like Figure 2 The diagram shown is a flowchart of an embodiment of the non-standard component identification method in this application. The non-standard component identification method includes:
[0065] 21. Obtain the initial two-dimensional image and initial three-dimensional point cloud corresponding to the package to be identified.
[0066] In the embodiments of this application, it is necessary to acquire an initial two-dimensional image and an initial three-dimensional point cloud corresponding to the package to be identified, and to use the initial two-dimensional image and the initial three-dimensional point cloud for the identification of non-standard packages. Non-standard packages refer to excessively heavy or large packages, which may damage the conveying device and therefore need to be identified and processed separately.
[0067] In the embodiments of this application, conventional 2D cameras and 3D point cloud cameras can be used to acquire 2D images and 3D point clouds, respectively. Before acquiring the initial 2D image and initial 3D point cloud corresponding to the package to be identified, it is necessary to unify the 2D image and 3D point cloud so that the 2D image acquired by the 2D camera can be mapped onto the 3D point cloud. Unifying the 2D image and 3D point cloud is necessary because the 3D point cloud includes the (x, y, z) coordinates of each pixel, allowing the size of the package to be determined. However, due to differences in viewing angle and shooting distance, the size of the package cannot be determined from the 2D image.
[0068] In some embodiments, a checkerboard calibration method can be used to calibrate the two-dimensional image and the three-dimensional point cloud separately. Specifically, this can be done as follows: a checkerboard calibration plate is placed flat on the conveyor belt, and the corresponding two-dimensional image and three-dimensional point cloud of the checkerboard calibration plate are acquired; a two-dimensional coordinate system corresponding to the two-dimensional image and a three-dimensional coordinate system corresponding to the three-dimensional point cloud are created respectively; the first coordinate of the checkerboard corner point in the two-dimensional coordinate system and the second coordinate of the checkerboard corner point in the three-dimensional coordinate system are determined respectively; the two-dimensional image and the three-dimensional point cloud are unified based on the first and second coordinates. Unifying the two-dimensional and three-dimensional images essentially determines the mapping relationship between the two-dimensional and three-dimensional coordinate systems; this allows for subsequent mapping of the two-dimensional image to the three-dimensional coordinate system, obtaining the mapped three-dimensional coordinates, and using the three-dimensional coordinates to determine the dimensions of the package. Specific methods for unifying the two-dimensional image and the three-dimensional point cloud using the checkerboard calibration method can refer to existing technologies and are not limited here.
[0069] While unifying 2D images and 3D point clouds, hardware synchronization is also required. Specifically, this may include connecting the 2D camera and the 3D point cloud camera to the same trigger, and adjusting the 2D and 3D cameras so that they capture images from the same angle, avoiding the acquisition of images from different angles that could affect the recognition of unconventional components later.
[0070] 22. Use a preset express delivery identification and positioning model to identify and locate the express delivery in the initial two-dimensional image, and determine the detection box of the express delivery to be identified in the preset express delivery identification and positioning model and the express delivery type of the express delivery to be identified.
[0071] In the embodiments of this application, a preset express delivery recognition model can be used to recognize the initial two-dimensional image, so as to output the detection box corresponding to the express delivery to be recognized and determine the express delivery type corresponding to the express delivery to be recognized.
[0072] Specifically, when the preset parcel recognition model identifies parcels in the initial two-dimensional image, it generates a detection box that includes the parcel to be identified. The location of the detection box is the location of the identified parcel. At the same time, one detection box also represents one or more overlapping parcels. Therefore, multiple detection boxes can be obtained, and each detection box includes at least one parcel to be identified.
[0073] It should be noted that, in the embodiments of this application, the initial two-dimensional image can be an image acquired by a conventional camera or other device. Multiple packages can exist simultaneously in the initial two-dimensional image, therefore there can also be multiple detection boxes; and each detection box can also include multiple packages with partial overlap. When training the package recognition model, the model can be trained to determine whether a detection box corresponds to one or multiple packages while simultaneously recognizing the detection box. This application does not consider the situation where a package is completely occluded by other packages, resulting in it not being recognized in the initial two-dimensional image.
[0074] In the above embodiments, YOLO, EfficientNet, and Transformer series parcel recognition models can be used to recognize the initial two-dimensional image. Furthermore, while obtaining multiple detection boxes, the parcel type corresponding to each parcel can also be determined, including but not limited to foam boxes, sacks, irregularly shaped parcels, and ordinary parcels.
[0075] 23. Based on the detection frame, the initial two-dimensional image, and the initial three-dimensional point cloud, determine the outline of the package to be identified.
[0076] 24. Based on the outline and type of the package, determine whether the package to be identified is an unconventional package.
[0077] In the embodiments of this application, it is necessary not only to use the initial two-dimensional image and initial three-dimensional point cloud corresponding to the package to be identified for identification, but also to use the package outline and package type corresponding to the package to be identified to determine whether the package to be identified is an unconventional package.
[0078] The non-standard item identification method provided in this application first acquires an initial two-dimensional image and an initial three-dimensional point cloud corresponding to the item to be identified. The item is then identified and located in the initial two-dimensional image to determine the corresponding detection box and item type. The item outline is determined by combining the initial two-dimensional image, the initial three-dimensional point cloud, and the detection box. Finally, the item outline and item type are combined to determine whether the item is a non-standard item that is not excessively heavy or large. This application combines the two-dimensional image and three-dimensional point cloud corresponding to the item to be identified, along with the item outline and item type, to identify excessively heavy or large non-standard items. Furthermore, it utilizes computer vision methods for identification, which can effectively improve the accuracy and efficiency of non-standard item identification.
[0079] In the embodiments of this application, after determining the detection box corresponding to the package to be identified, it is necessary to further determine the outline of the package using the detection box. Since the initial two-dimensional image may simultaneously include multiple packages to be identified, multiple detection boxes can be obtained. For cases where multiple detection boxes are included, the following method is needed to determine the package outline. For example... Figure 3 The diagram shown is a flowchart illustrating an embodiment of determining the outline of a package provided in this application, which may include:
[0080] 31. Determine whether multiple detection boxes in the initial two-dimensional image overlap.
[0081] 32. If multiple detection boxes do not overlap, the initial two-dimensional image is clustered using a preset clustering method to obtain the outline of the package to be identified.
[0082] In the embodiments of this application, since one detection box corresponds to one package to be identified, it can first be determined whether multiple detection boxes overlap. If they do not overlap, the outline of the package corresponding to each package in each detection box can be directly determined using the initial two-dimensional image.
[0083] Specifically, each detection box is typically square and usually represented by (x, y, w, h); where x is the x-coordinate of the top-left corner of the detection box, y is the y-coordinate of the top-left corner of the detection box, w is the width of the detection box, and h is the height of the detection box. Determining the detection box also determines the positions of its four corner points. In this embodiment, the overlap between multiple detection boxes can be determined by judging the positional relationship of the four corner points of each detection box.
[0084] If there is no overlap between multiple detection boxes, the pixels in the initial 2D image are directly clustered to obtain the package outline corresponding to each package in each detection box. The specific steps for clustering pixels to obtain the package outline can refer to existing technologies and are not limited here.
[0085] 33. If multiple detection boxes overlap, remove noise from the initial 3D point cloud to obtain the 3D point cloud corresponding to the package to be identified.
[0086] 34. Fit the pixels in the 3D point cloud to obtain the initial outline of the package to be identified.
[0087] In the above embodiments, if there is no overlap between multiple detection boxes, the outline of the package is directly determined using the initial two-dimensional image; however, if there is overlap between multiple detection boxes, the outline of the package needs to be determined using the initial three-dimensional point cloud.
[0088] Specifically, if multiple detection boxes overlap, noise in the initial 3D point cloud is removed. Noise in the 3D point cloud refers to unclear or misplaced pixels caused by shadows, subsurface scattering, etc., during the acquisition of the 3D point cloud. This noise affects the determination of the package's outline, therefore it is necessary to remove noise from the initial 3D point cloud first. Different 3D point clouds produce noise for different reasons, resulting in different types of noise. Therefore, noise removal from the initial 3D point cloud can be performed according to the actual situation. The specific process can refer to existing technologies and is not limited here.
[0089] After removing noise from the initial 3D point cloud, a noise-free 3D point cloud can be obtained. In this embodiment, the pixels in the 3D point cloud can be fitted to obtain the outline of the package corresponding to the package to be identified.
[0090] Specifically, methods such as curvature fitting, image gradient solving, triangular meshing, and quadratic curvature fitting can be used to fit pixels in a 3D point cloud. The basic principle of pixel fitting is as follows: take any point (x, y, z) in the 3D point cloud, and then use this point to fit and optimize other surrounding points to obtain a locally optimal fitting surface; then combine multiple locally optimal fitting surfaces into an optimal fitting contour, which is the package contour. The steps for fitting pixels in a 3D point cloud to obtain the package contour can refer to existing technologies, and are not limited in the embodiments of this application.
[0091] 35. Segment the initial package outline to obtain the package outline corresponding to the package to be identified.
[0092] In the above embodiments, the initial outline of the package to be identified is obtained by fitting the pixels in the three-dimensional point cloud; however, since there is overlap between the multiple detection boxes corresponding to the package to be identified, it is necessary to segment the initial outline to obtain the package outlines corresponding to different packages in the package to be identified.
[0093] Specifically, the initial package outline can be input into a 3D deep network, such as a trained PointNet series neural network model, to obtain segmented point cloud information; that is, to determine which package each point cloud (x, y, z) belongs to, and thus obtain the package outline corresponding to each package.
[0094] In the embodiments of this application, simply identifying the outline of the package is insufficient to determine whether it is an irregular item; the actual size of the package is also required for judgment. Specifically, as shown... Figure 4 As shown, determining whether a package is an unconventional package can include:
[0095] 41. If multiple detection boxes do not overlap, map the outline of the package to be identified onto a three-dimensional coordinate system to determine the package size.
[0096] 42. If multiple detection frames overlap, determine the size of the package to be identified based on the package outline and the initial 3D point cloud.
[0097] 43. Determine whether the package to be identified is an irregular package based on its size and type.
[0098] In this embodiment, after determining the outline of the package, it is necessary to further obtain the size of the package; then, by combining the size and type of the package, it is determined whether the package to be identified is an unconventional item.
[0099] Specifically, while acquiring the initial 3D point cloud, the point cloud coordinates (x, y, z) corresponding to each pixel in the initial 3D point cloud are also acquired. Therefore, after determining the outline of the package, the point cloud coordinates corresponding to multiple pixels that make up the package outline can also be determined. Assuming that the plane formed by the X and Y axes in the 3D coordinate system is the length and width plane of the package, then the extreme points of the point cloud coordinates of multiple pixels that make up the package outline on the X and Y axes can be compared. Specifically, these could be the leftmost, rightmost, topmost, and bottommost pixels; or the two pixels with the largest and smallest x-coordinates, and the two pixels with the largest and smallest y-coordinates.
[0100] If we use the X-axis to measure the length of the package, then the length of the package is the x-coordinate of the rightmost point minus the x-coordinate of the leftmost point; similarly, we can calculate the width and height of the package.
[0101] The method described in the above embodiments involves overlapping detection boxes and using 3D point clouds to determine the size of the package. If the detection boxes do not overlap, the package outline obtained from clustering needs to be mapped to a 3D coordinate system to determine the package size. Specifically, a mapping matrix between 2D and 3D coordinate systems can be used to map the 2D coordinates corresponding to the 2D image to the 3D coordinate system, thereby obtaining the point cloud coordinates (x, y, z); and then the aforementioned method can be used to determine the package size data.
[0102] In the embodiments of this application, multiple detection boxes corresponding to the package to be identified are obtained using a two-dimensional image, and it is determined whether the multiple detection boxes overlap. Regardless of whether the multiple detection boxes overlap, the size of the package to be identified is determined using a three-dimensional point cloud.
[0103] In the embodiments of this application, after calculating the dimensions of the package, such as its length, width, and height, it is also necessary to combine the dimensions and type of the package to determine whether it is a regular package. Figure 5The diagram shown is a flowchart illustrating an embodiment of determining the type of regular mail based on the size and type of the mail, as provided in this application. It may include:
[0104] 51. Obtain multiple preset package size ranges corresponding to different package types.
[0105] 52. Based on the type of express shipment, determine the target size range corresponding to the express shipment from multiple size ranges.
[0106] In real-world parcel delivery scenarios, different types of parcels typically correspond to a range of parcel sizes. For example, the size of a parcel like clothing is completely different from the size of a parcel like fresh produce. Therefore, by historically analyzing the actual dimensions of different parcel types, we can determine the corresponding parcel size ranges for each type, thus obtaining multiple parcel size ranges.
[0107] In this embodiment of the application, while determining the detection frame corresponding to the express delivery, the type of express delivery corresponding to the express delivery is also determined. Therefore, the target express delivery size range corresponding to the express delivery can be directly determined within multiple express delivery size ranges based on the express delivery type.
[0108] 53. Based on the size of the parcel and the target parcel size range, determine whether the parcel to be identified is a regular parcel.
[0109] After determining the target package size range for the package to be identified, it can be judged whether the size of the package to be identified falls within the target package size range. If the package size is within the target package size range, it can be determined that the package is a regular package; if the package size is not within the target package size range, it can be determined that the package is a non-regular package.
[0110] In actual express delivery scenarios, the larger the size of the package, the greater its weight. If the actual size of the package exceeds the target size range, it is usually considered overweight. Oversized and overweight packages can damage the conveyor system, therefore these packages are classified as non-standard items. These packages are screened out to prevent damage to the conveyor system.
[0111] In the embodiments of this application, it is assumed that the larger the size of the package, the greater its weight; therefore, the weight of the package is not used to determine whether it is an unconventional package. However, to improve the accuracy of identifying unconventional packages, the actual weight of the package can also be used to determine whether it is an unconventional package.
[0112] like Figure 6 As shown, the non-standard part identification device provided in this application embodiment may include:
[0113] The image acquisition unit 601 is used to acquire the initial two-dimensional image and the initial three-dimensional point cloud corresponding to the express package to be identified.
[0114] The parcel identification and positioning unit 602 is used to identify and locate the parcel in the initial two-dimensional image by means of a preset parcel identification and positioning model, and to determine the detection box of the parcel to be identified in the preset parcel identification and positioning model and the parcel type of the parcel to be identified.
[0115] The package outline determination unit 603 is used to determine the package outline of the package to be identified based on the detection frame, the initial two-dimensional image and the initial three-dimensional point cloud.
[0116] The non-standard item identification unit 604 is used to determine whether the item to be identified is a non-standard item based on the item outline and the item type.
[0117] The non-standard item identification device provided in this application first acquires an initial two-dimensional image and an initial three-dimensional point cloud corresponding to the item to be identified. It then identifies and locates the item in the initial two-dimensional image, determines the corresponding detection box and item type, and combines the initial two-dimensional image, initial three-dimensional point cloud, and detection box to determine the item's outline. Finally, it combines the item outline and item type to determine whether the item is a non-standard item that is not excessively heavy or large. This application combines the two-dimensional image and three-dimensional point cloud corresponding to the item to be identified, along with the item outline and item type, to identify excessively heavy or large non-standard items. Furthermore, it utilizes computer vision methods for identification, which can effectively improve the accuracy and efficiency of non-standard item identification.
[0118] In some embodiments, the initial two-dimensional image includes multiple detection boxes; the package contour determination unit 603 can specifically be used for:
[0119] Determine whether multiple detection boxes in the initial two-dimensional image overlap.
[0120] If the multiple detection boxes do not overlap, the initial two-dimensional image is clustered using a preset clustering method to obtain the outline of the package to be identified.
[0121] In some embodiments, the express delivery contour determination unit 603 may specifically be used for:
[0122] If multiple detection boxes overlap, then the noise in the initial 3D point cloud is removed to obtain the 3D point cloud of the package to be identified.
[0123] The pixels in the three-dimensional point cloud are fitted to obtain the initial outline of the package to be identified.
[0124] The initial package outline is segmented to obtain the package outline of the package to be identified.
[0125] In some embodiments, the non-standard component identification unit 604 may specifically be used for:
[0126] If the multiple detection boxes do not overlap, the outline of the package to be identified is mapped onto the three-dimensional coordinate system to determine the package size corresponding to the package to be identified;
[0127] Based on the package size and package type, determine whether the package to be identified is an unconventional package.
[0128] In some embodiments, the non-standard component identification unit 604 may specifically be used for:
[0129] If the multiple detection boxes overlap, the size of the package to be identified is determined based on the package outline and the initial 3D point cloud.
[0130] Based on the package size and package type, determine whether the package to be identified is a regular package.
[0131] In some embodiments, the non-standard component identification unit 604 may specifically be used for:
[0132] Obtain multiple preset package size ranges corresponding to different package types;
[0133] Based on the type of parcel, determine the target parcel size range corresponding to the parcel to be identified from multiple parcel size ranges;
[0134] Based on the size of the package and the target package size range, determine whether the package to be identified is a standard item.
[0135] This application also provides a computer device that integrates any of the non-standard component identification devices provided in this application, the computer device comprising:
[0136] One or more processors;
[0137] Memory; and
[0138] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor in the steps of the unconventional component identification method described in any of the embodiments of the above-described unconventional component identification method.
[0139] This application also provides a computer device that integrates any of the non-standard component identification devices provided in this application. For example... Figure 7 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:
[0140] The computer device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 7 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:
[0141] The processor 701 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.
[0142] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.
[0143] The computer device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0144] The computer device may also include an input unit 704, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
[0145] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 runs the application programs stored in the memory 702 to realize various functions, as follows:
[0146] Acquire the initial 2D image and initial 3D point cloud of the parcel to be identified; perform parcel identification and localization on the initial 2D image using a preset parcel identification and localization model to determine the detection box and parcel type of the parcel in the preset parcel identification and localization model; determine the parcel outline based on the detection box, the initial 2D image, and the initial 3D point cloud; and determine whether the parcel is an unconventional parcel based on the parcel outline and parcel type.
[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.
[0148] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, which is loaded by a processor to execute the steps in any of the non-standard device identification methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:
[0149] Acquire the initial 2D image and initial 3D point cloud of the parcel to be identified; perform parcel identification and localization on the initial 2D image using a preset parcel identification and localization model to determine the detection box and parcel type of the parcel in the preset parcel identification and localization model; determine the parcel outline based on the detection box, the initial 2D image, and the initial 3D point cloud; and determine whether the parcel is an unconventional parcel based on the parcel outline and parcel type.
[0150] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.
[0151] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.
[0152] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0153] The above provides a detailed description of a method, apparatus, computer device, and storage medium for identifying unconventional components according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for identifying unconventional components, characterized in that, The method includes: Acquire the initial two-dimensional image and initial three-dimensional point cloud of the package to be identified; The initial two-dimensional image is used to identify and locate the package by means of a preset package identification and location model, thereby determining the detection box of the package to be identified in the preset package identification and location model and the package type of the package to be identified. The outline of the package to be identified is determined based on the detection boxes, the initial two-dimensional image, and the initial three-dimensional point cloud. This includes: determining whether multiple detection boxes in the initial two-dimensional image overlap; if the multiple detection boxes do not overlap, clustering the initial two-dimensional image to obtain the outline of the package to be identified; if the multiple detection boxes overlap, fitting and segmenting the pixels in the three-dimensional point cloud to obtain the outline of the package to be identified. Based on the package outline and the package type, determine whether the package to be identified is an unconventional package; including: if the multiple detection boxes do not overlap, map the package outline of the package to be identified onto a three-dimensional coordinate system to determine the package size corresponding to the package to be identified; if the multiple detection boxes overlap, determine the package size of the package to be identified based on the package outline and the initial three-dimensional point cloud; and determine whether the package to be identified is a regular package based on the package size and the package type.
2. The method for identifying non-standard parts according to claim 1, characterized in that, Before acquiring the initial two-dimensional image and initial three-dimensional point cloud corresponding to the package to be identified, the method further includes: Obtain the two-dimensional image and three-dimensional point cloud of the chessboard of the preset chessboard calibration board; Establish a two-dimensional coordinate system corresponding to the two-dimensional image, and a three-dimensional coordinate system corresponding to the three-dimensional point cloud; Determine the first coordinates of the corner points of the chessboard in the two-dimensional image of the chessboard in the two-dimensional coordinate system; Determine the second coordinates of the corner points of the chessboard grid in the three-dimensional point cloud of the chessboard grid in the three-dimensional coordinate system; The mapping matrix between the two-dimensional image and the three-dimensional point cloud is determined based on the first coordinate and the second coordinate.
3. The method for identifying non-standard parts according to claim 1, characterized in that, If multiple detection boxes overlap, the pixels in the 3D point cloud are fitted and segmented to obtain the outline of the package to be identified, including: Noise is removed from the initial 3D point cloud to obtain the 3D point cloud of the package to be identified; The pixels in the three-dimensional point cloud are fitted to obtain the initial outline of the package to be identified. The initial package outline is segmented to obtain the package outline of the package to be identified.
4. The method for identifying non-standard parts according to any one of claims 1-3, characterized in that, The step of determining whether the package to be identified is a regular package based on the package size and package type includes: Obtain multiple preset size ranges for different express delivery types; Based on the type of express delivery, determine the target size range of the express delivery to be identified from the plurality of express delivery size ranges; Based on the dimensions of the package and the target package size range, determine whether the package to be identified is a standard package.
5. A device for identifying unconventional parts, characterized in that, The device includes: The image acquisition unit is used to acquire the initial two-dimensional image and initial three-dimensional point cloud of the package to be identified; The express delivery identification and positioning unit is used to identify and locate the express delivery in the initial two-dimensional image through a preset express delivery identification and positioning model, and to determine the detection box of the express delivery to be identified in the preset express delivery identification and positioning model and the express delivery type of the express delivery to be identified; A package outline determination unit is used to determine the package outline of the package to be identified based on the detection boxes, the initial two-dimensional image, and the initial three-dimensional point cloud; including: determining whether multiple detection boxes in the initial two-dimensional image overlap; if multiple detection boxes do not overlap, clustering the initial two-dimensional image to obtain the package outline of the package to be identified; if multiple detection boxes overlap, fitting and segmenting the pixels in the three-dimensional point cloud to obtain the package outline of the package to be identified. An unconventional shipment identification unit is used to determine whether a shipment to be identified is an unconventional shipment based on the shipment outline and the shipment type; including: if the multiple detection boxes do not overlap, mapping the shipment outline of the shipment to be identified onto a three-dimensional coordinate system to determine the shipment size corresponding to the shipment to be identified; if the multiple detection boxes overlap, determining the shipment size of the shipment to be identified based on the shipment outline and the initial three-dimensional point cloud; and determining whether the shipment to be identified is a regular shipment based on the shipment size and the shipment type.
6. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the non-routine component identification method as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps in the non-standard component identification method as described in any one of claims 1 to 4.
Citation Information
Patent Citations
Object recognition and positioning method and device based on two-dimensional-three-dimensional fusion features
CN112070838A
Contour extraction method, device and equipment, program product and storage medium
CN113420735A
