Object volume detection method and device, computer equipment and storage medium

Abstract

The invention relates to an object volume detection method and device, computer equipment and a storage medium. The method comprises the steps of extracting plane data and foreground data in point cloud data of a to-be-detected object, performing region growth on the foreground data, determining supporting plane data based on an inclusion relation between the grown foreground data and the plane data, and determining a shape type of the to-be-detected object based on a point cloud shape of the foreground data. And performing volume calculation on the point cloud data based on a volume algorithm corresponding to the shape type to obtain the volume of the to-be-detected object. Compared with a traditional detection mode that only the volume of the regular box body is detected, the scheme performs shape detection on the to-be-detected object by combining the point cloud data of the to-be-detected object, so that the volume of the to-be-detected object is calculated by adopting an algorithm conforming to the shape of the to-be-detected object, and the volume calculation accuracy of objects such as logistics express items is improved.

Description

Technical Field

[0001] This application relates to the technical field of express delivery detection, and particularly to a method, device, computer device, storage medium, and computer program product for detecting the volume of an object. Background Art

[0002] In the logistics transportation business, there are a large number of express packages, and the volume detection of objects such as express packages is involved in the entire process of the logistics business. Therefore, how to more accurately detect the volume of objects such as express packages is an urgent problem to be solved. Currently, the methods for detecting the volume of objects such as express packages usually only detect express packages in regular boxes, and the detection accuracy for express package objects in non-regular boxes is relatively low.

[0003] Therefore, the current methods for detecting the volume of objects such as express packages have the defect of low detection accuracy. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting the volume of an object that can improve the detection accuracy.

[0005] In a first aspect, this application provides a method for detecting the volume of an object, and the method includes:

[0006] Obtain the point cloud data collected by an image acquisition device for an object to be detected, and extract each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device;

[0007] Perform region growing on the foreground data according to the point cloud data, and determine the support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data;

[0008] Determine the shape type of the object to be detected according to the point cloud shape of the foreground data;

[0009] Obtain the volume algorithm corresponding to the shape type, and determine the volume of the object to be detected according to the volume algorithm, the foreground data, and the support plane data.

[0010] In one of the embodiments, the extraction of each plane data in the point cloud data includes:

[0011] Extract each original plane data in the point cloud data, and obtain the normal vector of each original plane data;

[0012] For each original plane data, obtain the connection line of the center points of the original plane data and each other original plane data;

[0013] Obtain other original plane data from the various other original plane data, where the included angle between the line connecting the center points and the normal vector of the original plane data is within a first preset angle range, and the included angle between the normal vector and the normal vector of the original plane data is less than or equal to a second preset angle, as coplanar plane data;

[0014] Merge the original plane data with the corresponding coplanar plane data to obtain the plane data in the point cloud data.

[0015] In one embodiment, the step of extracting foreground data from the point cloud data includes:

[0016] Obtain the plane data in the various plane data that has an intersection with a preset central window as candidate plane data; the preset central window represents a window with a preset size set at the center of the acquisition area of the image acquisition device;

[0017] Obtain the candidate plane data with the closest distance to the image acquisition device among the candidate plane data as foreground data.

[0018] In one embodiment, the step of extracting foreground data from the point cloud data further includes:

[0019] Obtain the top layer point data in the point cloud data. If the top layer point data is not a plane, determine the point data with the closest distance to the image acquisition device from the top layer point data as foreground data.

[0020] In one embodiment, the region growing of the foreground data according to the point cloud data includes:

[0021] Obtain each point data adjacent to the foreground data in the point cloud data;

[0022] Use the point data and the foreground data as new foreground data, and return to the step of obtaining each point data adjacent to the foreground data in the point cloud data until a preset end condition is met, and then obtain the grown foreground data.

[0023] In one embodiment, the determination of the support plane data in the various plane data according to the inclusion relationship between the grown foreground data and the various plane data includes:

[0024] Determine each candidate support plane data from the various plane data according to the foreground data;

[0025] Obtain the first projection of the grown foreground data and the second projection of each candidate support plane data;

[0026] For each candidate support plane data, if the first projection is within the range of the second projection of the candidate support plane data, determine the candidate support plane data as the support plane data.

[0027] In one embodiment, the determining of each candidate support plane data from the respective plane data according to the foreground data includes:

[0028] If the foreground data is a plane, obtain the plane data parallel to the foreground data from the respective plane data as the candidate support plane data;

[0029] If the foreground data is non-planar, obtain the plane data that intersects with the boundary of the view area from the respective plane data as the candidate support plane data; the boundary of the view area represents the boundary of the view area of the image acquisition device.

[0030] In one embodiment, the determining of the shape type of the object to be detected according to the point cloud shape of the foreground data includes:

[0031] If the foreground data is a plane, determine the shape type of the object to be detected as a regular type;

[0032] If the foreground data is non-planar, determine the shape type of the object to be detected as an irregular type.

[0033] In one embodiment, the obtaining of the volume algorithm corresponding to the shape type includes:

[0034] If the shape type is regular, obtain the first projection of the foreground data after growth and the third projection of the foreground data;

[0035] Determine the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection.

[0036] In one embodiment, the determining of the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection includes:

[0037] If the area ratio of the first projection to the third projection is less than a preset threshold, determine the object to be detected as a regular object;

[0038] Obtain the first preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0039] In one embodiment, the determining of the regular type corresponding to the object to be detected according to the area ratio of the first projection to the third projection, and obtaining the volume algorithm corresponding to the regular type includes:

[0040] If the area ratio of the first projection to the third projection is greater than or equal to a preset threshold, determine that the rule type of the object to be detected is a quasi-regular object;

[0041] Obtain the second preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0042] In one embodiment, the determining the volume of the object to be detected according to the volume algorithm, the foreground data, and the support plane data includes:

[0043] If the shape type is irregular, obtain the length and width of the support plane data of the object to be detected;

[0044] Determine the height of the object to be detected according to the grown foreground data;

[0045] Determine the volume of the object to be detected according to the product of the length, width, and height.

[0046] In a second aspect, the present application provides an object volume detection device, which includes:

[0047] An acquisition module, configured to acquire point cloud data collected by an image acquisition device for an object to be detected, and extract each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device;

[0048] A growth module, configured to perform region growth on the foreground data according to the point cloud data, and determine the support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data;

[0049] A determination module, configured to determine the shape type of the object to be detected according to the point cloud shape of the foreground data;

[0050] A detection module, configured to obtain the volume algorithm corresponding to the shape type, and determine the volume of the object to be detected according to the volume algorithm and the point cloud data.

[0051] In a third aspect, the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program implements the steps of the above method when executed by a processor.

[0053] In a fifth aspect, the present application provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the above method.

[0054] For the above object volume detection method, device, computer device, storage medium and computer program product, by extracting the plane data and foreground data in the point cloud data of the object to be detected, performing region growing on the foreground data, determining the support plane data based on the inclusion relationship between the grown foreground data and the plane data, determining the shape type of the object to be detected based on the point cloud shape of the foreground data, and calculating the volume of the point cloud data based on the volume algorithm corresponding to the shape type, the volume of the object to be detected is obtained. Compared with the traditional detection method that only detects the volume of regular boxes, this solution combines the point cloud data of the object to be detected, performs shape detection on the object to be detected, and thus calculates the volume of the object to be detected using an algorithm that conforms to the shape of the object to be detected, improving the accuracy of volume calculation for objects such as logistics express items. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is an application environment diagram of the object volume detection method in an embodiment;

[0056] Figure 2 It is a flowchart of the object volume detection method in an embodiment;

[0057] Figure 3 It is a flowchart of the object volume detection method in another embodiment;

[0058] Figure 4 It is a flowchart of the object shape determination step in an embodiment;

[0059] Figure 5 It is a flowchart of the object volume determination step in an embodiment;

[0060] Figure 6 It is a structural block diagram of the object volume detection device in an embodiment;

[0061] Figure 7 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] The object volume detection method provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal communicates with the image acquisition device through the network. The image acquisition device can collect the point cloud data of the object in the detection area and send the point cloud data to the terminal, and the terminal can determine the volume of the object based on the above point cloud data. Among them, the terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, and tablet computers.

[0064] In one embodiment, as Figure 2 shown, a method for detecting the volume of an object is provided. Taking the terminal in Figure 1 as an example, the method includes the following steps:

[0065] Step S202: Obtain the point cloud data collected by the image acquisition device for the object to be detected, and extract each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device.

[0066] Among them, the image acquisition device can be a structured light camera or a tof (Time-of-Flight) camera, and the image acquisition device can collect the 3D stereo information of the object to be detected. For example, taking the object to be detected as an express delivery in the logistics business, when the image acquisition device performs a photo shooting action on the express delivery, the point cloud data of the express delivery can be collected as the point cloud data of the object to be detected. Among them, the point cloud data includes multiple point data, and the point cloud data can represent the spatial shape information of the object to be detected through multiple point data. Among them, the above object to be detected can be placed in the viewing area of the image acquisition device according to preset requirements. For example, the object to be detected can be placed in the center of the viewing area of the image acquisition device, and the distance between the object to be detected and the image acquisition device is more than 5 cm; the above object to be detected needs to be fully displayed in the viewing area of the image acquisition device; the connection line from the image acquisition device to the support surface of the object to be detected is preferably perpendicular to the support surface of the express delivery, and the support surface is a flat plane, that is, the object to be detected needs to be placed on the above viewing area with a flat plane as the support surface, and there should be no sundries around the above object to be detected to interfere.

[0067] After obtaining the point cloud data of the object to be detected by the image acquisition device, the plane data and foreground data can be extracted from the point cloud data. Among them, there can be multiple planes on the appearance of the above object to be detected, so there can be multiple plane data. The above foreground data represents the point cloud data closest to the image acquisition device in the point cloud data. Among them, the foreground data can be composed of a plane formed by multiple point data in the point cloud data, or can be represented by a single point data. When the foreground data is a plane composed of multiple point data, the foreground data can be called the top surface.

[0068] Specifically, the plane data in the above point cloud data can be obtained through the PEAC plane extraction algorithm. Among them, the terminal extracts all the plane data in the point cloud data through the PEAC algorithm. This extraction can be a segmentation behavior. For example, the terminal identifies the maximum contour of each plane in the point cloud data and generates the corresponding plane data in space for each plane based on the maximum contour of each plane. The terminal can fill 255 in each plane data to indicate the existence of the plane, and fill the rest with 0. In addition, the terminal can also obtain the 3D center point coordinates, normal vectors, and the distances from each plane data to the image acquisition device in the above-extracted plane data.

[0069] Among them, the above object to be detected can have various shapes. For objects with different shapes, the point cloud shapes of their corresponding foreground data can be different. Then the above foreground data can have various point cloud shapes, including planes or single points, etc. When the point cloud shape of the foreground data is a plane, the terminal can identify the foreground data from the plane data in the point cloud data.

[0070] Step S204: Perform region growing on the foreground data according to the point cloud data, and determine the support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data.

[0071] Among them, the foreground data represents the point cloud data closest to the above image acquisition device. The terminal can determine the support plane data of each plane data in the above point cloud data by performing region growing on the foreground data. The support plane data represents the plane on the surface of the object to be detected for supporting the object to be detected to be placed in the acquisition area. Therefore, the support plane needs to be large enough to achieve the support effect. Therefore, the terminal can determine the support plane data in each plane data through the above region growing. For example, the terminal performs region growing on the foreground data based on the above point cloud data to obtain the grown foreground data. Among them, the above region growing can be performed by selecting the points adjacent to the points in the foreground data and including them in the point cloud set of the foreground data.

[0072] In one embodiment, performing region growing on the foreground data according to the point cloud data includes: obtaining each point data adjacent to the foreground data in the point cloud data; using each point data and the foreground data as new foreground data, and returning to the step of obtaining each point data adjacent to the foreground data in the point cloud data until the preset end condition is met, and then obtaining the grown foreground data.

[0073] In this embodiment, the terminal can obtain multiple point data adjacent to the foreground data from the above point cloud data, for example, multiple point data adjacent to the boundary points of the foreground data, and incorporate each point data into the current foreground data to form new foreground data. The terminal can return to the step of obtaining each point data adjacent to the foreground data in the point cloud data and perform the process of incorporating adjacent points into the foreground data multiple times until the preset end condition is met, and then obtain the grown foreground data. The grown foreground data can represent a part of the surface at the top of the object to be detected.

[0074] Among them, the above preset end condition can represent the amplitude of region growth, and the amplitude of region growth can be determined by a preset threshold. For example, when the adjacent point is a point of the above plane data, or the projection of the adjacent point coincides with the point of the above plane data, etc.

[0075] Since the support plane needs to be large enough to achieve the support function, after the terminal grows the above foreground data, it can determine the support plane data in each plane data by detecting the inclusion relationship between the grown foreground data and each plane data. For example, the terminal can determine the support plane data by comparing the projection of the grown foreground data and the projection of each plane data. Among them, the support plane data can enclose the above grown foreground data, that is, it means that the support plane data has a larger projected area than the grown foreground data, so that it can represent that the support plane can achieve the function of supporting the object to be detected. Thus, the terminal can determine the plane data that can contain the above grown foreground data as the support plane data.

[0076] Step S206, determine the shape type of the object to be detected according to the point cloud shape of the foreground data.

[0077] Among them, the foreground data can contain one or more point data. Then the foreground data can be presented in various point cloud shapes, such as presented as a plane or a non-plane. Among them, when the foreground data is presented as a non-plane, it can be presented as individual points. Different point cloud shapes can represent different shapes of the object to be detected. That is, the terminal can determine the spatial shape of the object to be detected by detecting the point cloud shape of the foreground data. The terminal can obtain the point cloud shape of the above foreground data and determine the shape type of the object to be detected based on this point cloud shape.

[0078] As an example, in one embodiment, according to the point cloud shape of the foreground data, determining the shape type of the object to be detected includes: if the foreground data is a plane, determining that the shape type of the object to be detected is regular; if the foreground data is non-planar, determining that the shape type of the object to be detected is irregular. In this embodiment, the terminal can detect the point cloud shape of the foreground data. If the terminal detects that the foreground data is a plane, the terminal can determine that the shape type of the object to be detected is regular. Among them, the regular type means that the object to be detected has a top surface and a bottom surface presented by plane data, that is, there is a supporting plane data and a plane data as the top surface, and the supporting plane data can enclose the above foreground data. Specifically, taking the object to be detected as a courier package as an example, the regular type means that the upper surface (top surface) and the bottom surface of the courier package are parallel and approximately the same in size and shape. In some embodiments, it also includes a courier package of a quasi-regular type, and the quasi-regular type means that the upper surface (top surface) and the bottom surface of the courier package are parallel but the upper surface is significantly smaller than the bottom surface.

[0079] If the terminal detects that the above foreground data is non-planar, such as a point or a curved surface, the terminal can determine that the shape type of the above object to be detected is irregular. Among them, the irregular type means that the above object to be detected only has a bottom surface presented in a plane state, that is, the above supporting plane data, and there is no top surface presented in a plane state, or the above supporting plane data cannot contain the above foreground data. Specifically, taking the object to be detected as a courier package as an example, the irregular type means that the courier package has no upper surface (top surface) parallel to the bottom surface.

[0080] Step S208, obtaining a volume algorithm corresponding to the shape type, and determining the volume of the object to be detected according to the volume algorithm, the foreground data, and the supporting plane data.

[0081] Among them, objects to be detected with different shape types can correspond to different volume algorithms. For example, for a courier package belonging to the regular type, the terminal can determine the spatial three-dimensional structure to which the courier package belongs, such as a cuboid, a cylinder, a trapezoid, etc. Thus, the terminal can obtain the volume algorithm of the spatial three-dimensional structure to which the above regular-type courier package belongs as the volume algorithm of the object to be detected; for a courier package belonging to the irregular type, after the terminal determines the spatial structure of the irregular-type courier package, it can use the product of length, width, and height as the volume algorithm of the irregular-type object to be detected. Thus, the terminal can obtain the volume of the object to be detected based on the above volume algorithm, the foreground data, and the supporting plane data.

[0082] For example, for a regular object to be detected, the terminal can use the foreground data as the top surface of the object to be detected and the support plane data as the bottom surface of the object to be detected. Thus, the terminal can combine the above volume algorithm to obtain corresponding data from the top surface and the bottom surface to determine the volume of the object to be detected. For an irregular object to be detected, the terminal can use the support plane data as the ground surface of the object to be detected and the foreground data after region growing as the upper surface of the object to be detected. If the upper surface is non-planar, the terminal can determine the height of the irregular object to be detected based on the distance between the upper surface and the bottom surface, determine the length and width of the irregular object to be detected based on the bottom surface, and thus the terminal can determine the volume of the irregular object to be detected based on the product of the above length, width, and height.

[0083] In the above object volume detection method, by extracting each plane data and foreground data in the point cloud data of the object to be detected, performing region growing on the foreground data, determining the support plane data based on the inclusion relationship between the foreground data after growing and each plane data, determining the shape type of the object to be detected based on the point cloud shape of the foreground data, and performing volume calculation on the point cloud data based on the volume algorithm corresponding to the shape type, the volume of the object to be detected is obtained. Compared with the traditional detection method that only detects the volume of regular boxes, this solution performs shape detection on the object to be detected by combining the point cloud data of the object to be detected, and thus uses an algorithm that conforms to the shape of the object to be detected to perform volume calculation on the object to be detected, improving the accuracy of volume calculation for objects such as logistics express items.

[0084] In one embodiment, extracting each plane data in the point cloud data includes: extracting each original plane data in the point cloud data and obtaining the normal vector of each original plane data; for each original plane data, obtaining the connection line of the center points between the original plane data and each other original plane data; obtaining, from each other original plane data, the other original plane data whose included angle between the connection line of the center points and the normal vector of the original plane data is within the first preset angle range and the included angle between the normal vector and the normal vector of the original plane data is less than or equal to the second preset angle as the coplanar plane data; and merging the original plane data with the corresponding coplanar plane data to obtain the plane data in the point cloud data.

[0085] In this embodiment, there may be multiple approximately coplanar and parallel planes on the object to be detected, and these planes can be regarded as the same plane. The terminal can determine the plane data corresponding to the object to be detected through a merging method. For example, the terminal can extract the original plane data in the point cloud data. Here, the original plane data represents the plane data that has not been merged. The terminal can also obtain the normal vectors of the above-mentioned original plane data. For each original plane data, the terminal can obtain the connection lines between the center points of the original plane data and each other original plane data. Here, the connection line between the center points represents the connection line between the 3D center point of the original plane data and the 3D center points of each other original plane data. The terminal can also obtain, from each other original plane data, the other original plane data whose included angle between the connection line between the center points and the normal vector of the original plane data is within the first preset angle range and the included angle between the normal vector and the normal vector of the original plane data is less than or equal to the second preset angle as the coplanar plane data. The first preset angle range and the second preset angle can be set according to the actual situation. For example, the first preset angle range is set to 87 degrees to 93 degrees, which is approximately orthogonal, and the second preset angle is set to 5 degrees, which is approximately parallel.

[0086] The terminal can determine one or more coplanar plane data from the above-mentioned other original plane data, so that the terminal can obtain the plane data in the point cloud data by merging the original plane data with the corresponding coplanar plane data.

[0087] Specifically, the above-mentioned original plane data can be stored in set A. When the foreground data of the object to be detected is a plane, the original plane data in set A is parallel to the foreground data. Taking the original plane data a in set A as an example, if the terminal detects that there is another original plane data b in set A, and the included angle between the normal vector of the other original plane data b and the normal vector of the original plane data a is less than 5 degrees (the two planes are approximately parallel), and the connection line between the center points of the original plane data a and the other original plane data b is approximately orthogonal to the normal vector of the original plane data a, for example, the included angle is greater than 87 degrees (approximately coplanar), then the terminal can merge the two planes a and b to obtain the plane data. The terminal can perform multiple merges in set A, and finally obtain the merged plane data and store these plane data in set A1.

[0088] In addition, in some embodiments, the terminal can also select the plane data that can include the above-mentioned foreground data from set A1 to form set A2. The plane data in set A2 is sorted from near to far based on the distance from the foreground data, so that the terminal can determine the support plane data from the plane data in set A2. If the terminal cannot determine the plane data that can include the foreground data from set A1, the terminal can determine that the object to be detected is an irregular object.

[0089] Through the above embodiments, the terminal can form the plane data in the point cloud data of the object to be detected by merging coplanar planes, and then the terminal determines the shape type of the object to be detected based on the plane data, and determines the volume of the object to be detected based on the volume algorithm corresponding to the shape type, improving the accuracy of volume detection.

[0090] In one embodiment, the steps of extracting foreground data from the point cloud data include: obtaining the plane data in each plane data that has an intersection with a preset central window as candidate plane data; the preset central window represents a window of a preset size set at the center of the acquisition area of the image acquisition device; obtaining the candidate plane data that is closest to the image acquisition device among the candidate plane data as foreground data.

[0091] In this embodiment, the foreground data in the above point cloud data can be a plane. For an object to be detected whose foreground data is a plane, in the viewing area of the above image acquisition device, a preset central window can be preset, and the size of the central window can be set according to the actual situation, for example, 4 pixels * 4 pixels. The terminal can obtain the plane data in each plane data in the above point cloud data that has an intersection with the preset central window as candidate plane data. Among them, the candidate plane data represents a plane whose position is approximately at the top center of the object to be detected, and these planes are highly likely to be the top surface of the object to be detected. The terminal can obtain the candidate plane data that is closest to the image acquisition device among the candidate plane data as the above foreground data.

[0092] Specifically, taking the object to be detected as an express delivery item as an example, after the terminal extracts each plane data in the point cloud data through the PEAC plane extraction algorithm, the plane data that has an intersection with the above preset central window can be used as candidate plane data, and the candidate plane data with the smallest absolute value of the distance to the image acquisition device is taken as the top surface of the express delivery item, that is, the above foreground data. In addition, if the terminal detects that the number of the above candidate plane data is zero, it means that there is no plane data that can be used as the top surface in the above plane data, then the terminal can determine that the shape type of the object to be detected is irregular.

[0093] Among them, for an irregular object to be detected, the terminal can determine the foreground data by extracting point data. For example, in one embodiment, the steps of extracting foreground data from the point cloud data further include: obtaining the top layer point data in the point cloud data, and if the top layer point data is not a plane, determining the point data that is closest to the image acquisition device from the top layer point data as foreground data.

[0094] In this embodiment, the terminal can obtain the top point data in the above point cloud data. Among them, the above point cloud data includes a plurality of point data, and the top point data represents the point data with the highest height in the above point cloud data. Then, there may be one or more top point data. The terminal can detect whether the above top point data can form a plane. If not, the terminal can determine the point data closest to the image acquisition device from the top point data as the foreground data. Among them, the above point data closest to the image acquisition device is also called seed data and can be used as the starting point during region growing.

[0095] Through the above embodiment, the terminal can detect the top surface of a regular object by using a preset central window, detect the vertices of an irregular object by using the top point data, and determine the plane or point data in the point cloud data of the object to be detected as the foreground data by detecting the distance between the top surface or vertices and the image acquisition device, improving the accuracy of foreground data determination.

[0096] In one embodiment, determining the support plane data in each plane data according to the inclusion relationship between the foreground data after growth and each plane data includes: determining each candidate support plane data from each plane data according to the foreground data; obtaining the first projection of the foreground data after growth and the second projection of each candidate support plane data; for each candidate support plane data, if the first projection is within the range of the second projection of the candidate support plane data, determining the candidate support plane data as the support plane data.

[0097] In this embodiment, the terminal can determine the support plane data in each plane data by detecting whether each plane data can contain the foreground data after region growing. The terminal can determine each candidate support plane data from each plane data according to the above foreground data. Among them, the candidate support plane data represents the data that may be the support plane of the object to be detected. For different point cloud shapes of the foreground data, the terminal can adopt different methods to determine the candidate support plane data.

[0098] For example, in one embodiment, determining each candidate support plane data from each plane data according to the foreground data includes: if the foreground data is a plane, obtaining the plane data parallel to the foreground data in each plane data as the candidate support plane data; if the foreground data is non-planar, obtaining the plane data in each plane data that has an intersection with the boundary of the viewing area as the candidate support plane data; the boundary of the viewing area represents the boundary of the viewing area of the image acquisition device.

[0099] In this embodiment, the terminal can determine the screening strategy for candidate support plane data based on whether the foreground data is planar. Among them, if the terminal detects that the foreground data is planar, it indicates that the top surface of the object to be detected exists in a planar state. The terminal can obtain the plane data parallel to the foreground data from the above-mentioned plane data as the candidate support plane data. Specifically, it can be represented as the above set A2. Each plane data in the set A2 is sorted from near to far based on the distance from the foreground data, so that the terminal can determine the support plane data from each plane data in the set A2.

[0100] The above image acquisition device has a corresponding viewfinder area for performing point cloud acquisition on the object to be detected, and the area has corresponding boundaries, which can be called the viewfinder area boundary. If the terminal detects that the above foreground data is non-planar, the terminal can obtain the plane data that intersects with the viewfinder area boundary from each plane data as the candidate support plane data. Among them, intersecting with the viewfinder area boundary means that the above viewfinder area boundary is within the range of the plane data. Since the range of the viewfinder area boundary is large, for an irregular object to be detected, the plane data that intersects with the boundary of the viewfinder area is likely to be the support plane data.

[0101] After the terminal determines the candidate support plane data, it can perform region growing on the foreground data and obtain the first projection of the foreground data after growth, as well as the second projection of each of the above candidate support plane data. For each candidate support plane data, if the terminal detects that the above first projection is within the range of the second projection of the candidate support plane data, it indicates that the above candidate support plane data can contain the foreground data after region growing, then the terminal can determine that the candidate support plane is the support plane data. Among them, the above candidate support planes can be sorted from near to far according to the distance from the top of the point cloud data, so that the terminal can select the candidate support plane data from near to far to detect whether it contains the first projection. When the terminal first detects that the second projection of the candidate support plane data can contain the above first projection, the terminal can use the candidate support plane data as the support plane data.

[0102] Specifically, taking the object to be detected as an express delivery item as an example, the above candidate support plane data can be stored in the above set A2. The candidate support plane data in set A2 is tried as the support plane of the express delivery item in the order of the distance from the top of the point cloud data from near to far. Assume that there is a plane a in set A2. The terminal obtains the point cloud data of plane a, that is, the candidate support plane data of plane a. If the above foreground data is a plane, it indicates that the express delivery item has a top surface. Then the terminal performs region growing on the top surface and determines whether the projection of plane a can enclose the projection of the top surface after region growing. If it cannot enclose, continue to select a plane in set A2 that is farther from the top surface as the next candidate support plane data, and repeat the above steps until the selected candidate support plane data can enclose the top surface after the above region growing, and determine the selected candidate support plane data as the support plane data of the express delivery item.

[0103] If the above foreground data is point data, it indicates that the express delivery item has no top surface and is an irregular object. The terminal can perform region growing on the point data and determine whether the projection of plane a can enclose the projection of the point data after region growing. If it cannot enclose, continue to select a plane in set A2 that is farther from the top surface as the next candidate support plane data, and repeat the above steps until the selected candidate support plane data can enclose the point data after the above region growing, and determine the selected candidate support plane data as the support plane data of the express delivery item.

[0104] Among them, if the terminal detects that none of the candidate support plane data in set A2 can enclose the top surface data after region growing, it also indicates that the express delivery item is an irregular object. The terminal uses the above method for determining the support plane data of an irregular object with foreground data being point data to determine the support plane data of the express delivery item.

[0105] Through the above embodiments, the terminal can determine candidate support plane data in different ways based on the point cloud shape of the foreground data, improving the accuracy of determining candidate support plane data. And by detecting the inclusion relationship between the foreground data after region growing and the candidate support plane data, the support plane data is determined, improving the accuracy of determining the support plane data, and further improving the accuracy of object volume detection.

[0106] In one embodiment, obtaining the volume algorithm corresponding to the shape type includes: if the shape type is regular, obtaining the first projection of the foreground data after growth and the third projection of the foreground data; determining the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection.

[0107] In this embodiment, the above-mentioned regular objects to be detected may further include various regular objects, such as regular objects with parallel bottom and top surfaces and similar sizes and shapes, and quasi-regular objects with parallel bottom and top surfaces and significantly different sizes. Different-shaped objects can correspond to different volume algorithms. For objects of the regular shape type, the terminal can obtain the first projection of the foreground data after growth and the third projection of the foreground data, where the foreground data represents the foreground data before region growth. For regular objects, the preset end condition for the above-mentioned region growth can be that the curvature of the foreground data reaches a preset curvature, so that the terminal takes the top surface and the approximate plane adjacent to the top surface as the foreground data after region growth. The terminal determines the volume algorithm corresponding to the object to be detected according to the area ratio of the above-mentioned first projection and the third projection. Among them, the difference between the area of the first projection and the area of the third projection can be used to determine whether the object to be detected is a regular object or a quasi-regular object. Thus, the terminal can determine different volume algorithms for regular objects and quasi-regular objects respectively.

[0108] For example, in one embodiment, determining the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection and the third projection includes: if the area ratio of the first projection and the second projection is less than a preset threshold, determining that the object to be detected is a regular object; obtaining the first preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0109] In this embodiment, if the terminal detects that the area ratio of the first projection and the third projection is less than the preset threshold, it means that the foreground data after region growth of the object to be detected is approximately the same as the foreground data before region growth. Then the terminal can determine that the object to be detected is a regular object, such as a cuboid, a cube, a cylinder, etc. Then the terminal can obtain the first preset volume algorithm corresponding to the above-mentioned regular object as the volume algorithm of the object to be detected. Among them, the first preset volume algorithm can be determined based on the specific type of regular object to which the object to be detected belongs, including but not limited to the volume algorithm of a cuboid, the volume algorithm of a cube, the volume algorithm of a cylinder, etc.

[0110] In one embodiment, determining the regular type corresponding to the object to be detected according to the area ratio of the first projection and the third projection and obtaining the volume algorithm corresponding to the regular type includes: if the area ratio of the first projection and the third projection is greater than or equal to the preset threshold, determining that the regular type of the object to be detected is a quasi-regular object; obtaining the second preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0111] In this embodiment, if the terminal detects that the area ratio of the first projection to the third projection is less than a preset threshold, it indicates that there is a large difference in the size between the foreground data after the region growth of the object to be detected and the foreground data before the region growth. Then the terminal can determine that the object to be detected is a regular-like object, such as a frustum or a barrel-shaped object. Then the terminal can obtain the second preset volume algorithm corresponding to the above regular-like object as the volume algorithm of the object to be detected. The second preset volume algorithm can be determined based on the specific type of the regular-like object to which the object to be detected belongs, including but not limited to the volume algorithm of a frustum, the volume algorithm of a barrel-shaped object, etc.

[0112] Specifically, taking the object to be detected as an express delivery item, for the judgment of regular and regular-like objects, the terminal can obtain the ratio of the projection of the top surface point cloud (foreground data) after region growth to the projection of the top surface point cloud before region growth. If the ratio is greater than the preset threshold, the express delivery item is defined as a regular-like object, and the terminal can use the top surface after region growth as the foreground data of the above regular-like object. If the ratio is less than or equal to the preset threshold, the express delivery item can be defined as a regular object.

[0113] In addition, the terminal can also calculate the volume of irregular objects. In one embodiment, determining the volume of the object to be detected according to the volume algorithm, foreground data, and support plane data includes: if the shape type is irregular, obtaining the length and width of the support plane data of the object to be detected; determining the height of the object to be detected according to the foreground data after growth; determining the volume of the object to be detected according to the product of the length, width, and height.

[0114] In this embodiment, for an irregular object, the terminal can obtain the length and width of the support plane data of the object to be detected, and determine the height of the object to be detected according to the foreground data after growth. For example, the distance between the support plane data and the topmost point of the above foreground data after growth is used as the height of the object to be detected. Thus, the terminal can determine the volume of the object to be detected according to the product of the length, width, and height. That is, the volume algorithm for an irregular object to be detected can be realized by calculating the product of the length, width, and height of the object to be detected.

[0115] Specifically, taking the object to be detected as an express delivery item, the terminal can select the support plane data and the foreground data after growth of the above irregular express delivery item, segment the above point cloud through the mincut minimum cut algorithm to obtain the final point cloud data of the express delivery item. Thus, the terminal determines the length, width, height, and volume based on the support plane data and the foreground data after growth in the point cloud data and outputs them.

[0116] Through the above embodiments, the terminal can determine the volume algorithm of the to-be-detected object of the regular class by judging the projection ratio between the foreground data after region growing and the foreground data before growing, and determine the volume of the irregular object by combining the support plane data and the foreground data after growing, thereby improving the accuracy of volume detection for different types of objects.

[0117] In one embodiment, as Figure 3 shown, Figure 3 FIG. is a schematic flowchart of an object volume detection method in another embodiment. In this embodiment, the image acquisition device may be a structured light camera or a TOF camera, and the to-be-detected object may be an express package. After the depth map captured by the structured light camera or the TOF camera is input into the system, the terminal first converts the depth map into initial point cloud data through an algorithm, and then uses the PEAC plane extraction algorithm to process the point cloud to extract all the plane data. If the plane cannot be extracted from the point cloud data, that is, the number of all extracted planes is 0, the terminal outputs "Please place the express package on a flat plane, clear the surrounding clutter, and measure again!", and the algorithm ends; if the plane can be extracted, that is, the number of all extracted planes is greater than or equal to 1, the terminal enters the regular algorithm module, the quasi-regular algorithm module, or the irregular algorithm module, calculates the volume of the express package and outputs it, and the algorithm ends.

[0118] Among them, as Figure 4 shown, Figure 4 FIG. is a schematic flowchart of an object shape judgment step in one embodiment. The terminal extracts all the plane data in the point cloud data through the PEAC algorithm, that is, divides the planes in the point cloud data, and obtains the 3D center point coordinates, normal vectors, and the distance information from the plane to the camera of each plane data. A mask for each plane is generated according to the maximum contour of each plane data, and 255 is filled in the mask of each plane data, and the values of the remaining point cloud data are 0, which is used to mark the plane data in the point cloud data.

[0119] The terminal can further select the top surface (foreground data) of the express package. For example, the terminal sets the plane data that intersects with the preset center window (4 pixels * 4 pixels) as the candidate top surface (candidate plane data), and takes the candidate top surface with the smallest absolute value of the distance from the plane to the camera as the top surface (foreground data) of the express package. If the number of candidate top surfaces is 0, the terminal can determine that the top surface selection fails, and determine that the express package is an irregular object, and call the irregular algorithm module to process the express package.

[0120] The terminal can also screen out the plane data parallel to the top surface from the above-mentioned plane data, and these plane data do not include the top surface. For example, the terminal can define the plane data with an angle less than 10 degrees between the normal vector of the top surface and the normal vector of the plane as the plane data parallel to the top surface, and store these parallel plane data in set A. If the number of planes in set A is 0, the terminal determines that the express item is an irregular object and calls the irregular algorithm module for processing. Moreover, the terminal can also merge the planes in set A. Taking plane a in set A as an example, if there is a plane b in set A such that the included angle between the normal vector of plane b and the normal vector of plane a is less than 5 degrees, it indicates that the two planes are approximately parallel, and the line connecting the center points of plane a and b is approximately orthogonal to the normal vector of plane a, for example, the included angle is greater than 87 degrees, indicating that the two planes are approximately coplanar, then the terminal can merge planes a and b. The terminal can define the merged set A as set A1, and can screen out the planes that can surround the top surface of the express item from set A1 to form set A2, and sort the planes in A2 in ascending order of the distance from the top surface. Among them, if the number of planes in set A2 is 0, the terminal can determine that the express item is an irregular object and call the irregular algorithm module to process the express item.

[0121] The terminal can determine the support plane data from the plane data. The terminal can try the planes in set A2 as the support surface of the express item in ascending order of the distance from the top surface. For example, assuming plane a in set A2, the terminal obtains the 3D point cloud data on plane a to form plane data, and determines whether plane a can surround the top surface after region growing (the foreground data after growing). If it cannot surround, continue to select the plane data farther from the top surface in set A2 as the candidate support surface (candidate support plane data), and repeat until the selected candidate support surface can surround the top surface after region growing, and determine the selected candidate support surface as the support surface of the express item (support plane data). Among them, if none of the planes in set A2 can surround the top surface after region growing, the terminal can determine that the express item is an irregular object and call the irregular algorithm module to process the express item.

[0122] Objects of the above rule types can also have multiple rule states, such as regular rules and class rules, etc. The terminal can obtain the ratio of the projection of the point cloud on the top surface after region growing to the projection of the point cloud on the top surface before region growing. If the above ratio is greater than the preset threshold, the terminal defines the express item as a class-regular object, marks it as 1, and redefines the top surface of the express item as the top surface after region growing; if the above ratio is less than or equal to the preset threshold, the terminal can define the measured express item as a regular object and mark it as 0. Among them, the above preset threshold depends on the point cloud situation and varies due to different hardware devices, and needs to be defined through self-experimentation during use. Subsequently, the terminal can determine the volume algorithm of regular or class-regular express items based on the marked values using regular or class-regular algorithm modules, calculate the corresponding volume based on the volume algorithm and output it. For non-regular express items, the volume algorithm for non-regular objects is used to determine the volume and output it.

[0123] Among them, for non-regular express items, the process for the terminal to determine the volume can be as Figure 5 shown Figure 5 which is a schematic flowchart of the steps for determining the volume of an object in an embodiment. In this embodiment, the terminal can initialize the seed in advance, and this seed can be used as the foreground data of the non-regular express item. The terminal extracts the plane data of the express item through the PEAC plane extraction algorithm. If the plane data cannot be extracted, it outputs the information of "Please place the express item on a flat plane, remove the surrounding clutter, and measure again". The terminal can select the support plane from the plane data according to the data and the foreground region after growth, and extract the specific spatial structure of the non-regular express item through the mincut minimum cut algorithm. After obtaining the length, width, and height of the non-regular express item, the volume of the non-regular express item is determined.

[0124] Among them, when the terminal determines the support plane data of the irregular express item and the foreground data after growth, it can first define the viewfinder frame (the boundary of the view area) of the image acquisition device, where the frame line has a preset width. The terminal puts the planes with intersections on the viewfinder frame into the queue q_out, and puts the planes within the viewfinder frame but without intersections with the boundary of the frame into the queue q_in. The terminal can first merge the planes in q_out to form q_merge. Among them, the plane data of q_merge is sorted from near to far according to the distance from the above seeds. The terminal takes the nearest merged plane that has not been tried as the support surface from q_merge as the current support surface, separates the foreground and the background, grows the seeds into the foreground area (the foreground data after growth), and determines whether the current support surface can surround the foreground area. If it can surround, the current support surface is used as the final support surface; otherwise, a merged plane that has not been tried as the support surface and is farther away is selected from q_merge as the current support surface for the surrounding judgment. Until all the merged planes in q_merge have been selected, if there is still no merged plane that can surround the above foreground data after growth, the terminal can merge the planes in q_in to form q_merge.

[0125] The terminal takes the nearest merged plane that has not been tried as the support surface from the q_merge corresponding to q_in as the current support surface, separates the foreground and the background, grows the seeds into the foreground area (the foreground data after growth), and determines whether the current support surface can surround the foreground area. If it can surround, the current support surface is used as the final support surface; otherwise, a merged plane that has not been tried as the support surface and is farther away is selected from the q_merge corresponding to q_in as the current support surface for the surrounding judgment. Until all the merged planes in the q_merge corresponding to q_in have been selected, if there is still no merged plane that can surround the above foreground data after growth, the terminal can output a prompt message: Please place the express item on a flat plane, remove the surrounding clutter, and re-measure.

[0126] Through the above embodiments, the terminal combines the point cloud data of the object to be detected to perform shape detection on the object to be detected, and thus uses an algorithm that conforms to the shape of the object to be detected to calculate the volume of the object to be detected, improving the accuracy of volume calculation for objects such as logistics express items. By first selecting the support surface in q_out and then in q_in, the efficiency of the algorithm for obtaining the support surface can be improved. Moreover, the terminal can measure the volume of regular, quasi-regular, and irregular types of express items, with strong robustness.

[0127] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0128] Based on the same inventive concept, an embodiment of the present application further provides an object volume detection device for implementing the object volume detection method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the object volume detection device provided below can refer to the limitations on the object volume detection method in the above text, and will not be repeated here.

[0129] In one embodiment, as Figure 6 shown, an object volume detection device is provided, including: an acquisition module 500, a growth module 502, a determination module 504, and a detection module 506, where:

[0130] The acquisition module 500 is configured to acquire the point cloud data collected by the image acquisition device for the object to be detected, and extract each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device.

[0131] The growth module 502 is configured to perform region growth on the foreground data according to the point cloud data, and determine the support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data.

[0132] The determination module 504 is configured to determine the shape type of the object to be detected according to the point cloud shape of the foreground data.

[0133] The detection module 506 is configured to obtain the volume algorithm corresponding to the shape type, and determine the volume of the object to be detected according to the volume algorithm and the point cloud data.

[0134] In one embodiment, the obtaining module 500 is configured to extract each piece of original plane data from the point cloud data and obtain the normal vector of each piece of original plane data; for each piece of original plane data, obtain the connection line between the center points of the original plane data and each other piece of original plane data; obtain, from each other piece of original plane data, the other piece of original plane data whose included angle between the connection line of the center points and the normal vector of the original plane data is within a first preset angle range and the included angle between the normal vectors of the two pieces of original plane data is less than or equal to a second preset angle, as the coplanar plane data; and merge the original plane data with the corresponding coplanar plane data to obtain the plane data in the point cloud data.

[0135] In one embodiment, the obtaining module 500 is configured to obtain the plane data that intersects with a preset central window in each piece of plane data as the candidate plane data; the preset central window represents a window with a preset size set at the center of the acquisition area of the image acquisition device; and obtain the candidate plane data that is closest to the image acquisition device in each piece of candidate plane data as the foreground data.

[0136] In one embodiment, the obtaining module 500 is configured to obtain the top layer point data in the point cloud data; if the top layer point data is not a plane, determine the point data that is closest to the image acquisition device from the top layer point data as the foreground data.

[0137] In one embodiment, the growing module 502 is configured to obtain each piece of point data adjacent to the foreground data in the point cloud data; use each piece of point data and the foreground data as the new foreground data, and return to the step of obtaining each piece of point data adjacent to the foreground data in the point cloud data until a preset end condition is met, and then obtain the grown foreground data.

[0138] In one embodiment, the growing module 502 is configured to determine each piece of candidate support plane data from each piece of plane data according to the foreground data; obtain the first projection of the grown foreground data and the second projection of each piece of candidate support plane data; for each piece of candidate support plane data, if the first projection is within the range of the second projection of the candidate support plane data, determine the candidate support plane data as the support plane data.

[0139] In one embodiment, the growing module 502 is configured to, if the foreground data is a plane, obtain the plane data parallel to the foreground data in each piece of plane data as the candidate support plane data; if the foreground data is not a plane, obtain the plane data that intersects with the boundary of the viewing area in each piece of plane data as the candidate support plane data; the boundary of the viewing area represents the boundary of the viewing area of the image acquisition device.

[0140] In one embodiment, the above-mentioned determination module 504 is configured to determine that the shape type of the object to be detected is a regular type if the foreground data is planar; and determine that the shape type of the object to be detected is an irregular type if the foreground data is non-planar.

[0141] In one embodiment, the above-mentioned detection module 506 is configured to, if the shape type is a regular type, obtain the first projection of the foreground data after growth and the third projection of the foreground data; and determine the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection.

[0142] In one embodiment, the above-mentioned detection module 506 is configured to determine that the object to be detected is a regular object if the area ratio of the first projection to the third projection is less than a preset threshold; and obtain the first preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0143] In one embodiment, the above-mentioned detection module 506 is configured to determine that the regular type of the object to be detected is a quasi-regular object if the area ratio of the first projection to the third projection is greater than or equal to the preset threshold; and obtain the second preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

[0144] In one embodiment, the above-mentioned detection module 506 is configured to, if the shape type is an irregular type, obtain the length and width of the support plane data of the object to be detected; determine the height of the object to be detected according to the foreground data after growth; and determine the volume of the object to be detected according to the product of the length, width, and height.

[0145] Each module in the above object volume detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above respective modules.

[0146] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for detecting the volume of an object. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0147] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0148] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the above-mentioned method for detecting the volume of an object.

[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the above-mentioned method for detecting the volume of an object.

[0150] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the above-mentioned method for detecting the volume of an object.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0153] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0154] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for detecting the volume of an object, characterized in that, The method includes: Obtaining point cloud data collected by an image acquisition device for an object to be detected, and extracting each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device; Performing region growing on the foreground data according to the point cloud data, and determining support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data; Determining the shape type of the object to be detected according to the point cloud shape of the foreground data; Obtaining a volume algorithm corresponding to the shape type, and determining the volume of the object to be detected according to the volume algorithm, the foreground data, and the support plane data.

2. The method according to claim 1, wherein The extracting each plane data in the point cloud data includes: Extracting each original plane data in the point cloud data, and obtaining the normal vector of each original plane data; For each original plane data, obtaining the connecting line of the center points between the original plane data and each other original plane data; Obtaining, from each other original plane data, other original plane data whose included angle between the connecting line of the center points and the normal vector of the original plane data is within a first preset angle range and the included angle between the normal vector and the normal vector of the original plane data is less than or equal to a second preset angle, as coplanar plane data; Merging the original plane data and the corresponding coplanar plane data to obtain the plane data in the point cloud data.

3. The method according to claim 1, wherein The steps of extracting the foreground data in the point cloud data include: Obtaining plane data that intersects with a preset central window in each plane data as candidate plane data; the preset central window represents a window with a preset size set at the center of the acquisition area of the image acquisition device; Obtaining the candidate plane data closest to the image acquisition device among the candidate plane data as the foreground data.

4. The method according to claim 1, wherein The steps of extracting the foreground data in the point cloud data further include: Obtaining the top-level point data in the point cloud data, and if the top-level point data is not a plane, determining the point data closest to the image acquisition device from the top-level point data as the foreground data.

5. The method according to claim 1, wherein The performing region growing on the foreground data according to the point cloud data includes: Obtaining each point data adjacent to the foreground data in the point cloud data; Taking the each point data and the foreground data as new foreground data, and returning to the step of obtaining each point data adjacent to the foreground data in the point cloud data until the grown foreground data is obtained when a preset end condition is satisfied.

6. The method according to claim 1, wherein The determining support plane data in each plane data according to the inclusion relationship between the grown foreground data and each plane data includes: Determining each candidate support plane data from each plane data according to the foreground data; Obtaining the first projection of the grown foreground data and the second projection of each candidate support plane data; For each candidate support plane data, if the first projection is within the range of the second projection of the candidate support plane data, determining the candidate support plane data as the support plane data.

7. The method according to claim 6, wherein Determining each candidate support plane data from the respective plane data according to the foreground data includes: If the foreground data is a plane, obtaining the plane data parallel to the foreground data from the respective plane data as the candidate support plane data; If the foreground data is a non-plane, obtaining the plane data in the respective plane data that intersects with the boundary of the viewing area as the candidate support plane data; the boundary of the viewing area represents the boundary of the viewing area of the image acquisition device.

8. The method according to claim 1, wherein Determining the shape type of the object to be detected according to the point cloud shape of the foreground data includes: If the foreground data is a plane, determining that the shape type of the object to be detected is a regular type; If the foreground data is a non-plane, determining that the shape type of the object to be detected is an irregular type.

9. The method according to claim 8, wherein Obtaining the volume algorithm corresponding to the shape type includes: If the shape type is a regular type, obtaining the first projection of the foreground data after growth and the third projection of the foreground data; Determining the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection.

10. The method according to claim 9, characterized in that Determining the volume algorithm corresponding to the object to be detected according to the area ratio of the first projection to the third projection includes: If the area ratio of the first projection to the third projection is less than a preset threshold, determining that the object to be detected is a regular object; Obtaining the first preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

11. The method according to claim 9, wherein Determining the regular type corresponding to the object to be detected according to the area ratio of the first projection to the third projection, and obtaining the volume algorithm corresponding to the regular type includes: If the area ratio of the first projection to the third projection is greater than or equal to the preset threshold, determining that the regular type of the object to be detected is a quasi-regular object; Obtaining the second preset volume algorithm corresponding to the regular object as the volume algorithm of the object to be detected.

12. The method according to claim 8, wherein Determining the volume of the object to be detected according to the volume algorithm, the foreground data, and the support plane data includes: If the shape type is an irregular type, obtaining the length and width of the support plane data of the object to be detected; Determining the height of the object to be detected according to the foreground data after growth; Determining the volume of the object to be detected according to the product of the length, width, and height.

13. An object volume detection device, characterized in that, The device includes: An acquisition module, configured to acquire point cloud data collected by an image acquisition device for an object to be detected, and extract each plane data and foreground data in the point cloud data; the foreground data represents the point cloud data closest to the image acquisition device; A growth module, configured to perform region growth on the foreground data according to the point cloud data, and determine the support plane data in the respective plane data according to the inclusion relationship between the foreground data after growth and the respective plane data; A determination module, configured to determine the shape type of the object to be detected according to the point cloud shape of the foreground data; A detection module, configured to obtain the volume algorithm corresponding to the shape type, and determine the volume of the object to be detected according to the volume algorithm and the point cloud data.

14. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 12.

Images