Object volume determination method, apparatus, computer device, and readable storage medium

Abstract

The application relates to an object volume determination method and device, computer equipment and a readable storage medium, and relates to the technical field of logistics. The method comprises the following steps: acquiring an image of an object; the image comprises a reference object region, the reference object region being obtained by frame selection on the object in the image; all planes in the image are determined; based on the similarity between each plane and the reference object region, an intersection plane intersecting the reference object region is determined in each plane; based on the positional relationship between the intersection plane and the reference object region, a support plane of the object is determined; according to the intersection plane and the reference object region, an object point cloud of the object is determined; and according to the plane point cloud of the support plane and the object point cloud, the volume of the object is determined. The method can improve the determination efficiency of the volume of the object.

Description

Technical Field

[0001] This application relates to the field of logistics technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for determining the volume of an object. Background Technology

[0002] In the field of logistics technology, the volume of an object is not only the core basis for calculating transportation costs in the logistics process, but also a key factor that plays a decisive role in the planning of transportation tasks and the formulation of warehousing space storage strategies. Therefore, it is often necessary to measure the volume of an object.

[0003] In related technologies, the volume is usually calculated by manually measuring the key dimensions of an object, such as its length, width, and height, which results in low efficiency in determining the volume of an object. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the efficiency of determining the volume of an object, in order to address the aforementioned technical problem of low efficiency in determining the volume of an object.

[0005] In a first aspect, this application provides a method for determining the volume of an object, including:

[0006] Acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image;

[0007] All planes in the image are identified. Based on the similarity between each plane and the reference object region, intersecting planes that intersect with the reference object region are determined in each plane. Based on the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object is determined.

[0008] The object point cloud of the object is determined based on the intersecting plane and the reference object region;

[0009] The volume of the object is determined based on the planar point cloud of the supporting plane and the object point cloud.

[0010] Secondly, this application also provides an object volume determining device, comprising:

[0011] An image acquisition module is used to acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image;

[0012] A plane determination module is used to determine all planes in the image, determine intersecting planes that intersect with the reference object region in each plane based on the similarity between each plane and the reference object region, and determine the supporting plane of the object based on the positional relationship between the intersecting plane and the reference object region.

[0013] The point cloud determination module is used to determine the object point cloud of the object based on the intersecting plane and the reference object region;

[0014] The volume determination module is used to determine the volume of the object based on the planar point cloud of the supporting plane and the object point cloud.

[0015] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0016] Acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image;

[0017] All planes in the image are identified. Based on the similarity between each plane and the reference object region, intersecting planes that intersect with the reference object region are determined in each plane. Based on the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object is determined.

[0018] The object point cloud of the object is determined based on the intersecting plane and the reference object region;

[0019] The volume of the object is determined based on the planar point cloud of the supporting plane and the object point cloud.

[0020] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0021] Acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image;

[0022] All planes in the image are identified. Based on the similarity between each plane and the reference object region, intersecting planes that intersect with the reference object region are determined in each plane. Based on the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object is determined.

[0023] The object point cloud of the object is determined based on the intersecting plane and the reference object region;

[0024] The volume of the object is determined based on the planar point cloud of the supporting plane and the object point cloud.

[0025] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0026] Acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image;

[0027] All planes in the image are identified. Based on the similarity between each plane and the reference object region, intersecting planes that intersect with the reference object region are determined in each plane. Based on the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object is determined.

[0028] The object point cloud of the object is determined based on the intersecting plane and the reference object region;

[0029] The volume of the object is determined based on the planar point cloud of the supporting plane and the object point cloud.

[0030] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for determining the volume of an object can initially determine the corresponding region of the object in an image by selecting the object whose volume is to be measured, thus obtaining a reference object region. By utilizing the similarity between each plane in the image and the reference object region, intersecting planes that intersect with the reference object region can be determined in each plane. Based on the positional relationship between each intersecting plane and the reference object region, the supporting plane of the object can be determined in each intersecting plane. Based on the intersecting planes and the reference object region, the object's point cloud can be accurately determined. Based on the planar point cloud of the supporting plane and the object's point cloud, the volume of the object can be calculated. This method for determining the volume of an object can achieve the measurement of the object's volume based solely on an image containing the object, eliminating the need for manual measurement of key dimensions such as the object's length, width, and height, thereby improving the efficiency of determining the object's volume. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a diagram illustrating the application environment of an object volume determination method in one embodiment.

[0033] Figure 2 This is a flowchart illustrating a method for determining the volume of an object in one embodiment;

[0034] Figure 3 This is a flowchart illustrating the steps of determining the supporting plane of an object based on the positional relationship between the intersecting plane and the reference object region in one embodiment.

[0035] Figure 4 This is a flowchart illustrating the steps of determining the object point cloud of an object based on intersecting planes and a reference object region in one embodiment.

[0036] Figure 5 This is a flowchart illustrating the steps for determining the volume of an object based on a rotated planar point cloud and a rotated object point cloud in one embodiment.

[0037] Figure 6 This is a flowchart illustrating an interactive object volume measurement method in one embodiment;

[0038] Figure 7 This is a structural block diagram of an object volume determination device in one embodiment;

[0039] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0043] The object volume determination method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the environment includes server 102 and terminal 104, with server 102 communicating with terminal 104 via a network. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0044] Specifically, first, server 102 acquires an image of the object through terminal 104; the image includes a reference object region, which is obtained by selecting the object in the image; then, server 102 determines all planes in the image, and based on the similarity between each plane and the reference object region, determines intersecting planes that intersect with the reference object region in each plane, and determines the supporting plane of the object based on the positional relationship between the intersecting plane and the reference object region; next, server 102 determines the object point cloud based on the intersecting plane and the reference object region; finally, server 102 determines the volume of the object based on the planar point cloud of the supporting plane and the object point cloud.

[0045] In one exemplary embodiment, such as Figure 2 As shown, a method for determining the volume of an object is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps S202 to S208. Wherein:

[0046] Step S202: Obtain an image of the object.

[0047] The images were captured using image acquisition equipment.

[0048] The image includes a reference object region, which is obtained by the user selecting objects in the image.

[0049] Specifically, the server receives an image of an object uploaded by the terminal, which contains a reference object area selected by the user. The terminal can be an image acquisition device that captures the image, or it can be any other device besides an image acquisition device that captures the image.

[0050] In practical applications, the server prompts the user via the terminal to select objects in the image to obtain the reference object region in the image.

[0051] In the field of logistics technology, the volume of an object is not only the core basis for calculating transportation costs in the logistics process, but also a key factor that plays a decisive role in the planning of transportation tasks and the formulation of warehousing space storage strategies. In specific applications, the object is the transported goods; the user can be at least one of the sender and logistics staff. For example, when a sender schedules a shipment, they can determine the volume of the transported goods using the object volume determination method provided in this application, and thus estimate the transportation costs. Similarly, when logistics staff need to calculate the transportation costs of transported goods, plan transportation tasks, and plan warehousing space storage strategies, they can determine the volume of the transported goods using the object volume determination method provided in this application, thereby accurately calculating the transportation costs, rationally planning transportation tasks, and formulating warehousing space storage strategies.

[0052] Step S204: Determine all planes in the image; based on the similarity between each plane and the reference object region, determine the intersecting planes that intersect with the reference object region in each plane; and based on the positional relationship between the intersecting planes and the reference object region, determine the supporting planes of the object.

[0053] The similarity between the plane and the reference object region includes at least the similarity between the plane and the reference object region in terms of area and position.

[0054] The positional relationship between the intersecting plane and the reference object region includes at least the distance between the reference object region and the intersecting plane, and the distribution of each point in the reference object region relative to the intersecting plane, such as whether it is located on the same side of the intersecting plane.

[0055] The supporting plane of an object refers to the plane used to support the object, such as the plane on which the object is placed. If the image is taken by placing the object on the ground, then the supporting plane is the ground. If the image is taken by placing the object on a table, then the supporting plane is the plane on which the table is located.

[0056] Specifically, first, the server obtains the depth map corresponding to the image and calculates all possible planes in the image based on the depth map; then, the server determines the planes that intersect with the reference object region in each plane according to the similarity between the reference object region and each plane, thus obtaining each intersecting plane; next, the server determines the plane supporting the object in each intersecting plane according to the positional relationship between the reference object region and each intersecting plane, thus obtaining the supporting plane.

[0057] The image comprises multiple first points, and the depth map comprises multiple second points. The resolution of the depth map is the same as that of the image; therefore, there is a correspondence between first and second points with the same coordinates. The pixel value corresponding to each second point represents the distance between the corresponding first point's location in the real world and the image acquisition device that captured the image. In practical applications, the image acquisition device that captures the image has the ability to generate a depth map, and the server obtains the depth image corresponding to the image from the image acquisition device.

[0058] Step S206: Determine the object point cloud based on the intersecting plane and the reference object region.

[0059] Specifically, the server filters out the target object region from the reference object region based on each intersecting plane, and obtains the object point cloud based on each point within the target object region.

[0060] Step S208: Determine the volume of the object based on the planar point cloud of the supporting plane and the object point cloud.

[0061] Specifically, the server obtains a planar point cloud of the support plane based on each point within the support plane. The server rotates the planar point cloud and the object point cloud to obtain a rotated planar point cloud and a rotated object point cloud. Based on the rotated planar point cloud and the rotated object point cloud, the server determines the base and height of the object, and then calculates the volume of the object based on the base and height.

[0062] In the above method for determining the volume of an object, by selecting the object whose volume to be measured in an image, the corresponding region of the object can be initially determined in the image, thus obtaining a reference object region. Based on the similarity between each plane in the image and the reference object region, intersecting planes that intersect with the reference object region can be determined in each plane. According to the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object can be determined. Based on the intersecting planes and the reference object region, the object's point cloud can be accurately determined. Based on the planar point cloud of the supporting plane and the object's point cloud, the volume of the object can be calculated. This method for determining the volume of an object can achieve the calculation of the object's volume based on an image containing the object, eliminating the need for manual measurement of key dimensions such as the object's length, width, and height, thus improving the efficiency of determining the object's volume. Furthermore, existing terminals can directly output 3D point cloud information. Obtaining the volume of box-shaped objects based on the 3D point cloud information output by the terminal is relatively easy, but obtaining the volume of non-box-shaped objects remains more difficult. At the same time, there are currently few methods that can calculate the volume of both box-shaped and non-box-shaped objects. However, the aforementioned method for determining the volume of objects does not have special requirements on the shape of the object to be measured. It can calculate the volume of both box-shaped and non-box-shaped objects, greatly reducing the difficulty for logistics companies in calculating the volume of express delivery items.

[0063] In an exemplary embodiment, the process by which the server calculates all possible planes in an image based on a depth map is as follows: First, the server determines the three-dimensional coordinates of each first point in a preset three-dimensional coordinate system based on the depth map and the intrinsic parameters of the image acquisition device, wherein the preset three-dimensional coordinate system has the image acquisition device as its origin; then, the server calculates all possible planes in the image using a point cloud plane detection algorithm based on the three-dimensional coordinates of each first point and the depth map.

[0064] In this embodiment, the server calculates the three-dimensional coordinates of each first point based on the following formula:

[0065] (Formula 1)

[0066] Where X is the x-coordinate of the first point, Y is the y-coordinate of the first point, and Z is the z-coordinate of the first point; x0 and y0 are the coordinates of the second point corresponding to the first point in the depth map; W depth H depth These are the width and height of the depth map, respectively; W image H image These are the width and height of the image, respectively; depth is the pixel value of the second point corresponding to the first point; c x c y , where are the coordinates of the optical axis center of the image acquisition device; f is the focal length of the image acquisition device.

[0067] In an exemplary embodiment, the step S202 above, which involves acquiring an image of an object, specifically includes the following steps: when acquiring an image through a terminal, the object in the image is selected by means of a prompt on the terminal to obtain a reference object region in the image.

[0068] Specifically, when a user captures an image through a terminal, the server can prompt the user to select an object within the captured image. The server can also prompt the user to select an object within the uploaded image when the user uploads an image to the server through the terminal.

[0069] In practical applications, the server needs to prompt the user to select an area that is as close to the object as possible.

[0070] In this embodiment, the server interacts with the user through a terminal and determines the reference object region of the object in the image based on the interaction.

[0071] In an exemplary embodiment, step S204 above, which determines the intersecting planes that intersect with the reference object region in each plane based on the similarity between each plane and the reference object region, specifically includes the following steps: obtaining the intersection-union ratio between each plane and the reference object region; determining the planes that intersect with the reference object region from each plane according to the intersection-union ratio and intersection-union ratio threshold corresponding to each plane, thereby obtaining each intersecting plane.

[0072] The intersection-union ratio of the plane and the reference object region is the ratio of the intersection area to the union area of ​​the plane and the reference object region. Therefore, the intersection-union ratio of the plane and the reference object region characterizes the degree of overlap between the plane and the reference object region, and can thus characterize the similarity between the plane and the reference object region. The higher the intersection-union ratio, the more similar the plane and the reference object region are.

[0073] Specifically, for each plane, the server calculates the intersection-union ratio (IUR) between the plane and the reference object region, then compares the IUR with a preset IUR threshold, and determines whether the plane intersects with the reference object region based on the comparison result. If the plane intersects with the reference object region, the server identifies it as an intersecting plane.

[0074] In practical applications, the server determines the plane type based on the comparison between the intersection-union ratio (IU) and a preset IU threshold. Specifically, it can be categorized into inner planes located within the reference object region, outer planes located outside the reference region, and intersecting planes that intersect the reference object region. For example, when the IU is greater than or equal to the IU threshold, the server determines the plane as an intersecting plane; when the IU is less than the IU threshold but greater than the preset IU, the server determines the plane as an outer plane; and when the IU is equal to the preset IU, the server determines the plane as an inner plane. In practical applications, the IU threshold is 0.05, and the preset IU is 0.

[0075] Furthermore, based on the planar classification performance of different candidate cross-union ratio (CUP) thresholds, the server determines the candidate CUP with the best planar classification performance as the preset CUP threshold.

[0076] In this embodiment, the server can determine the positional relationship between the plane and the reference object region based on the intersection-union ratio of the plane and the reference object region, thereby classifying the planes and quickly identifying the planes that intersect with the reference object region from each plane.

[0077] In one exemplary embodiment, the number of intersecting planes is at least one.

[0078] like Figure 3 As shown, step S204 above, which determines the supporting plane of the object based on the positional relationship between the intersecting plane and the reference object region, specifically includes the following steps:

[0079] Step S302: Determine the target points of the reference object region under each intersecting plane.

[0080] Step S304: Determine the target distance between the reference object region and each intersecting plane based on the target points of the reference object region under each intersecting plane.

[0081] Step S306: For each intersecting plane, if all target points in the reference object region under the intersecting plane are located on the same side of the intersecting plane, the intersecting plane is determined as a candidate support plane.

[0082] Step S308: From each candidate support plane, select the candidate support plane with the smallest target distance from the reference object area to obtain the support plane.

[0083] The target point of the reference object region under the intersecting plane is the point other than the intersecting plane among all points within the reference object region.

[0084] Among them, the target distance between the reference object region and the intersecting plane is the minimum distance among the distances between each corresponding target point and the intersecting plane.

[0085] Specifically, for each intersecting plane, the server first identifies the points within the reference object region that lie on the intersecting plane. Then, it identifies the points within the reference object region other than those on the intersecting plane as target points of the reference object region under the intersecting plane. Next, for each target point, the server determines the distance from the target point to the intersecting plane based on the target point's 3D coordinates and the plane equation of the intersecting plane. Then, the server determines the minimum distance among the distances from each target point to the intersecting plane as the target distance from the reference object region to the intersecting plane. Simultaneously, the server determines whether each target point is located on the same side of the intersecting plane based on the 3D coordinates of each target point and the plane equation of the intersecting plane. If all target points are located on the same side of the intersecting plane, the server identifies the intersecting plane as a candidate support plane.

[0086] Finally, the server determines the candidate support plane with the smallest target distance from the reference object region among all candidate support planes as the support plane of the object.

[0087] In this embodiment, the server can determine the supporting plane of the object from each intersecting plane based on the distance between each point in the reference object area and the intersecting plane, as well as the distribution of each point in the reference object area relative to the intersecting plane.

[0088] like Figure 4As shown, in an exemplary embodiment, step S206 above, which determines the object point cloud based on the intersecting plane and the reference object region, specifically includes the following steps:

[0089] Step S402: Determine the intersection region between the intersecting plane and the reference object region.

[0090] Step S404: Remove the intersection region in the reference object region to obtain the target object region of the object.

[0091] Step S406: Cluster the points within the target object region to obtain multiple point clusters.

[0092] Step S408: Among multiple point clusters, determine the point cluster containing the most points to obtain the object point cloud.

[0093] The number of intersecting planes is at least one.

[0094] Specifically, the server takes the intersection of each intersecting plane and the reference object region to obtain the intersection region between each intersecting plane and the reference object region. Then, the server removes the intersection region with each intersecting plane in the reference object region to obtain the target object region of the object. Next, the server performs clustering processing on each point in the target object region, dividing it into multiple point clusters, and takes the point cluster with the most points as the object point cloud of the object.

[0095] In practical applications, the server can also use statistical filtering to filter out outliers from the point clusters containing the most points, and the remaining points after filtering are used as the object point cloud.

[0096] In practical applications, the server first sets the entire image (image, depth map, or virtual map) to black, then sets the area outside the reference object region to white, and then sets the intersections of each intersecting plane with the reference object region to white as well. The areas that remain black in the image represent the target object region. The virtual map is constructed based on the image or depth map, and its resolution is the same as the image and depth map.

[0097] In this embodiment, the server can optimize the selected reference object region based on the intersection of the intersecting plane and the reference object region to obtain a target object region that more accurately describes the object.

[0098] In an exemplary embodiment, both the planar point cloud and the object point cloud are three-dimensional point clouds in a three-dimensional coordinate system preset with the image acquisition device as the origin.

[0099] Step S208 above, which determines the volume of the object based on the planar point cloud and the object point cloud of the supporting plane, specifically includes the following steps: determining the rotation matrix based on the three-dimensional coordinates of the planar point cloud; rotating the planar point cloud and the object point cloud according to the rotation matrix to obtain the rotated planar point cloud and the rotated object point cloud; and determining the volume of the object based on the rotated planar point cloud and the rotated object point cloud.

[0100] The rotation matrix is ​​used to rotate the planar point cloud and the object point cloud around the origin of the three-dimensional coordinate system so that the rotated planar point cloud is parallel to the horizontal plane of the three-dimensional coordinate system.

[0101] The horizontal plane refers to the xoy plane of the three-dimensional coordinate system; it is parallel to the xoy plane, that is, perpendicular to the z-axis of the three-dimensional coordinate system.

[0102] Specifically, the server determines the normal vector of the planar point cloud based on the 3D coordinates of each point in the planar point cloud, and then determines the rotation matrix of the supporting plane according to the Rodriguez rotation formula and the normal vector of the planar point cloud. Next, the server rotates the planar point cloud and the object point cloud with the origin of the 3D coordinate system as the rotation center according to the rotation matrix, so that the rotated planar point cloud (and the rotated object point cloud) are parallel to the horizontal plane of the 3D coordinate system, thus obtaining the rotated planar point cloud and the rotated object point cloud. Finally, the server determines the base and height of the object based on the rotated planar point cloud and the rotated object point cloud, and then calculates the volume of the object.

[0103] Rotating a point cloud refers to the process of rotating the three-dimensional coordinates of each point in the point cloud according to a rotation matrix; the three-dimensional coordinates of each point in the point cloud after rotation are the result of the rotation of the three-dimensional coordinates of each point in the point cloud before rotation.

[0104] In this embodiment, after rotation, the rotated plane point cloud of the object's supporting plane is parallel to the horizontal plane of the three-dimensional coordinate system. Therefore, the z-coordinate of each point in the three-dimensional coordinate system of the rotated plane point cloud and the rotated object point cloud reflects the height of that point in the three-dimensional coordinate system, which makes it easier to calculate the height of the object and thus the volume of the object.

[0105] like Figure 5 As shown, in an exemplary embodiment, the above steps, based on the rotated planar point cloud and the rotated object point cloud, determine the volume of the object, specifically including the following steps:

[0106] Step S502: Determine the projection of the rotated object point cloud onto the horizontal plane, and determine the minimum bounding rectangle of the projection.

[0107] Step S504: Determine the average height of the rotated planar point cloud and the maximum height of the rotated object point cloud.

[0108] Step S506: Calculate the volume of the object using the smallest bounding rectangle as the base and the difference between the highest height and the average height as the height.

[0109] The average height is the average of the height coordinates of each point in the three-dimensional coordinate system of the rotated planar point cloud.

[0110] The highest height is the maximum value of the height coordinates of each point in the three-dimensional coordinates of the rotated object point cloud.

[0111] Here, the height coordinate refers to the z-coordinate.

[0112] Specifically, the server projects the rotated object point cloud onto the xoy plane of the 3D coordinate system to obtain the projection of the rotated object point cloud onto the xoy plane, and determines the minimum bounding rectangle of this projection. Then, the server calculates the average value of the z-coordinates of each point in the 3D coordinates of the rotated planar point cloud as the average height of the rotated planar point cloud, and determines the maximum value of the z-coordinates of each point in the 3D coordinates of the rotated object point cloud as the maximum height of the rotated object point cloud. Finally, the server uses the minimum bounding rectangle as the base of the object, subtracts the average height of the rotated planar point cloud from the maximum height of the rotated object point cloud to obtain the height of the object, and then calculates the volume of the object.

[0113] In practical applications, the server removes the z-coordinate from the 3D coordinates of each point in the rotated object point cloud to obtain a set of 2D coordinates for each point in the rotated object point cloud, and determines the minimum bounding rectangle of the 2D coordinate set as the base of the object. At the same time, the server takes the average z-coordinate of the 3D coordinates of each point in the rotated planar point cloud as z0, takes the maximum z-coordinate of the 3D coordinates of each point in the rotated object point cloud as z1, and subtracts z0 from z1 to obtain the height of the object.

[0114] In this embodiment, after rotation, the rotated point cloud of the object's supporting plane is parallel to the horizontal plane of the three-dimensional coordinate system, which facilitates the server to quickly determine the projection of the rotated object point cloud onto the horizontal plane of the three-dimensional coordinate system. By projecting the rotated object point cloud, the server can quickly determine the object's base. Based on the average height and the maximum height of the rotated object point cloud, the server can quickly determine the object's height, which facilitates the calculation of the object's volume.

[0115] In an exemplary embodiment, after calculating the volume of the object in step S506 above, the method further includes the following steps: determining the two-dimensional coordinates of each vertex of the minimum bounding rectangle in the horizontal plane; determining the three-dimensional coordinates of each target vertex used to describe the object based on the two-dimensional coordinates, average height, and maximum height of each vertex; and sending the volume and the three-dimensional coordinates of each target vertex to the terminal.

[0116] The terminal is used to display the volume, and to draw the bounding box of the object in the image based on the three-dimensional coordinates of each target vertex, and to display the image with the drawn bounding box.

[0117] The two-dimensional coordinates of the vertex in the horizontal plane refer to the (x, y) coordinates of the vertex in the xoy plane.

[0118] Specifically, the server determines the (x, y) coordinates of the four vertices of the minimum bounding rectangle, and then substitutes the average height of the rotated planar point cloud and the highest height of the rotated object point cloud into the (x, y) coordinates of the four vertices to obtain the three-dimensional coordinates of the eight target vertices in a preset three-dimensional coordinate system. For example, suppose the two-dimensional coordinates of the four vertices of the minimum bounding rectangle are (x, y)... a y a ), (x b y b ), (x c y c ) and (x d y d If the average height is z0 and the maximum height is Z1, then the server determines the three-dimensional coordinates of the eight target vertices as (x, y, z) respectively. a y a Z0), (x b y b Z0), (x c y c Z0), (x d y d Z0), (x a y a Z1), (x b y b Z1), (x c y c (Z1) and (x) d y d Z1).

[0119] Then, the server sends the object's volume and the three-dimensional coordinates of its eight target vertices to the terminal; the terminal displays the object's volume to the user, and draws a bounding box of the object in the image based on the three-dimensional coordinates of the eight target vertices to highlight the object, and displays the image with the bounding box drawn to the user.

[0120] In this embodiment, the server interacts with the user through a terminal and displays objects and their volumes to the user based on the interaction.

[0121] To more clearly illustrate the object volume determination method provided in the embodiments of this application, a specific embodiment is given below for detailed description. However, it should be understood that the embodiments of this application are not limited thereto. Figure 6 As shown, in one exemplary embodiment, this application also provides an interactive object volume measurement method, specifically including the following steps:

[0122] 1. Select objects in the image.

[0123] When a sender schedules a shipment, and when logistics staff calculate transportation costs, plan transportation tasks, and devise storage strategies, the sender or logistics staff can obtain images of objects through a terminal. The terminal needs to prompt the sender or logistics staff to select objects within the image while obtaining it.

[0124] 2. Calculate 3D point clouds.

[0125] Obtain the depth map of the image, and calculate the 3D point cloud of the image based on the depth map and terminal intrinsic parameters.

[0126] 3. Perform planar detection on the image.

[0127] Based on 3D point cloud and point cloud plane detection algorithms, planes in images are detected.

[0128] 4. Determine the plane type based on the intersection-union ratio.

[0129] Obtain the intersection-union ratio (IUGR) between each plane and the selected area; based on the IUGR and IUGR threshold of each plane, classify each plane into planes located within the selected area, planes located outside the selected area, and planes intersecting with the selected area.

[0130] 5. Determine the intersecting planes and supporting planes.

[0131] The planes intersecting the selected area are defined as intersecting planes. For each intersecting plane, all target points within the selected area, excluding those intersecting planes, are identified. The distance between each target point and the intersecting plane is calculated, and the minimum distance is determined as the target distance between the selected area and the intersecting plane. When all target points are located on the same side of the intersecting plane, it is defined as a candidate supporting plane. The candidate supporting plane with the smallest target distance to the selected area is determined as the supporting plane.

[0132] 6. Determine the object point cloud based on the selected area and intersecting planes.

[0133] Remove the portion of the selected area that intersects with the intersecting plane to obtain the target area of ​​the object; perform clustering on the points within the target area to obtain multiple point clusters; among the multiple point clusters, determine the point cluster containing the most points to obtain the object point cloud of the object.

[0134] 7. Denoise the point cloud of the object.

[0135] Statistical filtering is used to filter out outliers in object point clouds.

[0136] 8. Rotate the planar point cloud and the object point cloud of the supporting plane so that the planar point cloud is perpendicular to the z-axis.

[0137] The rotation matrix is ​​determined based on the three-dimensional coordinates of the planar point cloud. According to the rotation matrix, the planar point cloud and the object point cloud are rotated around the origin of the three-dimensional coordinate system so that the rotated planar point cloud is perpendicular to the z-axis of the three-dimensional coordinate system, thus obtaining the rotated planar point cloud and the rotated object point cloud.

[0138] 9. Project the rotated object point cloud to obtain the minimum bounding rectangle of the projection and the two-dimensional coordinates of the four vertices of the minimum bounding rectangle.

[0139] Determine the projection of the rotated object point cloud onto the xoy plane of the 3D coordinate system, determine the minimum bounding rectangle of the projection, and determine the (x, y) coordinates of the four vertices of the minimum bounding rectangle.

[0140] 10. Determine the height of the object based on the rotated object point cloud and the rotated planar point cloud.

[0141] Determine the average height of the height coordinates of each point in the three-dimensional coordinates of the rotated planar point cloud to obtain the average height of the rotated planar point cloud. Determine the maximum height of the height coordinates of each point in the three-dimensional coordinates of the rotated object point cloud to obtain the maximum height of the rotated object point cloud. Subtract the average height of the rotated planar point cloud from the maximum height of the rotated object point cloud to obtain the height of the object.

[0142] 11. Calculate the volume of the object based on the minimum bounding rectangle and its height.

[0143] Using the smallest bounding rectangle as the base of the object, calculate the volume of the object based on its base and height.

[0144] 12. Determine the eight vertices of the object's bounding box based on the two-dimensional coordinates of the four vertices of the minimum bounding rectangle.

[0145] By substituting the maximum height and average height into the two-dimensional coordinates of the four vertices of the minimum bounding rectangle, we obtain the three-dimensional coordinates of the eight vertices of the object's bounding box.

[0146] 13. Output the volume of the object and the three-dimensional coordinates of the eight vertices of the object's bounding box to the terminal for display and drawing.

[0147] Send the volume of an object and the three-dimensional coordinates of the eight vertices of the object's bounding box to the terminal; the terminal is used to display the volume to the sender or logistics staff, and to draw the bounding box of the object in an image based on the eight vertices of the object's bounding box and display it to the sender or logistics staff.

[0148] This embodiment enables high-precision volume measurement of both non-boxed and boxed objects at a low cost. Based on the volume measurement method provided in this embodiment, on the one hand, senders can accurately estimate the transportation costs of their goods, improving their experience; on the other hand, logistics companies can accurately calculate transportation costs, rationally plan transportation tasks and warehouse storage strategies, thereby optimizing business operations.

[0149] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0150] Based on the same inventive concept, this application also provides an object volume determining device for implementing the object volume determining method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more object volume determining device embodiments provided below can be found in the limitations of the object volume determining method described above, and will not be repeated here.

[0151] In one exemplary embodiment, such as Figure 7 As shown, an object volume determination device is provided, including: an image acquisition module 702, a plane determination module 704, a point cloud determination module 706, and a volume determination module 708, wherein:

[0152] Image acquisition module 702 is used to acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image.

[0153] The plane determination module 704 is used to determine all planes in the image, and based on the similarity between each plane and the reference object region, to determine the intersecting planes that intersect with the reference object region in each plane, and based on the positional relationship between the intersecting planes and the reference object region, to determine the supporting planes of the object.

[0154] The point cloud determination module 706 is used to determine the object point cloud of an object based on the intersecting plane and the reference object region.

[0155] The volume determination module 708 is used to determine the volume of an object based on the planar point cloud of the supporting plane and the object point cloud.

[0156] In an exemplary embodiment, the plane determination module 704 is further configured to obtain the intersection-union ratio (IUGR) between each plane and the reference object region; the IUGR is used to characterize the similarity between the plane and the reference object region; based on the IUGR and IUGR threshold corresponding to each plane, the planes intersecting with the reference object region are determined from each plane to obtain each intersecting plane.

[0157] In one exemplary embodiment, the number of intersecting planes is at least one.

[0158] The plane determination module 704 is further configured to determine each target point of the reference object region under each intersecting plane; the target points of the reference object region under the intersecting plane are the points in the reference object region other than the intersecting plane; based on each target point of the reference object region under each intersecting plane, the target distance between the reference object region and each intersecting plane is determined; the target distance between the reference object region and the intersecting plane is the minimum distance among the distances between each corresponding target point and the intersecting plane; for each intersecting plane, if all target points of the reference object region under the intersecting plane are located on the same side of the intersecting plane, the intersecting plane is determined as a candidate support plane; from each candidate support plane, the candidate support plane with the smallest target distance to the reference object region is selected to obtain the support plane.

[0159] In an exemplary embodiment, the point cloud determination module 706 is further configured to determine the intersection region between the intersecting plane and the reference object region; remove the intersection region in the reference object region to obtain the target object region of the object; perform clustering processing on the points in the target object region to obtain multiple point clusters; and determine the point cluster containing the most points among the multiple point clusters to obtain the object point cloud of the object.

[0160] In an exemplary embodiment, both the planar point cloud and the object point cloud are three-dimensional point clouds in a preset three-dimensional coordinate system.

[0161] The volume determination module 708 is also used to determine a rotation matrix based on the three-dimensional coordinates of the planar point cloud; the rotation matrix is ​​used to rotate the planar point cloud and the object point cloud around the origin of the three-dimensional coordinate system so that the rotated planar point cloud is parallel to the horizontal plane of the three-dimensional coordinate system; the planar point cloud and the object point cloud are rotated according to the rotation matrix to obtain the rotated planar point cloud and the rotated object point cloud; the volume of the object is determined based on the rotated planar point cloud and the rotated object point cloud.

[0162] In an exemplary embodiment, the volume determination module 708 is further configured to determine the projection of the rotated object point cloud onto the horizontal plane, determine the minimum bounding rectangle of the projection; determine the average height corresponding to the rotated planar point cloud, and determine the maximum height corresponding to the rotated object point cloud; the average height is the average value of the height coordinates in the three-dimensional coordinates of each point in the rotated planar point cloud, and the maximum height is the maximum value of the height coordinates in the three-dimensional coordinates of each point in the rotated object point cloud; and calculate the volume of the object using the minimum bounding rectangle as the base and the difference between the maximum height and the average height as the height.

[0163] In an exemplary embodiment, the image acquisition module 702 is further configured to select objects in the image by means of terminal prompts when acquiring an image through the terminal, so as to obtain a reference object region in the image.

[0164] In an exemplary embodiment, the object volume determination device further includes a front-end display module for determining the two-dimensional coordinates of each vertex of the minimum bounding rectangle in the horizontal plane; determining the three-dimensional coordinates of each target vertex for describing the object based on the two-dimensional coordinates of each vertex, the average height, and the maximum height; sending the volume and the three-dimensional coordinates of each target vertex to a terminal; and a terminal for displaying the volume, and drawing a bounding box of the object in an image based on the three-dimensional coordinates of each target vertex, and displaying an image with the drawn bounding box.

[0165] Each module in the aforementioned object volume determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0166] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for determining the volume of an object.

[0167] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0169] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0170] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0171] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the volume of an object, characterized in that, The method includes: Acquire an image of an object; the image includes a reference object region, which is obtained by selecting the object in the image; All planes in the image are identified. Based on the similarity between each plane and the reference object region, intersecting planes that intersect with the reference object region are determined in each plane. Based on the positional relationship between the intersecting planes and the reference object region, the supporting plane of the object is determined. The object point cloud of the object is determined based on the intersecting plane and the reference object region; The volume of the object is determined based on the planar point cloud of the supporting plane and the point cloud of the object. The number of intersecting planes is at least one; Determining the supporting plane of the object based on the positional relationship between the intersecting plane and the reference object region includes: Determine each target point of the reference object region under each intersecting plane; the target points of the reference object region under the intersecting plane are the points within the reference object region other than the intersecting plane. Based on each target point of the reference object region under each intersecting plane, the target distance between the reference object region and each intersecting plane is determined; the target distance between the reference object region and the intersecting plane is the minimum distance among the distances between each corresponding target point and the intersecting plane. For each intersecting plane, if all target points in the reference object region under the intersecting plane are located on the same side of the intersecting plane, the intersecting plane is determined as a candidate support plane; From the candidate support planes, the candidate support plane with the smallest target distance from the reference object region is selected to obtain the support plane.

2. The method according to claim 1, characterized in that, The step of determining the intersecting planes that intersect with the reference object region in each of the planes based on the similarity between each of the planes and the reference object region includes: Obtain the intersection-union ratio (CIU) between each plane and the reference object region; the CIU is used to characterize the similarity between the plane and the reference object region. Based on the intersection-union ratio and intersection-union ratio threshold corresponding to each plane, the planes that intersect with the reference object region are determined from each plane, thus obtaining each intersecting plane.

3. The method according to claim 1, characterized in that, Determining the object point cloud based on the intersecting plane and the reference object region includes: Determine the intersection region between the intersecting plane and the reference object region; Remove the intersection region from the reference object region to obtain the target object region of the object; Clustering is performed on the points within the target object region to obtain multiple point clusters; Among the multiple point clusters, the point cluster containing the most points is determined to obtain the object point cloud of the object.

4. The method according to any one of claims 1 to 3, characterized in that, Both the planar point cloud and the object point cloud are three-dimensional point clouds in a preset three-dimensional coordinate system; Determining the volume of the object based on the planar point cloud of the supporting plane and the object point cloud includes: Based on the three-dimensional coordinates of the planar point cloud, a rotation matrix is ​​determined; the rotation matrix is ​​used to rotate the planar point cloud and the object point cloud around the origin of the three-dimensional coordinate system so that the rotated planar point cloud is parallel to the horizontal plane of the three-dimensional coordinate system. The planar point cloud and the object point cloud are rotated according to the rotation matrix to obtain the rotated planar point cloud and the rotated object point cloud. The volume of the object is determined based on the rotated planar point cloud and the rotated object point cloud.

5. The method according to claim 4, characterized in that, Determining the volume of the object based on the rotated planar point cloud and the rotated object point cloud includes: Determine the projection of the rotated object point cloud onto the horizontal plane, and determine the minimum bounding rectangle of the projection; Determine the average height corresponding to the rotated planar point cloud, and determine the maximum height corresponding to the rotated object point cloud; the average height is the average value of the height coordinates in the three-dimensional coordinates of each point in the rotated planar point cloud, and the maximum height is the maximum value of the height coordinates in the three-dimensional coordinates of each point in the rotated object point cloud; The volume of the object is calculated using the smallest bounding rectangle as the base and the difference between the highest height and the average height as the height.

6. The method according to claim 5, characterized in that, The acquisition of the image of the object includes: When acquiring the image through the terminal, the object in the image is selected by prompting the terminal to obtain the reference object region in the image; After calculating the volume of the object, the method also includes: Determine the two-dimensional coordinates of each vertex of the minimum bounding rectangle in the horizontal plane; Based on the two-dimensional coordinates of each vertex, the average height, and the maximum height, the three-dimensional coordinates of each target vertex used to describe the object are determined; The terminal sends the volume and the three-dimensional coordinates of each of the target vertices to the terminal; the terminal is used to display the volume, and to draw a bounding box of the object in the image based on the three-dimensional coordinates of each of the target vertices, and to display the image with the bounding box drawn.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images