Item volume calculation method and apparatus, and electronic device and storage medium

Abstract

Disclosed in the present application are an item volume calculation method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring an image of the interior of a cargo box by means of a camera; inputting the image of the interior into a trained target depth estimation model, so as to obtain depth information of the image of the interior; on the basis of the depth information and parameter information of the camera, obtaining initial point clouds of the interior of the cargo box; and on the basis of a target point cloud among the initial point clouds that satisfies a preset condition, calculating the volume of a target object inside the cargo box, wherein the target object is an object corresponding to the target point cloud. The present application can ensure that the precision and accuracy of object volume measurement are not affected by an item material, and can effectively control the cost of volume measurement.

Description

Method, device, electronic device and storage medium for calculating item volume Technical Field

[0001] The present application relates to the field of logistics technology, and in particular to a method, device, electronic device and storage medium for calculating the volume of an item. Background Art

[0002] As a crucial link in the modern logistics industry, logistics delivery is not only a crucial guarantee for the circulation of goods, but also a key factor in promoting economic development and enhancing corporate competitiveness. Container volume calculation technology has broad applications and research significance in the logistics industry, warehouse management, transportation planning, and other fields. By accurately measuring the volume of cargo containers, it can help logistics companies achieve goals such as precise loading and optimized transportation routes, thereby improving logistics efficiency and reducing transportation costs. Furthermore, container volume calculation technology can be combined with intelligent warehousing systems to achieve automated cargo sorting, storage, and retrieval, thereby improving warehousing efficiency and accuracy, building intelligent warehousing systems, and promoting enterprise digital transformation. Technical issues

[0003] In view of this, how to ensure that the precision and accuracy of object volume measurement are not affected by the material of the object while effectively controlling the cost of volume measurement is an issue that needs to be considered at present. Technical Solutions

[0004] One of the purposes of the embodiments of the present application is to provide a method, device, electronic device and storage medium for calculating the volume of an item.

[0005] In a first aspect, an embodiment of the present application provides a method for calculating the volume of an item, comprising:

[0006] Acquire images of the interior of the cargo box through a camera;

[0007] Inputting the internal image into a trained target depth estimation model to obtain depth information of the internal image;

[0008] Obtaining an initial point cloud inside the cargo box based on the depth information and parameter information of the camera;

[0009] The volume of the target object inside the cargo box is calculated based on the target point cloud that meets the preset conditions in the initial point cloud, and the target object is the object corresponding to the target point cloud.

[0010] In one embodiment, the parameter information includes focal length; and obtaining the initial point cloud inside the cargo box based on the depth information and the parameter information of the camera includes:

[0011] Obtaining pixel coordinates of pixels in the internal image;

[0012] Obtaining the three-dimensional coordinates of the pixels in the internal image inside the cargo box according to the focal length of the camera, the pixel coordinates of the pixels, and the depth information corresponding to the pixels;

[0013] An initial point cloud of the interior of the cargo box is determined based on the three-dimensional coordinates corresponding to the pixels in the internal image.

[0014] In one embodiment, calculating the volume of the target object inside the container based on the target point cloud that meets the preset conditions in the initial point cloud includes:

[0015] Obtain the depth of the cargo box when empty, as well as the distance from the camera installation location to the four planes inside the cargo box;

[0016] Filtering the initial point cloud based on the distance to obtain a target point cloud;

[0017] The volume of the target object inside the cargo box is calculated based on the target point cloud and the depth of the cargo box.

[0018] In one embodiment, filtering the initial point cloud based on the distance to obtain the target point cloud includes:

[0019] Projecting the initial point cloud onto a two-dimensional plane to obtain a two-dimensional point coordinate set;

[0020] Performing discrete conversion processing on the two-dimensional point coordinates in the two-dimensional point coordinate set according to a preset size to obtain discretized grid units;

[0021] Traversing the discrete grid cells to determine whether the grid coordinates corresponding to the grid cells are within the distance;

[0022] Point clouds corresponding to grid cells exceeding the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds.

[0023] In one embodiment, calculating the volume of the target object inside the cargo box based on the target point cloud includes:

[0024] Filling a target grid cell according to the depth information, and calculating the volume of a cuboid corresponding to the target grid cell based on the depth of the container, the target grid cell being the grid cell corresponding to the target point cloud;

[0025] The volume of the target object is calculated according to the volume of the cuboid corresponding to the target grid unit and the number of the target grid units.

[0026] In one embodiment, the method further comprises:

[0027] Construct a supervised training dataset;

[0028] Performing supervised training on the initial depth estimation model using the supervised training dataset to obtain a training loss;

[0029] The initial depth estimation model is updated based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

[0030] In a second aspect, an embodiment of the present application provides an object volume calculation device, comprising:

[0031] An image acquisition unit, used to acquire an image of the interior of the cargo box through a camera;

[0032] a depth acquisition unit, configured to input the internal image into a trained target depth estimation model to obtain depth information of the internal image;

[0033] a point cloud acquisition unit, configured to obtain an initial point cloud of the interior of the cargo box based on the depth information and parameter information of the camera;

[0034] A volume calculation unit is used to calculate the volume of the target object inside the cargo box based on the target point cloud that meets the preset conditions in the initial point cloud, where the target object is the object corresponding to the target point cloud.

[0035] In one embodiment, the parameter information includes focal length, and the point cloud acquisition unit is configured to:

[0036] Obtaining pixel coordinates of pixels in the internal image;

[0037] Obtaining the three-dimensional coordinates of the pixels in the internal image inside the cargo box according to the focal length of the camera, the pixel coordinates of the pixels, and the depth information corresponding to the pixels;

[0038] An initial point cloud of the interior of the cargo box is determined based on the three-dimensional coordinates corresponding to the pixels in the internal image.

[0039] In one embodiment, the volume calculation unit includes:

[0040] The camera distance acquisition module is used to obtain the depth of the cargo box when the vehicle is empty, as well as the distance from the camera installation position to the four planes inside the cargo box;

[0041] a point cloud filtering module, configured to filter the initial point cloud based on the distance to obtain a target point cloud;

[0042] The volume calculation module is used to calculate the volume of the target object inside the cargo box based on the target point cloud and the depth of the cargo box.

[0043] In one embodiment, the point cloud filtering module is used to:

[0044] Projecting the initial point cloud onto a two-dimensional plane to obtain a two-dimensional point coordinate set;

[0045] Performing discrete conversion processing on the two-dimensional point coordinates in the two-dimensional point coordinate set according to a preset size to obtain discretized grid units;

[0046] Traversing the discrete grid cells to determine whether the grid coordinates corresponding to the grid cells are within the distance;

[0047] Point clouds corresponding to grid cells exceeding the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds.

[0048] In one embodiment, the volume calculation module is used to:

[0049] Filling a target grid cell according to the depth information, and calculating the volume of a cuboid corresponding to the target grid cell based on the depth of the container, the target grid cell being the grid cell corresponding to the target point cloud;

[0050] The volume of the target object is calculated according to the volume of the cuboid corresponding to the target grid unit and the number of the target grid units.

[0051] In one embodiment, the apparatus further comprises:

[0052] A training set construction unit, used to construct a supervised training data set;

[0053] A training unit, configured to perform supervised training on a target depth estimation model using the supervised training dataset to obtain a training loss;

[0054] A model updating unit is used to update the target depth estimation model based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

[0055] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for calculating the volume of an item as described in the first aspect above is implemented.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for calculating the volume of an item as described in the first aspect above is implemented.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method for calculating the volume of an item as described in the first aspect above. Beneficial effects

[0058] In an embodiment of the present application, a camera is used to capture an image of the interior of a cargo box. This image is then fed into a trained target depth estimation model, which is used to obtain depth information from the image. An initial point cloud of the interior of the cargo box is then obtained based on this depth information and the camera's parameter information. This eliminates the need for expensive lidar to capture point cloud information, reducing measurement costs. Point cloud acquisition is unaffected by the material of the object. The volume of the target object inside the cargo box is then calculated based on the target point cloud in the initial point cloud that meets preset conditions. This solution can effectively reduce the hardware cost of volume measurement and ensure that the precision and accuracy of the measurement are unaffected by the object's material. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or exemplary technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0060] FIG1 is a flowchart of an implementation method for calculating the volume of an item provided in an embodiment of the present application;

[0061] FIG2 is a flowchart of a specific implementation of training and acquiring a target depth estimation model in the object volume calculation method provided in an embodiment of the present application;

[0062] FIG3 is a flowchart of a specific implementation of step S103 in the method for calculating the volume of an item provided in an embodiment of the present application;

[0063] FIG4 is a flowchart of a specific implementation of step S104 in the method for calculating the volume of an item provided in an embodiment of the present application;

[0064] FIG5 is a schematic diagram of a scenario in which the volume of a rectangular parallelepiped corresponding to a grid unit is calculated in the method for calculating the volume of an object provided in an embodiment of the present application;

[0065] FIG6 is a structural block diagram of an apparatus for calculating the volume of an item provided in an embodiment of the present application;

[0066] FIG7 is a schematic diagram of an electronic device provided in an embodiment of the present application. Modes for Carrying Out the Invention

[0067] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit this application.

[0068] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0069] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0070] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0071] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0072] Currently, the logistics industry generally uses LiDAR to measure the volume of items in cargo boxes. Lasers scan the contents of the boxes with lasers, generating point cloud data that is then used to perform volume measurements. However, LiDAR volume measurement is expensive, and LiDAR has low reflectivity for special items like black objects and glass, which can lead to inaccurate measurements. This makes it impossible to guarantee the precision and accuracy of volume measurements, which in turn affects the sorting and storage of objects and reduces logistics efficiency.

[0073] To address this issue, the present invention provides a method, device, electronic device, and storage medium for calculating the volume of an object. These methods effectively reduce the hardware cost of volume measurement and ensure that the precision and accuracy of measurement are not affected by the material of the object. For more specific technical implementation details of the present invention, please refer to the various embodiments described below.

[0074] The object volume calculation method provided in the embodiments of the present application can be applied to various types of electronic devices that need to perform object volume calculations, specifically including smart vehicle terminals, mobile phones, tablet computers, wearable devices, laptops, desktop computers, servers, etc.

[0075] In order to illustrate the technical solution provided by this application, a detailed description is given below with reference to specific drawings and embodiments.

[0076] FIG1 shows the implementation process of the method for calculating the volume of an item provided in an embodiment of the present application, which includes steps S101 to S104. The specific implementation principles of each step are as follows:

[0077] Step S101: Acquire an image of the interior of the cargo box through a camera.

[0078] In this embodiment, a monocular camera (SLR) is used to capture images of the cargo box's interior. When a monocular camera is used to capture images, light from the subject is focused by the lens and reflected by an angled mirror onto a focusing screen to form an image. Monocular cameras are relatively inexpensive, and using them to capture images and determine point cloud information about the cargo box's interior helps reduce costs, thereby effectively lowering the hardware cost of volume measurement. The type and model of the monocular camera can be determined based on actual application requirements; this embodiment does not impose any restrictions on the type, model, or even brand of the monocular camera.

[0079] Step S102: inputting the internal image into a trained target depth estimation model to obtain depth information of the internal image.

[0080] The target depth estimation model is a trained neural network model used to estimate depth information. In this embodiment, using the target depth estimation model to obtain depth information eliminates the need for expensive hardware equipment such as lidar. Instead, the model can be combined with the 2D information of the image to generate a 3D point cloud, effectively reducing hardware costs.

[0081] Generally, laser point cloud data can be used as ground truth to train a neural network model, resulting in a depth estimation model for estimating depth information. However, this approach requires spatiotemporal calibration of the image and laser point cloud data, which is difficult to implement and results in low training efficiency.

[0082] As a possible implementation of the present application, FIG2 shows a specific implementation process of training and obtaining a target depth estimation model in the object volume calculation method provided in an embodiment of the present application, which is detailed as follows:

[0083] A1: Construct a supervised training dataset.

[0084] The supervised training dataset includes sample video images and sample depth images. Specifically, sample video images of the interior of a cargo box are collected, each containing an item. These sample video images are then fed into the DINOV2 depth estimation model to obtain corresponding sample depth images (or sample depth values). Based on these sample video images and sample depth images (or sample depth values), a supervised training dataset containing these sample video images and sample depth images (or sample depth values) is constructed.

[0085] For example, a supervised training dataset consists of an RGB image captured by a camera and a corresponding depth image containing distance information. First, an RGB image of the container interior is captured. This RGB image is then fed into the DINOV2 depth estimation model to obtain a depth image (or sample depth values) containing distance information. A supervised training dataset is then constructed based on these RGB images and the corresponding depth images (or sample depth values).

[0086] Depth images with distance information are typically acquired using LiDAR point cloud data as ground truth, a costly acquisition method. However, in the present embodiment, the use of the DINOV2 depth estimation model to acquire depth images containing distance information can effectively reduce costs.

[0087] A2: Perform supervised training on the initial depth estimation model using the supervised training dataset to obtain a training loss.

[0088] In this embodiment, the sample video image in the supervised training data set is used as the input of the initial depth estimation model to obtain the first depth image output by the initial depth model. The loss between the first depth image of the initial depth estimation and the sample depth image corresponding to the sample video image in the supervised training data set can be predicted by minimizing the loss function to obtain the training loss for supervised training of the initial depth estimation model.

[0089] A3: Update the initial depth estimation model based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

[0090] In one possible implementation, back propagation is used to update the initial depth estimation model based on the training loss.

[0091] Exemplarily, a parameter gradient is set for each convolution kernel in the initial depth estimation model. When back propagation is used, each model parameter that needs to be updated in the initial depth estimation model has a corresponding gradient. In the process of updating the initial depth estimation model based on the training loss, the model parameters therein can be optimized according to this gradient, thereby reducing the loss. When the training loss is less than or equal to the preset loss threshold, the trained target depth estimation model is obtained.

[0092] In the embodiment of the present application, the trained target depth estimation model is used to determine the depth information of the image inside the cargo box. No additional equipment is required, the process is simple, and the implementation efficiency is high. In addition, the target depth estimation model obtained by combining supervised learning and unsupervised learning methods for model training has higher accuracy, which can effectively improve the accuracy of depth information collection and avoid the influence of object material on the collection of depth information.

[0093] Step S103: obtaining an initial point cloud inside the cargo box according to the depth information and the parameter information of the camera.

[0094] In this embodiment, the depth information and camera parameters are combined to obtain an initial point cloud of the interior of the cargo box. The initial point cloud includes the point cloud of the cargo box and the point cloud of the objects inside the cargo box.

[0095] The camera parameter information includes focal length. As a possible implementation of the present application, FIG3 shows a specific implementation process of obtaining the initial point cloud inside the cargo box based on the depth information and the camera parameter information in the method for calculating the volume of an item provided in an embodiment of the present application, which is detailed as follows:

[0096] B1: Obtain pixel coordinates of pixels in the internal image. Pixel coordinates are two-dimensional coordinates.

[0097] B2: Obtain the three-dimensional coordinates of the pixel in the internal image inside the container based on the focal length of the camera, the pixel coordinates of the pixel, and the depth information corresponding to the pixel. The three-dimensional coordinates are the position coordinates of the pixel coordinates in the world coordinate system.

[0098] In the embodiment of the present application, the three-dimensional coordinates corresponding to the pixel coordinates P(u, v) are determined according to the following formula (1):

[0099] Among them, u represents the horizontal coordinate of the pixel in the internal image in the pixel coordinate system, v represents the vertical coordinate of the pixel in the internal image in the pixel coordinate system, z represents the depth information corresponding to the pixel in the internal image (the depth information is obtained according to the above target depth estimation model), c U and c V is the center coordinate of the pixel coordinate system (usually the center of the image), fU 、f V where x represents the focal length of the camera in the x-axis and y-axis directions in the world coordinate system, respectively. x is the horizontal x-axis position of pixel P(u, v) in the world coordinate system, and y is the vertical y-axis position of pixel P(u, v) in the world coordinate system. Based on equation (1), the three-dimensional coordinates P(x, y, z) corresponding to pixel P(u, v) in the internal image can be obtained.

[0100] B3: Determine an initial point cloud inside the cargo box based on the three-dimensional coordinates corresponding to the pixels in the internal image.

[0101] According to the three-dimensional coordinates corresponding to the pixels in the internal image, the internal image is spatially transformed to obtain an initial point cloud including a cargo box point cloud and a target object point cloud, where the target object refers to an object inside the cargo box.

[0102] In this embodiment, the initial point cloud is determined by combining the internal image of the cargo box taken by the camera, the depth information determined by the target depth estimation model, and the parameter information of the camera, without the need for expensive lidar, which can effectively reduce costs.

[0103] Step S104: Calculating the volume of the target object inside the cargo box based on the target point cloud that meets the preset conditions in the initial point cloud, where the target object is the object corresponding to the target point cloud.

[0104] The above-mentioned target point cloud that meets the preset conditions is the point cloud of the target object. Since the initial point cloud includes the cargo box point cloud, in this embodiment, the initial point cloud is filtered by setting preset conditions, and the point cloud that meets the preset conditions is determined as the target point cloud, that is, the point cloud corresponding to the target object.

[0105] As a possible implementation of the present application, FIG4 shows a specific implementation process of step S104 in the method for calculating the volume of an item provided in an embodiment of the present application, which is described in detail as follows:

[0106] C1: Obtain the depth of the cargo box when empty, and the distance from the camera installation position to four planes inside the cargo box. The four planes inside the cargo box are the left plane, the right plane, the top plane, and the bottom plane.

[0107] C2: Filter the initial point cloud based on the distance to obtain the target point cloud. The distance is the distance from the camera installation position to the four planes inside the cargo box.

[0108] In this embodiment, the distances from the camera to four planes within the cargo box are specifically acquired: the left interior plane, the right interior plane, the top plane, and the bottom plane. The distances from the camera installation position to the four planes within the cargo box include a first distance from the camera to the left interior plane, a second distance from the camera to the right interior plane, a third distance from the camera to the top plane, and a fourth distance from the camera to the bottom plane. The initial point cloud is filtered based on the relationship between the distances from the 3D coordinates in the initial point cloud to the corresponding planes and the distances between the camera and the corresponding planes to obtain the target point cloud.

[0109] As a possible implementation of the present application, the initial point cloud is projected onto a two-dimensional plane. Specifically, the initial point cloud is projected onto a two-dimensional plane along the direction from the camera installation position toward the bottom of the cargo box to obtain a two-dimensional point coordinate set. The two-dimensional point coordinates in the two-dimensional point coordinate set are discretely transformed according to a preset size to obtain a discretized grid unit, wherein a grid unit includes a plurality of dense two-dimensional point coordinates; the discrete grid units are traversed to determine whether the grid coordinates corresponding to the grid units are within the distance from the camera installation position to the four planes inside the cargo box, the point clouds corresponding to the grid units that exceed the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds. Determining whether the grid coordinates corresponding to the grid unit are within the distance from the camera installation position to the four planes inside the cargo box refers to determining whether the grid coordinates corresponding to the grid unit are within the first distance, the second distance, the third distance, or the fourth distance. If the grid coordinates are within any one of the first distance, the second distance, the third distance, or the fourth distance, then the grid coordinates corresponding to the grid unit are determined to be within the distance from the camera installation position to the four planes inside the cargo box. If none of the grid coordinates are within the first distance, the second distance, the third distance, or the fourth distance, then the grid coordinates corresponding to the grid unit are determined to be outside the distance from the camera installation position to the four planes inside the cargo box.

[0110] In this embodiment, all three-dimensional coordinates in the initial point cloud are projected onto a two-dimensional plane to obtain a set of two-dimensional point coordinates on the two-dimensional plane, and the two-dimensional point coordinates are discretely converted according to a preset size. The preset size can be customized, for example, the two-dimensional point coordinates are discretized according to a size of 0.1m×0.1m, and the continuous two-dimensional point coordinates in the two-dimensional point coordinate set are converted into discrete grid units, and the plane area of ​​the grid unit is determined according to the preset size.

[0111] In some embodiments, starting from the bottom of the cargo box, a coordinate system is constructed with the camera installation position as the center, and the discretized grid cells are traversed in order from left to right and from bottom to top to determine whether the grid coordinates corresponding to the grid cells are within the distance between the camera and the cargo box plane. The point clouds corresponding to the grid cells that exceed the distance between the camera and the interior of the cargo box are marked as cargo box point clouds, and the grid cells within the distance between the camera and the interior of the cargo box are marked as target point clouds.

[0112] In some embodiments, a grid unit includes a plurality of dense two-dimensional point coordinates, a central two-dimensional point of the plurality of dense two-dimensional point coordinates is determined, and the central two-dimensional point is determined as the grid coordinate of the grid unit.

[0113] C3: Calculate the volume of the target object inside the cargo box based on the target point cloud and the depth of the cargo box.

[0114] In the embodiment of the present application, the volume of the target object can be calculated based on the target point cloud corresponding to the target object.

[0115] In one possible implementation, the target grid cell is filled according to the depth information, and the volume of the cuboid corresponding to the target grid cell is calculated based on the cargo box depth, where the target grid cell is the grid cell corresponding to the target point cloud; the volume of the target object is calculated based on the volume of the cuboid corresponding to the target grid cell and the number of target grid cells.

[0116] Exemplarily, as shown in Figure 5, the grid cells marked as target items are filled with the depth value z0 to obtain a cuboid. Based on the difference between the cargo box depth L and the depth value z0 in the depth information, the length len of the cuboid can be determined. Combined with the preset size x0×y0 of the grid cell, the volume of the cuboid can be calculated. For example, if the preset size of the grid cell is 0.1×0.1, the volume of the cuboid corresponding to the grid cell is 0.1×0.1×len, len=(L-z0), and then by accumulating the volumes of all target grid cells, the volume of the target object inside the cargo box can be obtained.

[0117] As can be seen from the above, in the embodiment of the present application, an image of the interior of the cargo box is obtained by a camera, the image is input into a trained target depth estimation model, and the depth information of the image is obtained using the target depth estimation model. Then, based on the depth information and the parameter information of the camera, an initial point cloud of the interior of the cargo box is obtained. This eliminates the need to use a high-cost lidar to collect point cloud information, thereby reducing measurement costs. The collection of point clouds is not affected by the material of the object. The volume of the target object inside the cargo box is then calculated based on the target point cloud that meets the preset conditions in the initial point cloud. This application solution can effectively reduce the hardware cost of volume measurement and ensure that the precision and accuracy of the measurement are not affected by the material of the object.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0119] Corresponding to the method for calculating the volume of items described in the above embodiment, FIG6 shows a structural block diagram of the device for calculating the volume of items provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0120] 6 , the device for calculating the volume of an object includes: an image acquisition unit 61 , a depth acquisition unit 62 , a point cloud acquisition unit 63 , and a volume calculation unit 64 , wherein:

[0121] An image acquisition unit 61 is used to acquire an image of the interior of the cargo box through a camera;

[0122] a depth acquisition unit 62, configured to input the internal image into a trained target depth estimation model to obtain depth information of the internal image;

[0123] a point cloud acquisition unit 63, configured to obtain an initial point cloud of the interior of the cargo box according to the depth information and the parameter information of the camera;

[0124] The volume calculation unit 64 is configured to calculate the volume of a target object inside the container based on a target point cloud that meets a preset condition in the initial point cloud, where the target object is the object corresponding to the target point cloud.

[0125] As a possible implementation manner of the present application, the device further includes:

[0126] A training set construction unit, used to construct a supervised training data set;

[0127] A training unit, configured to perform supervised training on a target depth estimation model using the supervised training dataset to obtain a training loss;

[0128] A model updating unit is used to update the target depth estimation model based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

[0129] As a possible implementation of the present application, the parameter information includes focal length, and the point cloud acquisition unit 63 is specifically configured to:

[0130] Obtaining pixel coordinates of pixels in the internal image;

[0131] Obtaining the three-dimensional coordinates of the pixels in the internal image inside the cargo box according to the focal length of the camera, the pixel coordinates of the pixels, and the depth information corresponding to the pixels;

[0132] An initial point cloud of the interior of the cargo box is determined based on the three-dimensional coordinates corresponding to the pixels in the internal image.

[0133] As a possible implementation of the present application, the volume calculation unit 64 includes:

[0134] The camera distance acquisition module is used to obtain the depth of the cargo box when the vehicle is empty, as well as the distance from the camera installation position to the four planes inside the cargo box;

[0135] a point cloud filtering module, configured to filter the initial point cloud based on the distance to obtain a target point cloud;

[0136] The volume calculation module is used to calculate the volume of the target object inside the cargo box based on the target point cloud and the depth of the cargo box.

[0137] As a possible implementation of the present application, the point cloud filtering module is specifically used to:

[0138] Projecting the initial point cloud onto a two-dimensional plane to obtain a two-dimensional point coordinate set;

[0139] Performing discrete conversion processing on the two-dimensional point coordinates in the two-dimensional point coordinate set according to a preset size to obtain discretized grid units;

[0140] Traversing the discrete grid cells to determine whether the grid coordinates corresponding to the grid cells are within the distance;

[0141] Point clouds corresponding to grid cells exceeding the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds.

[0142] As a possible implementation of the present application, the volume calculation module is specifically used to:

[0143] Filling a target grid cell according to the depth information, and calculating the volume of a cuboid corresponding to the target grid cell based on the depth of the container, the target grid cell being the grid cell corresponding to the target point cloud;

[0144] The volume of the target object is calculated according to the volume of the cuboid corresponding to the target grid unit and the number of the target grid units.

[0145] As can be seen from the above, in the embodiment of the present application, an image of the interior of the cargo box is obtained by a camera, the image is input into a trained target depth estimation model, and the depth information of the image is obtained using the target depth estimation model. Then, based on the depth information and the parameter information of the camera, an initial point cloud of the interior of the cargo box is obtained. This eliminates the need to use a high-cost lidar to collect point cloud information, thereby reducing measurement costs. The collection of point clouds is not affected by the material of the object. The volume of the target object inside the cargo box is then calculated based on the target point cloud that meets the preset conditions in the initial point cloud. This application solution can effectively reduce the hardware cost of volume measurement and ensure that the precision and accuracy of the measurement are not affected by the material of the object.

[0146] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0147] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the steps of any one of the object volume calculation methods shown in Figures 1 to 5 are implemented.

[0148] An embodiment of the present application also provides an electronic device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of any one of the object volume calculation methods shown in Figures 1 to 5 are implemented.

[0149] An embodiment of the present application also provides a computer-readable instruction product, which, when executed on an electronic device, enables the electronic device to execute the steps of any one of the object volume calculation methods shown in FIG. 1 to FIG. 5 .

[0150] Figure 7 is a schematic diagram of an electronic device provided in one embodiment of the present application. As shown in Figure 7 , the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and computer-readable instructions 72 stored in the memory 71 and executable by the processor 70. When the processor 70 executes the computer-readable instructions 72, it implements the steps in the aforementioned embodiments of the method for calculating the volume of an item, such as steps S101 to S104 shown in Figure 1 . Alternatively, when the processor 70 executes the computer-readable instructions 72, it implements the functions of the modules / units in the aforementioned device embodiments, such as the functions of units 61 to 64 shown in Figure 6 .

[0151] Exemplarily, the computer-readable instructions 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to implement the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which are used to describe the execution process of the computer-readable instructions 72 in the electronic device 7.

[0152] The electronic device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that FIG7 is merely an example of the electronic device 7 and does not limit the electronic device 7. The electronic device 7 may include more or fewer components than shown in the figure, or may combine certain components or different components. For example, the electronic device 7 may also include input and output devices, network access devices, buses, and the like.

[0153] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0154] The memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. The memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Furthermore, the memory 71 may include both an internal storage unit of the electronic device 7 and an external storage device. The memory 71 is used to store the computer-readable instructions and other programs and data required by the electronic device. The memory 71 may also be used to temporarily store data that has been output or is about to be output.

[0155] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Wherein, the computer-readable instructions include computer-readable instruction codes, which can be in source code form, object code form, executable files or some intermediate forms. The computer-readable medium can at least include: any entity or device that can carry the computer-readable instruction code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0158] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0159] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for calculating the volume of an item, characterized in that: include: Acquire images of the interior of the cargo box through a camera; Inputting the internal image into a trained target depth estimation model to obtain depth information of the internal image; Obtaining an initial point cloud inside the cargo box based on the depth information and parameter information of the camera; The volume of the target object inside the cargo box is calculated based on the target point cloud that meets the preset conditions in the initial point cloud, and the target object is the object corresponding to the target point cloud.

2. The method according to claim 1, characterized in that The parameter information includes focal length; and obtaining an initial point cloud inside the container based on the depth information and the parameter information of the camera includes: Obtaining pixel coordinates of pixels in the internal image; Obtaining the three-dimensional coordinates of the pixels in the internal image inside the cargo box according to the focal length of the camera, the pixel coordinates of the pixels, and the depth information corresponding to the pixels; An initial point cloud of the interior of the cargo box is determined based on the three-dimensional coordinates corresponding to the pixels in the internal image.

3. The method according to claim 1, characterized in that Calculating the volume of the target object inside the container based on the target point cloud that meets the preset conditions in the initial point cloud includes: Obtain the depth of the cargo box when empty, as well as the distance from the camera installation location to the four planes inside the cargo box; Filtering the initial point cloud based on the distance to obtain a target point cloud; The volume of the target object inside the cargo box is calculated based on the target point cloud and the depth of the cargo box.

4. The method according to claim 3, characterized in that The filtering of the initial point cloud based on the distance to obtain a target point cloud includes: Projecting the initial point cloud onto a two-dimensional plane to obtain a two-dimensional point coordinate set; Performing discrete conversion processing on the two-dimensional point coordinates in the two-dimensional point coordinate set according to a preset size to obtain discretized grid units; Traversing the discrete grid cells to determine whether the grid coordinates corresponding to the grid cells are within the distance; Point clouds corresponding to grid cells exceeding the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds.

5. The method according to claim 4, characterized in that Calculating the volume of the target object inside the cargo box based on the target point cloud includes: Fill the target grid cells according to the depth information, and calculate the target grid cells based on the depth of the container. The unit corresponds to the volume of a cuboid, and the target grid unit is the grid unit corresponding to the target point cloud; The volume of the target object is calculated according to the volume of the cuboid corresponding to the target grid unit and the number of the target grid units.

6. The method according to claim 1, characterized in that The method further comprises: Construct a supervised training dataset; Performing supervised training on the initial depth estimation model using the supervised training dataset to obtain a training loss; The initial depth estimation model is updated based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

7. An item volume calculation device, characterized in that: include: An image acquisition unit, used to acquire an image of the interior of the cargo box through a camera; a depth acquisition unit, configured to input the internal image into a trained target depth estimation model to obtain depth information of the internal image; a point cloud acquisition unit, configured to obtain an initial point cloud of the interior of the cargo box based on the depth information and parameter information of the camera; A volume calculation unit is used to calculate the volume of the target object inside the cargo box based on the target point cloud that meets the preset conditions in the initial point cloud, where the target object is the object corresponding to the target point cloud.

8. The device according to claim 7, characterized in that The parameter information includes focal length, and the point cloud acquisition unit is used to: Obtaining pixel coordinates of pixels in the internal image; Obtaining the three-dimensional coordinates of the pixels in the internal image inside the cargo box according to the focal length of the camera, the pixel coordinates of the pixels, and the depth information corresponding to the pixels; An initial point cloud of the interior of the cargo box is determined based on the three-dimensional coordinates corresponding to the pixels in the internal image.

9. The device according to claim 7, characterized in that The volume calculation unit includes: The camera distance acquisition module is used to obtain the depth of the cargo box when the vehicle is empty, as well as the distance from the camera installation position to the four planes inside the cargo box; a point cloud filtering module, configured to filter the initial point cloud based on the distance to obtain a target point cloud; The volume calculation module is used to calculate the volume of the target object inside the cargo box based on the target point cloud and the depth of the cargo box.

10. The device according to claim 9, characterized in that The point cloud filtering module is used to: Projecting the initial point cloud onto a two-dimensional plane to obtain a two-dimensional point coordinate set; The two-dimensional point coordinates in the two-dimensional point coordinate set are discretely transformed according to a preset size to obtain a discretized Grid cells; Traversing the discrete grid cells to determine whether the grid coordinates corresponding to the grid cells are within the distance; Point clouds corresponding to grid cells exceeding the distance are filtered out from the initial point cloud, and the remaining point clouds in the initial point cloud are determined as target point clouds.

11. The device according to claim 10, characterized in that The volume calculation module is used to: Filling a target grid cell according to the depth information, and calculating the volume of a cuboid corresponding to the target grid cell based on the depth of the container, the target grid cell being the grid cell corresponding to the target point cloud; The volume of the target object is calculated according to the volume of the cuboid corresponding to the target grid unit and the number of the target grid units.

12. The device according to claim 7, characterized in that The device further comprises: A training set construction unit, used to construct a supervised training data set; A training unit, configured to perform supervised training on a target depth estimation model using the supervised training dataset to obtain a training loss; A model updating unit is used to update the target depth estimation model based on the training loss until the training loss meets a preset loss condition, thereby obtaining a trained target depth estimation model.

13. An electronic device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein: When the processor executes the computer-readable instructions, the method for calculating the volume of an item according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the method for calculating the volume of an item according to any one of claims 1 to 6 is implemented.

Images