Object volume measurement method, device, equipment and storage medium based on point cloud

Abstract

The embodiment of the present invention provides a method, device, equipment and storage medium for measuring the volume of an object based on point clouds, the method comprising: acquiring multiple frames of images obtained by photographing a target object; performing point cloud data processing on each frame of images to obtain point cloud data corresponding to each frame of images; filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data to obtain target point cloud data; and determining the volume corresponding to the target object based on the target point cloud data corresponding to each frame of images. In the embodiment of the present invention, the volume corresponding to the target object is determined based on the target point cloud data corresponding to each frame of images, the target point cloud data filters out erroneous point clouds determined according to the depth corresponding to each point cloud, and the accuracy of the volume measurement result of the object is improved by filtering out erroneous point clouds that affect the accuracy of the volume measurement result.

Description

Technical Field

[0001] The present invention relates to the field of point cloud processing technology, and in particular to a point cloud-based object volume measurement method, device, equipment and storage medium. Background Art

[0002] In the freight field, cargo volume measurement is a painful demand. Users need to fill in the cargo volume in the freight order, and the freight platform calculates the freight cost based on information such as cargo volume and freight distance.

[0003] In the prior art, users can only estimate the volume of goods and then fill in the estimated volume of goods in the freight order. However, the estimated volume of goods is often inaccurate, which results in large errors in freight costs.

[0004] Therefore, how to accurately measure the volume of an object is a technical problem to be solved in this field. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, device, computer equipment and storage medium for measuring the volume of an object based on point cloud, aiming to solve the technical problem of not being able to accurately measure the volume of an object.

[0006] To achieve the above object, the present invention provides a method for measuring the volume of an object based on a point cloud, the method comprising:

[0007] Acquire multiple frames of images obtained by shooting the target object;

[0008] Performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image;

[0009] According to the depth corresponding to each point cloud in the point cloud data, filter out erroneous point clouds in the point cloud data to obtain target point cloud data;

[0010] Based on the target point cloud data corresponding to each frame of image, the volume corresponding to the target object is determined.

[0011] Optionally, filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data includes:

[0012] Performing image segmentation on the point cloud data corresponding to each frame of the image to obtain point cloud data within the mask area;

[0013] Calculate the distance between each point cloud in the mask area and the optical center of the camera to obtain the depth corresponding to each point cloud;

[0014] According to the depth corresponding to each point cloud, erroneous point clouds in the point cloud data are filtered out.

[0015] Optionally, filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud includes:

[0016] Sorting each of the point clouds in descending order of corresponding depth;

[0017] The first N point clouds in the sorting are determined as candidate point clouds, and N candidate point clouds are obtained, where N is a positive integer greater than 1;

[0018] For any candidate point cloud, determine K reference point clouds associated with the candidate point cloud, where K is a positive integer greater than 1;

[0019] Determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud;

[0020] Filter out erroneous point clouds from the N candidate point clouds.

[0021] Optionally, determining K reference point clouds associated with the candidate point cloud includes:

[0022] Determine a plurality of first point clouds associated with the candidate point cloud; the depth corresponding to the first point cloud is less than or equal to the depth corresponding to the candidate point cloud;

[0023] K first point clouds closest to the candidate point cloud among the multiple first point clouds are determined as the K reference point clouds.

[0024] Optionally, determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud includes:

[0025] For any candidate point cloud, calculate the average depth corresponding to the K reference point clouds associated with the candidate point cloud;

[0026] According to the average depth, calculating the offset depth corresponding to the candidate point cloud and the average deviation of the K reference point clouds relative to the average depth;

[0027] When the offset depth is greater than the target value, determining the candidate point cloud as an erroneous point cloud;

[0028] When the offset depth is not greater than the target value, determining that the candidate point cloud is not an erroneous point cloud;

[0029] The target value is the product of a preset threshold and the average deviation.

[0030] Optionally, the method is applied to an electronic device, the electronic device comprising a camera, a first inertial element, and a second inertial element;

[0031] The step of performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image includes:

[0032] For each frame of image, applying a preset SLAM algorithm based on the image, preset pre-integration parameters, the preset origin of the first inertial element coordinate system, the preset origin of the second inertial element coordinate system, the preset origin of the camera coordinate system, and linear transformation parameters, to obtain point cloud data corresponding to each frame of image;

[0033] The linear transformation parameters are used to characterize the linear transformation relationship between the first inertial element coordinate system and the camera coordinate system, and the linear transformation relationship between the second inertial element coordinate system and the camera coordinate system.

[0034] Optionally, the linear conversion parameter is determined by the following formula:

[0035] +

[0036] in, is the coordinate of point P in the first inertial element coordinate system or the coordinate of point P in the second inertial element coordinate system, is the coordinate of point P in the camera coordinate system, is the external parameter of the camera relative to the first inertial element, is the external parameter of the camera relative to the second inertial element, and the first linear transformation parameters include and .

[0037] In addition, to achieve the above-mentioned purpose, the present invention further provides a point cloud-based object volume measurement device, the point cloud-based object volume measurement device comprising:

[0038] An acquisition module, used to acquire multiple frames of images obtained by photographing a target object;

[0039] A processing module, used for performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image;

[0040] A filtering module, used for filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data, so as to obtain target point cloud data;

[0041] The determination module is used to determine the volume corresponding to the target object based on the target point cloud data corresponding to each frame of image.

[0042] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0043] The computer device includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of any one of the point cloud-based object volume measurement methods proposed in the embodiments of the present application are implemented.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0045] The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of any one of the point cloud-based object volume measurement methods proposed in the embodiments of the present application are implemented.

[0046] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0047] The embodiment of the present invention provides a method, device, equipment and storage medium for measuring the volume of an object based on point clouds, the method comprising: acquiring multiple frames of images obtained by photographing a target object; performing point cloud data processing on each frame of images to obtain point cloud data corresponding to each frame of images; filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data to obtain target point cloud data; and determining the volume corresponding to the target object based on the target point cloud data corresponding to each frame of images. In the embodiment of the present invention, the volume corresponding to the target object is determined based on the target point cloud data corresponding to each frame of images, the target point cloud data filters out erroneous point clouds determined according to the depth corresponding to each point cloud, and the accuracy of the volume measurement result of the object is improved by filtering out erroneous point clouds that affect the accuracy of the volume measurement result. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0050] Figure 2 is a flow chart of a method for measuring object volume based on point cloud provided by an embodiment of the present invention;

[0051] Figure 3 It is one of the projection schematic diagrams of the object volume measurement method based on point cloud provided by an embodiment of the present invention;

[0052] Figure 4This is the second projection schematic diagram of the object volume measurement method based on point cloud provided by an embodiment of the present invention;

[0053] Figure 5 is an application flow chart of the object volume measurement method based on point cloud provided by an embodiment of the present invention;

[0054] Figure 6 is a schematic structural diagram of an embodiment of a point cloud-based object volume measurement device according to the present application;

[0055] Figure 7 This is a basic structural block diagram of the computer device in this embodiment. DETAILED DESCRIPTION

[0056] The point cloud-based object volume measurement method provided by an embodiment of the present invention is applied to a point cloud-based object volume measurement device. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" in the specification and claims of the present application and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0057] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0060] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social online platform software, etc.

[0061] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0062] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0063] It should be noted that the point cloud-based object volume measurement method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the point cloud-based object volume measurement device is generally set in the server / terminal device.

[0064] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0065] Please refer to Figure 2 , shows a flow chart of an embodiment of a method for measuring the volume of an object based on point cloud proposed in the present application. The embodiment of the present application can acquire and process relevant data based on artificial intelligence technology.

[0066] The object volume measurement method based on point cloud provided by the embodiment of the present invention comprises the following steps:

[0067] S210, acquiring multiple frames of images obtained by photographing the target object.

[0068] The target object mentioned above is the object to be measured.

[0069] In an optional freight scenario, the object volume measurement method based on point cloud provided by an embodiment of the present invention can be applied to an electronic device used by a user. In the freight scenario, a user can use a general electronic device to shoot a target object to obtain multiple frames of images, wherein the general electronic device includes but is not limited to non-professional point cloud measurement devices such as mobile phones and tablet computers.

[0070] S220, performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image.

[0071] In this step, after obtaining multiple frames of images, point cloud data processing is performed on each frame of image to obtain point cloud data corresponding to each frame of image. For specific implementation methods, please refer to the subsequent embodiments.

[0072] S230, filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data, to obtain target point cloud data.

[0073] It should be understood that when the application scenario is the freight scenario mentioned above, the depth corresponding to the point cloud can be understood as the distance between the point cloud and the optical center of the camera in the electronic device; in other application scenarios, the depth corresponding to the point cloud can be understood as the distance between the point cloud and the preset expected point.

[0074] In this step, for the point cloud data corresponding to each frame of image, first determine the depth corresponding to each point cloud in the point cloud data, and then filter out the erroneous point clouds in the point cloud data corresponding to the frame of image according to the depth corresponding to each point cloud to obtain the target point cloud data, and then obtain the target point cloud data corresponding to each frame of image.

[0075] S240, determining a volume corresponding to the target object based on the target point cloud data corresponding to each frame of image.

[0076] In this step, after obtaining the target point cloud data corresponding to each frame of image, the volume corresponding to the target object can be determined based on the target point cloud data corresponding to each frame of image. A specific implementation method can be to fit the target point cloud data to obtain a hexahedron, and then calculate the volume of the hexahedron, and determine the volume as the volume corresponding to the target object.

[0077] In an embodiment of the present invention, the volume corresponding to the target object is determined based on the target point cloud data corresponding to each frame of the image. The target point cloud data filters out erroneous point clouds determined according to the depth corresponding to each point cloud. By filtering out erroneous point clouds that affect the accuracy of the volume measurement results, the accuracy of the volume measurement results of the object is improved.

[0078] Optionally, filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data includes:

[0079] Performing image segmentation on the point cloud data corresponding to each frame of the image to obtain point cloud data within the mask area;

[0080] Calculate the distance between each point cloud in the mask area and the optical center of the camera to obtain the depth corresponding to each point cloud;

[0081] According to the depth corresponding to each point cloud, erroneous point clouds in the point cloud data are filtered out.

[0082] In this embodiment, image segmentation is performed on the point cloud data corresponding to each frame of the image. In image segmentation, the mask can be used to mark and distinguish different areas to obtain the point cloud data within the mask area.

[0083] The distance between each point cloud in the mask area and the optical center of the camera is calculated, and the distance between each point cloud and the optical center of the camera is determined as the depth corresponding to each point cloud. Further, according to the depth corresponding to each point cloud, the erroneous point clouds in the point cloud data are filtered out. For the specific technical solution of how to filter out the erroneous point clouds, please refer to the subsequent embodiments.

[0084] Optionally, filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud includes:

[0085] Sorting each of the point clouds in descending order of corresponding depth;

[0086] The first N point clouds in the sorting are determined as candidate point clouds, and N candidate point clouds are obtained, where N is a positive integer greater than 1;

[0087] For any candidate point cloud, determine K reference point clouds associated with the candidate point cloud, where K is a positive integer greater than 1;

[0088] Determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud;

[0089] Filter out erroneous point clouds from the N candidate point clouds.

[0090] In this embodiment, after determining the depth corresponding to each point cloud, each point cloud is sorted in descending order according to the corresponding depth; the first N point clouds in the sorting are determined as candidate point clouds, and N candidate point clouds are obtained. Wherein, N is a positive integer greater than 1, and it should be understood that N is a custom value.

[0091] For each candidate point cloud, K reference point clouds associated with the candidate point cloud are determined, where K is a positive integer greater than 1. It should be understood that K is a custom-set value.

[0092] For easier understanding, see Figure 3 ,like Figure 3As shown, the point cloud in the two-dimensional projection space of the camera is determined as the point cloud located in the mask area, and the candidate two-dimensional projection points and other two-dimensional projection points are obtained according to the depth between each point cloud in the mask area and the optical center of the camera, where the candidate two-dimensional projection points are also called candidate point clouds.

[0093] Further, according to the depth corresponding to each reference point cloud, it is determined whether the candidate point cloud is an erroneous point cloud, and then the erroneous point clouds in the N candidate point clouds are filtered out. For specific implementation methods of how to determine whether a candidate point cloud is an erroneous point cloud, please refer to the subsequent embodiments.

[0094] Optionally, determining K reference point clouds associated with the candidate point cloud includes:

[0095] Determine a plurality of first point clouds associated with the candidate point cloud; the depth corresponding to the first point cloud is less than or equal to the depth corresponding to the candidate point cloud;

[0096] K first point clouds closest to the candidate point cloud among the multiple first point clouds are determined as the K reference point clouds.

[0097] In this embodiment, for each candidate point cloud, a point cloud whose corresponding depth is less than or equal to the depth corresponding to the candidate point cloud is obtained, and the point cloud is determined as the first point cloud, so as to obtain multiple first point clouds associated with the candidate point cloud.

[0098] The distance between each first point cloud and the candidate point cloud is obtained, and then K first point clouds closest to the candidate point cloud among the multiple first point clouds are determined as K reference point clouds.

[0099] To understand the technical solution provided by this embodiment, please refer to Figure 4 , Figure 4 The candidate two-dimensional points shown are the candidate point clouds in this embodiment. By the above method, the candidate two-dimensional points located at Figure 4 The point cloud within the reference point area is determined as the first point cloud.

[0100] Optionally, determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud includes:

[0101] For any candidate point cloud, calculate the average depth corresponding to the K reference point clouds associated with the candidate point cloud;

[0102] According to the average depth, calculating the offset depth corresponding to the candidate point cloud and the average deviation of the K reference point clouds relative to the average depth;

[0103] When the offset depth is greater than the target value, determining the candidate point cloud as an erroneous point cloud;

[0104] When the offset depth is not greater than the target value, determining that the candidate point cloud is not an erroneous point cloud;

[0105] The target value is the product of a preset threshold and the average deviation.

[0106] In this embodiment, the depth corresponding to the candidate point cloud can be defined as p_d, the depths corresponding to the K reference point clouds can be defined as d1, d2, ..., dk, and the average depth corresponding to the K reference point clouds can be calculated. Optionally, it can be obtained by the following formula:

[0107] mean_d = sum(d1 + d2 + … + dk) / k

[0108] Among them, mean_d represents the average depth corresponding to K reference point clouds.

[0109] The offset depth corresponding to the candidate point cloud is determined based on the following formula:

[0110] d_d = abs(p_d - mean_d)

[0111] Among them, d_d represents the offset depth corresponding to the candidate point cloud, p_d represents the depth corresponding to the candidate point cloud, and mean_d represents the average depth corresponding to K reference point clouds.

[0112] The average deviation of the K reference point clouds relative to the average depth can be determined based on the following formula:

[0113] std = sqrt( pow((d1 - mean_d),2) + pow((d2 - mean_d),2) + ... + pow((dk - mean_d),2) ) / k

[0114] Here, std represents the mean deviation.

[0115] Furthermore, the product of the preset threshold and the average deviation is determined as the target value.

[0116] When the offset depth is greater than the target value, it means that the candidate point cloud is not distributed on the surface of the object, and the candidate point cloud is determined to be an erroneous point cloud; when the offset depth is less than or equal to the target value, it means that the candidate point cloud is distributed on the surface of the object, and the candidate point cloud is determined not to be an erroneous point cloud.

[0117] In this embodiment, the candidate point cloud is determined to be distributed on the surface of the object according to the offset depth corresponding to the candidate point cloud, the average deviation corresponding to the K reference point clouds, and the preset threshold, so that the candidate point cloud that is not distributed on the surface of the object is filtered out. By filtering out the erroneous point cloud that affects the accuracy of the volume measurement result, the accuracy of the volume measurement result of the object is improved.

[0118] Optionally, the method is applied to an electronic device, the electronic device comprising a camera, a first inertial element, and a second inertial element;

[0119] The step of performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image includes:

[0120] For each frame of image, applying a preset SLAM algorithm based on the image, preset pre-integration parameters, the preset origin of the first inertial element coordinate system, the preset origin of the second inertial element coordinate system, the preset origin of the camera coordinate system, and linear transformation parameters, to obtain point cloud data corresponding to each frame of image;

[0121] The linear transformation parameters are used to characterize the linear transformation relationship between the first inertial element coordinate system and the camera coordinate system, and the linear transformation relationship between the second inertial element coordinate system and the camera coordinate system.

[0122] It should be noted that in the prior art, professional point cloud measurement equipment includes a camera and an inertial element, and the linear conversion relationship between the camera coordinate system and the inertial element coordinate system is known, so the target object can be photographed by the point cloud measurement equipment, and then the point cloud data corresponding to the target object can be determined based on the photographed image. Among them, the camera coordinate system is a coordinate system formed with the optical center of the camera as the origin, and the inertial element coordinate system is a coordinate system formed with the inertial element as the origin.

[0123] For general electronic devices, such as mobile phones, tablet computers and other devices, different electronic devices may include cameras and inertial elements of different specifications, which makes the linear conversion relationship between the camera coordinate system and the inertial element coordinate system unknown. Therefore, in the prior art, it is impossible to use general electronic devices to accurately measure the volume of objects.

[0124] In order to solve the above technical problems, in this embodiment, when the point cloud-based object volume measurement method is applied to an electronic device, the electronic device includes a camera, a first inertial element and a second inertial element, wherein the first inertial element may be an accelerometer and the second inertial element may be a gyroscope.

[0125] It should be noted that the Simultaneous Localization And Mapping (SLAM) algorithm is a technology used in the field of robotics and computer vision to achieve autonomous positioning and navigation of robots. The SLAM algorithm uses sensors to perceive the surrounding environment, and estimates the position and posture of the robot based on this perception information, and builds an environmental map.

[0126] In this embodiment, the image, the preset pre-integration parameters, the preset origin of the first inertial element coordinate system, the preset origin of the second inertial element coordinate system, the preset origin of the camera coordinate system and the linear transformation parameters are input into the SLAM algorithm, and the point cloud data corresponding to each frame of the image is obtained through the SLAM algorithm.

[0127] The linear transformation parameters are used to characterize the linear transformation relationship between the first inertial element coordinate system and the camera coordinate system, and the linear transformation relationship between the second inertial element coordinate system and the camera coordinate system.

[0128] In this embodiment, by applying the SLAM algorithm, general electronic devices can also accurately measure the volume of objects, thereby expanding the technical means for measuring the volume of objects.

[0129] Optionally, the linear conversion parameter is determined by the following formula:

[0130] +

[0131] in, is the coordinate of point P in the first inertial element coordinate system or the coordinate of point P in the second inertial element coordinate system, is the coordinate of point P in the camera coordinate system, is the external parameter of the camera relative to the first inertial element, is the external parameter of the camera relative to the second inertial element, and the first linear transformation parameters include and .

[0132] To understand the overall technical solution, please refer to Figure 5 ,like Figure 5 As shown, the electronic device captures an image of the target object, processes the point cloud data for each frame of the image, and obtains the point cloud data corresponding to each frame of the image; calculates the depth between each point cloud and the optical center of the camera, sorts the point clouds from large to small according to the depth, and obtains N candidate point clouds, and then performs external point judgment on each candidate point cloud, determines the candidate point clouds that meet the external point conditions as error point clouds, and filters out the error point clouds to obtain the target point cloud data, and then generates a three-dimensional wrapping frame based on the target point cloud data, and determines the volume of the three-dimensional wrapping frame as the volume of the target object.

[0133] Further references Figure 6 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a point cloud-based object volume measurement device 600. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0134] An embodiment of the present invention provides a point cloud-based object volume measurement device 600, and the point cloud-based object volume measurement device 600 includes:

[0135] An acquisition module 610 is used to acquire multiple frames of images obtained by photographing a target object;

[0136] The processing module 620 is used to process the point cloud data of each frame of image to obtain the point cloud data corresponding to each frame of image;

[0137] A filtering module 630 is used to filter out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data to obtain target point cloud data;

[0138] The determination module 640 is used to determine the volume corresponding to the target object based on the target point cloud data corresponding to each frame of image.

[0139] Optionally, the filtering module 630 is specifically used to:

[0140] Performing image segmentation on the point cloud data corresponding to each frame of the image to obtain point cloud data within the mask area;

[0141] Calculate the distance between each point cloud in the mask area and the optical center of the camera to obtain the depth corresponding to each point cloud;

[0142] According to the depth corresponding to each point cloud, erroneous point clouds in the point cloud data are filtered out.

[0143] Optionally, the filtering module 630 is further specifically configured to:

[0144] Sorting each of the point clouds in descending order of corresponding depth;

[0145] The first N point clouds in the sorting are determined as candidate point clouds, and N candidate point clouds are obtained, where N is a positive integer greater than 1;

[0146] For any candidate point cloud, determine K reference point clouds associated with the candidate point cloud, where K is a positive integer greater than 1;

[0147] Determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud;

[0148] Filter out erroneous point clouds from the N candidate point clouds.

[0149] Optionally, the filtering module 630 is further specifically configured to:

[0150] Determine a plurality of first point clouds associated with the candidate point cloud; the depth corresponding to the first point cloud is less than or equal to the depth corresponding to the candidate point cloud;

[0151] K first point clouds closest to the candidate point cloud among the multiple first point clouds are determined as the K reference point clouds.

[0152] Optionally, the filtering module 630 is further specifically configured to:

[0153] For any candidate point cloud, calculate the average depth corresponding to the K reference point clouds associated with the candidate point cloud;

[0154] According to the average depth, calculating the offset depth corresponding to the candidate point cloud and the average deviation of the K reference point clouds relative to the average depth;

[0155] When the offset depth is greater than the target value, determining the candidate point cloud as an erroneous point cloud;

[0156] When the offset depth is not greater than the target value, determining that the candidate point cloud is not an erroneous point cloud;

[0157] The target value is the product of a preset threshold and the average deviation.

[0158] Optionally, the object volume measurement device 600 based on point cloud includes a camera, a first inertial element and a second inertial element;

[0159] The processing module 620 is specifically used for:

[0160] For each frame of image, applying a preset SLAM algorithm based on the image, preset pre-integration parameters, the preset origin of the first inertial element coordinate system, the preset origin of the second inertial element coordinate system, the preset origin of the camera coordinate system, and linear transformation parameters, to obtain point cloud data corresponding to each frame of image;

[0161] The linear transformation parameters are used to characterize the linear transformation relationship between the first inertial element coordinate system and the camera coordinate system, and the linear transformation relationship between the second inertial element coordinate system and the camera coordinate system.

[0162] Optionally, the linear conversion parameter is determined by the following formula:

[0163] +

[0164] in, is the coordinate of point P in the first inertial element coordinate system or the coordinate of point P in the second inertial element coordinate system, is the coordinate of point P in the camera coordinate system, is the external parameter of the camera relative to the first inertial element, is the external parameter of the camera relative to the second inertial element, and the first linear transformation parameters include and .

[0165] To solve the above technical problems, the present application also provides a computer device. Figure 7 , Figure 7 This is a basic structural block diagram of the computer device in this embodiment.

[0166] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 7 with components 71-73, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0167] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0168] The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 71 can be an internal storage unit of the computer device 7, such as a hard disk or memory of the computer device 7. In other embodiments, the memory 71 can also be an external storage device of the computer device 7, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. equipped on the computer device 7. Of course, the memory 71 can also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7, such as the program code of the object volume measurement method based on point cloud, etc. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or are to be output.

[0169] The processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 72 is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to run the program code stored in the memory 71 or process data, such as running the program code of the point cloud-based object volume measurement method.

[0170] The network interface 73 may include a wireless network interface or a wired network interface. The network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0171] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores the point cloud-based object volume measurement program, and the point cloud-based object volume measurement program can be executed by at least one processor so that the at least one processor performs the steps of the point cloud-based object volume measurement method as described above.

[0172] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware online platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0173] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0174] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for measuring the volume of an object based on point cloud, characterized in that: include: Acquire multiple frames of images obtained by shooting the target object; Performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image; According to the depth corresponding to each point cloud in the point cloud data, filter out erroneous point clouds in the point cloud data to obtain target point cloud data; Determine the volume of the target object based on the target point cloud data corresponding to each frame of the image; The filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data comprises: Performing image segmentation on the point cloud data corresponding to each frame of the image to obtain point cloud data within the mask area; Calculate the distance between each point cloud in the mask area and the optical center of the camera to obtain the depth corresponding to each point cloud; Filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud; The filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud includes: Sorting each of the point clouds in descending order of corresponding depth; Determine the first N point clouds in the sorting as candidate point clouds, and obtain N candidate point clouds, where N is a positive integer greater than 1, and the candidate point clouds are located in a two-dimensional projection space; For any candidate point cloud, determine K reference point clouds associated with the candidate point cloud, where K is a positive integer greater than 1; Determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud; Filter out erroneous point clouds from the N candidate point clouds.

2. The method according to claim 1, characterized in that The determining K reference point clouds associated with the candidate point cloud comprises: Determine a plurality of first point clouds associated with the candidate point cloud; the depth corresponding to the first point cloud is less than or equal to the depth corresponding to the candidate point cloud; K first point clouds closest to the candidate point cloud among the multiple first point clouds are determined as the K reference point clouds.

3. The method according to claim 1, characterized in that The step of determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud comprises: For any candidate point cloud, calculate the average depth corresponding to the K reference point clouds associated with the candidate point cloud; According to the average depth, calculating the offset depth corresponding to the candidate point cloud and the average deviation of the K reference point clouds relative to the average depth; When the offset depth is greater than the target value, determining the candidate point cloud as an erroneous point cloud; When the offset depth is not greater than the target value, determining that the candidate point cloud is not an erroneous point cloud; The target value is the product of a preset threshold and the average deviation.

4. The method according to claim 1, characterized in that: The method is applied to an electronic device, the electronic device comprising a camera, a first inertial element and a second inertial element; The step of performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image includes: For each frame of image, applying a preset SLAM algorithm based on the image, preset pre-integration parameters, the preset origin of the first inertial element coordinate system, the preset origin of the second inertial element coordinate system, the preset origin of the camera coordinate system, and linear transformation parameters, to obtain point cloud data corresponding to each frame of image; The linear transformation parameters are used to characterize the linear transformation relationship between the first inertial element coordinate system and the camera coordinate system, and the linear transformation relationship between the second inertial element coordinate system and the camera coordinate system.

5. The method according to claim 4, characterized in that The linear transformation parameter is determined by the following formula: + in, is the coordinate of point P in the first inertial element coordinate system or the coordinate of point P in the second inertial element coordinate system, is the coordinate of point P in the camera coordinate system, is the external parameter of the camera relative to the first inertial element, is the external parameter of the camera relative to the second inertial element, and the first linear transformation parameters include and .

6. A point cloud-based object volume measurement device, characterized in that: include: An acquisition module, used to acquire multiple frames of images obtained by photographing a target object; A processing module, used for performing point cloud data processing on each frame of image to obtain point cloud data corresponding to each frame of image; A filtering module, used for filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud in the point cloud data, so as to obtain target point cloud data; A determination module, used to determine the volume corresponding to the target object based on the target point cloud data corresponding to each frame of image; The filtering module is specifically used for: Performing image segmentation on the point cloud data corresponding to each frame of the image to obtain point cloud data within the mask area; Calculate the distance between each point cloud in the mask area and the optical center of the camera to obtain the depth corresponding to each point cloud; Filtering out erroneous point clouds in the point cloud data according to the depth corresponding to each point cloud; The filtering module is also specifically used for: Sorting each of the point clouds in descending order of corresponding depth; Determine the first N point clouds in the sorting as candidate point clouds, and obtain N candidate point clouds, where N is a positive integer greater than 1, and the candidate point clouds are located in a two-dimensional projection space; For any candidate point cloud, determine K reference point clouds associated with the candidate point cloud, where K is a positive integer greater than 1; Determining whether the candidate point cloud is an erroneous point cloud according to the depth corresponding to each reference point cloud; Filter out erroneous point clouds from the N candidate point clouds.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the object volume measurement method based on point cloud as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the point cloud-based object volume measurement method according to any one of claims 1 to 5 are implemented.

Images