Size acquisition methods, devices, robots, and readable storage media

Abstract

This invention proposes a size acquisition method, apparatus, robot, and readable storage medium. The size acquisition method includes: acquiring first point cloud information of a target object in a first coordinate system, and second point cloud information of a target plane in the first coordinate system, wherein the first coordinate system is a camera coordinate system and the target object is located on the target plane; constructing a second coordinate system based on the second point cloud information, wherein the second coordinate system is the coordinate system corresponding to the target plane; and determining the size information of the target object based on the first point cloud information and the second coordinate system.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection technology, and more specifically, relates to a size acquisition method, device, robot, and readable storage medium. Background Technology

[0002] Detecting the length, width, and height of an object using visual inspection methods involves manually selecting points in the point cloud and measuring the object's size. These methods are cumbersome, cannot automatically obtain the object's length, width, and height, and require a lot of manual operation.

[0003] Depth cameras can acquire large amounts of 3D data in a single batch. To reduce manual steps, the method of using depth cameras to measure the length, width, and height of objects has gradually replaced traditional measurement methods, offering advantages such as convenience and labor-saving, and has been widely adopted.

[0004] In related technologies, images need to be acquired from multiple specific angles using a depth camera, and background images without target objects also need to be acquired, which makes the size detection process complex and inefficient. Summary of the Invention

[0005] The present invention aims to solve one of the technical problems existing in the prior art or related technologies.

[0006] Therefore, a first aspect of the present invention proposes a method for obtaining dimensions.

[0007] A second aspect of the present invention provides a size acquisition device.

[0008] A third aspect of the present invention provides a size acquisition device.

[0009] A fourth aspect of the present invention provides a readable storage medium.

[0010] The fifth aspect of the present invention provides a robot.

[0011] In view of this, according to a first aspect of the present invention, a size acquisition method is proposed, comprising: acquiring first point cloud information of a target object in a first coordinate system, and second point cloud information of a target plane in the first coordinate system, wherein the first coordinate system is a camera coordinate system and the target object is located on the target plane; constructing a second coordinate system based on the second point cloud information, wherein the second coordinate system is a coordinate system corresponding to the target plane; and determining the size information of the target object based on the first point cloud information and the second coordinate system.

[0012] The dimension acquisition method provided by this invention can be applied to robots, enabling the robot to acquire first point cloud information of the target object and second point cloud information of the target plane during operation.

[0013] In this technical solution, the first point cloud information is the set of points of the target object in the camera coordinate system, and the second point cloud information is the set of points of the target plane in the camera coordinate system. A planar coordinate system corresponding to the target plane can be constructed using the second point cloud information. In the planar coordinate system, the target plane is the reference plane. Specifically, after determining the second point cloud information, a second coordinate system is constructed based on the second point cloud information, with the target plane as the reference plane. Since both the first and second point cloud information are sets of points in the camera coordinate system, the first point cloud information located in the first coordinate system can be transformed to the second coordinate system, thereby obtaining the corresponding coordinates of the first point cloud information in the second coordinate system, i.e., the coordinates of the target object in the second coordinate system. Since the target plane is the reference plane in the second coordinate system, the size information of the target object can be determined based on the coordinates of the target object in the second coordinate system.

[0014] This invention acquires first point cloud information of a target object and second point cloud information of a target plane. A second coordinate system is constructed with the target plane as the reference plane, and the first point cloud information is transformed to the second coordinate system. The size information of the target object is then determined within the second coordinate system. Compared to existing technologies, this method eliminates the need to acquire images from multiple angles or at specific angles, simplifying the process of obtaining object dimensions using a robot.

[0015] In addition, the dimension acquisition method in the above-described technical solution provided by the present invention may also have the following additional technical features:

[0016] In the above technical solution, obtaining the first point cloud information of the target object in the first coordinate system and the second point cloud information of the target plane in the first coordinate system includes: obtaining a first image and a second image of the target scene, wherein the first image is a color image and the second image is a depth image, and the target scene includes the target object and the target plane.

[0017] Using the first image, a first mask and a second mask are determined. The first mask matches the target object, and the second mask matches the target plane.

[0018] Based on the first mask image, the second mask image, and the second image, determine the first point cloud information and the second point cloud information.

[0019] In the technical solution, the robot is equipped with an image acquisition device, which can acquire a first image and a second image corresponding to the target scene during the robot's operation. The first image is a color image, and the second image is a depth image.

[0020] It should be noted that the robot acquires image data using the same image acquisition device, which is mounted on the robot and moves with it. During image data acquisition, the entire top surface of the target object must be captured, and the image data must include the target plane.

[0021] During robot operation, a color image (first image) corresponding to the target scene and a depth image (second image) of the target scene are acquired via an image acquisition device. The content of the first and second images matches, and both images include the target object and the target plane. This image acquisition device can be a depth camera.

[0022] It should be noted that during the acquisition of the first and second images, the entire top surface of the target object must be captured, and both the first and second images must include the target plane. The first and second images are the same size. The second image is a single-channel depth image, which stores depth values ​​greater than or equal to 0. If there are areas with a depth value of 0 in the second image, it is determined that the corresponding depth value cannot be obtained for that area.

[0023] In this technical solution, after acquiring the first image, a first mask image corresponding to the target object and a second mask image corresponding to the target plane are determined based on the first image. The first mask image is a mask image obtained by masking the target object in the first image, and the second mask image is a mask image obtained by masking the target plane in the first image. Given the first and second mask images, the first point cloud information of the target object in the depth camera coordinate system and the second point cloud information of the target plane in the depth camera coordinate system can be determined based on the first and second mask images and the second image.

[0024] It should be noted that the first mask is a single-channel mask, and the second mask is also a single-channel mask.

[0025] In this technical solution, a first image and a second image of the target scene are acquired, and a masking process is performed on the second image to obtain a first mask image corresponding to the target object and a second mask image corresponding to the target plane. Based on the first and second mask images, and the second image including depth information, the first point cloud information of the target object and the second point cloud information of the target plane can be determined according to the corresponding depth camera parameters of the second image, without the need for manual operation by the user.

[0026] In any of the above technical solutions, determining the first mask and the second mask using the first image includes: identifying first image features and second image features in the first image, wherein the first image features match the target object and the second image features match the target plane; generating the first mask using the first image features; and generating the second mask using the second image features.

[0027] In this technical solution, a first image feature matching a target object in a first image and a second image feature matching a target plane in the first image are used. A first mask image matching the target object is generated based on the first image feature, and a second mask image matching the target object is generated based on the second image feature.

[0028] Specifically, by segmenting the first image using an instance segmentation algorithm, a first mask and a second mask can be obtained. Examples of instance segmentation algorithms include Mask R-CNN and the Yolact algorithm. The first and second masks are the same size, and both are single-channel images.

[0029] This invention acquires a first image of a color image and a second image of a depth image using a depth camera, and uses an instance segmentation algorithm to obtain a first mask and a second mask corresponding to the target object and the target plane. This enables automatic identification of the first mask and the second mask in the first image and the second image, eliminating the need for the user to manually select the mask in the image and simplifying the required operation.

[0030] In any of the above technical solutions, determining the first point cloud information and the second point cloud information based on the first mask image, the second mask image, and the second image includes: filtering the first coordinate set in the first mask image and the second coordinate set in the second mask image according to preset rules based on the depth values ​​in the second image; determining the first point cloud information based on the first coordinate set and depth camera parameters, wherein the depth camera parameters are the parameters of the depth camera that acquired the second image; and determining the second point cloud information based on the second coordinate set and the depth camera parameters.

[0031] In this embodiment of the application, by filtering the first coordinate set in the first mask image and the second coordinate set in the second mask image, and then calculating the first coordinate set, the second coordinate set, and the depth camera parameters, the corresponding first point cloud information and second point cloud information are obtained.

[0032] Specifically, the first and second mask images are filtered according to the same filtering rules, and the resulting first coordinate set is the coordinate set of the target object in the first mask image, and the second coordinate set is the coordinate set of the target plane in the second mask image.

[0033] In this technical solution, the first coordinate information is the coordinates of the target object in the first mask image, and the second coordinate information is the coordinates of the target plane in the second mask image. At this time, the first coordinate information and the second coordinate information need to be configured into the depth camera coordinate system to obtain the corresponding first point cloud information and second point cloud information.

[0034] It should be noted that the depth camera parameters refer to the depth camera used to acquire image data. When the image data includes a first image and a second image, the depth camera can simultaneously acquire both images. The depth camera parameters are intrinsic parameters of the depth camera, including but not limited to the scale factors of the depth camera in the u-axis and v-axis directions, and the coordinates of the principal point of the depth camera in the image coordinate system.

[0035] In this technical solution, the coordinates of the target object and the target plane in the depth camera calibration system can be calculated using the above formula, and the set of coordinates is used as the corresponding point cloud information.

[0036] It should be noted that the first coordinate information and the second coordinate information refer to the coordinate points in the first and second mask images, i.e., two-dimensional coordinates. The first point cloud information and the second point cloud information refer to the point clouds in the depth camera coordinate system, i.e., three-dimensional coordinates.

[0037] This invention can calculate the first point cloud information and the second point cloud information of the target object and the target plane in the depth camera coordinate system by using the first coordinate information, the second coordinate information and the depth camera parameters, so that the user does not need to control the depth camera to collect image data multiple times, which further simplifies the data acquisition process.

[0038] In any of the above technical solutions, the depth camera parameters include at least one of the following: the scale factor of the depth camera, and the coordinates of the principal point of the depth camera in the image coordinate system.

[0039] In this technical solution, the parameters of the depth camera are its intrinsic parameters. The size factor of the depth camera includes the scale factor of the depth camera in the u-axis and v-axis directions, that is, the scale factor of the depth camera in the u-axis and v-axis directions in the acquired image coordinate system. The principal point of the depth camera is the coordinate of the image data acquired by the depth camera in the image coordinate system. The principal point of the depth camera is the sampling point of the depth camera.

[0040] This invention, through depth camera parameters, can determine the first point cloud information and the second point cloud information of the corresponding target object and target plane in the image coordinate system of the image data acquired by the depth camera, based on the two-dimensional coordinate points in the first and second mask images.

[0041] In any of the above technical solutions, the preset rules include a depth value greater than a preset depth value and a color value of a preset color value.

[0042] In this technical solution, the first coordinate information in the first mask is filtered by combining the color values ​​of each coordinate point in the first mask and the depth information in the second image, and the second coordinate information in the second mask is filtered by combining the color values ​​of each coordinate point in the second mask and the depth information in the second image.

[0043] In this technical solution, the first coordinate information in the first mask image and the second coordinate information in the second mask image are filtered using the same preset rules.

[0044] It should be noted that for coordinate points with a depth value of 0 in the first and second mask images, it is determined that the depth value of that coordinate position cannot be obtained. Therefore, the corresponding first and second coordinate information for coordinate points with a depth value greater than 0 are obtained.

[0045] The present invention improves the accuracy of the first coordinate information and the second coordinate information by using the coordinates of the first mask image and the second mask image with the color value of the preset color value and the depth value greater than the preset depth value as the first coordinate information of the target object and the second coordinate information of the target plane.

[0046] In any of the above technical solutions, constructing a second coordinate system based on the second point cloud information includes: determining a first plane equation based on the second point cloud information, wherein the first plane equation matches the target plane; and constructing the second coordinate system based on the first plane equation.

[0047] In this technical solution, the second point cloud information is the point cloud information of the target plane. A second coordinate system is constructed based on the second point cloud information, with the target plane as the reference plane. First, the plane equation of the target plane is obtained by fitting based on the second point cloud information. According to the fitted first plane equation, a coordinate system rotation matrix is ​​generated. The coordinate system rotation matrix is ​​used to rotate and transform the depth camera coordinate system into the plane coordinate system of the target plane.

[0048] This invention obtains the plane equation of the target plane by fitting, and uses the target plane indicated by the plane equation as the reference plane of the second coordinate system to construct the second coordinate system. The second coordinate system is a coordinate system with the target plane as the reference plane, which facilitates the determination of the size information of the target object based on the coordinates of the first point cloud information in the second coordinate system.

[0049] In any of the above technical solutions, determining the size information of the target object based on the first point cloud information and the second coordinate system includes: obtaining the coordinate system rotation matrix between the first coordinate system and the second coordinate system; determining the third coordinate information of the first point cloud information in the second coordinate system through the coordinate system rotation matrix; and determining the size information of the target object based on the third coordinate information.

[0050] In this technical solution, after constructing a second coordinate system with the target plane as the reference plane, a coordinate system rotation matrix between the first and second coordinate systems is determined. This rotation matrix maps the first point cloud information in the first coordinate system to the second coordinate system, thereby obtaining the corresponding third coordinate information. The third coordinate information is the set of coordinate points of the target object in the second coordinate system. By calculating the third coordinate information, the size information of the target object can be determined. Since the size information of the target object is calculated in the second coordinate system with the target plane as the reference plane, the accuracy of determining the size information of the target object is improved.

[0051] Specifically, the height of the target object can be determined by calculating the distance between the highest point in the third coordinate information and the target plane. The length and width of the target object can be determined by calculating the coordinates of its outer contour in the third coordinate information.

[0052] In the technical solution defined by this invention, third coordinate information is obtained by transforming and projecting the first point cloud information of the target object onto a second coordinate system of the target plane. The size information of the target object is then determined in the second coordinate system based on the third coordinate information, thereby improving the accuracy of determining the size information of the target object.

[0053] In any of the above technical solutions, the size information includes the height value of the target object; determining the size information of the target object based on the third coordinate information includes: obtaining the distance value between each coordinate point in the third coordinate information and the target plane; and filtering the maximum distance value among multiple distance values ​​as the height value of the target object.

[0054] In this technical solution, the target object is located on the target plane, so the distance between the highest coordinate point of the target object and the target plane is the height value of the target object. The height value of the target object is determined by calculating the distance from each coordinate point in the third coordinate information of the target object to the target plane and then taking the maximum value among these multiple distances.

[0055] This invention improves the accuracy of determining the height of the target object by calculating the distance from each coordinate point of the target object to the target plane and determining the maximum distance value among multiple distance values ​​as the height of the target object.

[0056] In any of the above technical solutions, the size information includes the width and length values ​​of the target object; determining the size information of the target object based on the third coordinate information includes: determining the fourth coordinate information corresponding to the target object based on the third coordinate information, wherein the fourth coordinate information is the coordinate information of the bounding rectangle of the target object; and determining the width and length values ​​of the target object based on the fourth coordinate information.

[0057] In this technical solution, after determining the third coordinate information of the first point cloud information in the second coordinate system, the fourth coordinate information of the bounding rectangle of the target object is obtained. Based on the fourth coordinate information of the bounding rectangle of the target object, the width and length values ​​of the bounding rectangle can be obtained. The width and length values ​​are then determined as the width and length values ​​of the target object.

[0058] The fourth coordinate information is the coordinate information of the minimum bounding rectangle of the target object in the second coordinate system.

[0059] This invention determines the minimum bounding rectangle of a target object and its fourth coordinate information in a second coordinate system. Based on this fourth coordinate information, the width and length values ​​of the target object can be accurately calculated. By using the width and length values ​​of the minimum bounding rectangle of the target object as its width and length values, this invention further improves the accuracy of determining the width and length values ​​of the target object.

[0060] According to a second aspect of the present invention, a size acquisition device is provided, the size acquisition device comprising:

[0061] The acquisition module is used to acquire the first point cloud information of the target object in the first coordinate system and the second point cloud information of the target plane in the first coordinate system, wherein the first coordinate system is the camera coordinate system and the target object is located on the target plane;

[0062] The construction module is used to construct a second coordinate system based on the second point cloud information. The second coordinate system is the coordinate system corresponding to the target plane.

[0063] The determination module is also used to determine the size information of the target object based on the first point cloud information and the second coordinate system.

[0064] This invention acquires first point cloud information of a target object and second point cloud information of a target plane. A second coordinate system is constructed with the target plane as the reference plane, and the first point cloud information is transformed to the second coordinate system. The size information of the target object is then determined within the second coordinate system. Compared to existing technologies, this method eliminates the need to acquire images from multiple angles or at specific angles, simplifying the process of obtaining object dimensions using a robot.

[0065] According to a third aspect of the present invention, a size acquisition device is provided, comprising: a memory storing a program or instructions; and a processor executing the program or instructions stored in the memory to implement the steps of the size acquisition method as described in any of the technical solutions of the first aspect, thus possessing all the beneficial technical effects of the size acquisition method in any of the technical solutions of the first aspect, which will not be elaborated further here.

[0066] According to a fourth aspect of the present invention, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the dimension acquisition method as described in any of the technical solutions of the first aspect above. Therefore, it possesses all the beneficial technical effects of the dimension acquisition method in any of the technical solutions of the first aspect above, and will not be elaborated further here.

[0067] According to a fifth aspect of the present invention, a robot is provided, comprising: a size acquisition device as defined in the second or third aspect above, and / or a readable storage medium as defined in the fourth aspect above, thereby having all the beneficial technical effects of the size acquisition device as defined in the second or third aspect above, and / or the readable storage medium as defined in the fourth aspect above, which will not be elaborated further here.

[0068] In the above technical solution, the robot also includes: a depth camera, used to acquire image data, including color images and / or depth images.

[0069] In this technical solution, a depth camera acquires image data. The depth camera is mounted on the robot and moves with the robot. The image data can include depth images and color images.

[0070] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0071] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0072] Figure 1 One of the schematic flowcharts of the dimension acquisition method provided in some embodiments of the present invention is shown;

[0073] Figure 2 A second schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown;

[0074] Figure 3 A schematic diagram of a mask pattern provided in some embodiments of the present invention is shown;

[0075] Figure 4 A schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown as the third one;

[0076] Figure 5 A fourth schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown;

[0077] Figure 6 Fifth of the schematic flowcharts of the dimension acquisition method provided in some embodiments of the present invention is shown;

[0078] Figure 7 A schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown as Flowchart 6;

[0079] Figure 8 A schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown as Flowchart 7;

[0080] Figure 9 Eighth schematic flowchart of a dimension acquisition method provided in some embodiments of the present invention is shown;

[0081] Figure 10 Structural block diagrams of the dimension acquisition apparatus provided in some embodiments of the present invention are shown;

[0082] Figure 11 The diagram shows a structural block diagram of a dimension acquisition device provided in some embodiments of the present invention;

[0083] Figure 12 A structural block diagram of a robot provided by some embodiments of the present invention is shown. Detailed Implementation

[0084] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0085] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0086] The following reference Figures 1 to 12 Describes a method, apparatus, readable storage medium, and robot for obtaining dimensions according to some embodiments of the present invention.

[0087] According to one embodiment of the present invention, such as Figure 1 As shown, a method for obtaining dimensions is proposed, including:

[0088] Step 102: Obtain the first point cloud information of the target object in the first coordinate system, and the second point cloud information of the target plane in the first coordinate system;

[0089] The first coordinate system is the camera coordinate system, and the target object is located on the target plane.

[0090] Step 104: Based on the second point cloud information, construct a second coordinate system, which is the coordinate system corresponding to the target plane;

[0091] Step 106: Determine the size information of the target object based on the first point cloud information and the second coordinate system.

[0092] The dimension acquisition method provided in this embodiment can be applied to robots. During operation, the robot can acquire the first point cloud information of the target object and the second point cloud information of the target plane.

[0093] For example, the target plane can be the ground, and the target object can be furniture.

[0094] For example, the image data may include a depth image and a color image. By performing image feature recognition on the color image and determining first point cloud information and second point cloud information in the depth image based on the recognized image features.

[0095] For example, the image data can be image data of the target scene obtained by a 3D scanner, first point cloud information of the target object manually selected by the user, and second point cloud information of the target plane.

[0096] In this embodiment, the first point cloud information is the set of points of the target object in the camera coordinate system, and the second point cloud information is the set of points of the target plane in the camera coordinate system. The second point cloud information can be used to construct the planar coordinate system corresponding to the target plane. In the planar coordinate system, the target plane is the reference plane in the coordinate system.

[0097] Specifically, after determining the second point cloud information, a second coordinate system is constructed based on this information, with the target plane as the reference plane. Since both the first and second point cloud information are sets of points in the camera coordinate system, the first point cloud information located in the first coordinate system can be transformed into the second coordinate system, thereby obtaining the corresponding coordinates of the first point cloud information in the second coordinate system, which are the coordinates of the target object in the second coordinate system. Because the target plane is the reference plane in the second coordinate system, the size information of the target object can be determined based on its coordinates in the second coordinate system.

[0098] For example: the second coordinate system is a three-dimensional coordinate system. In the second coordinate system, the target plane is the xoy plane, that is, the Z-axis of the target plane is 0 in the second coordinate system.

[0099] For example, the target plane is the ground. Using principal component analysis on the second point cloud information of the ground, the plane equation of the ground is fitted as: ax + by + cz + d = 0. The normal vector of the plane is (a, b, c), which is a unit vector and perpendicular to the plane. Taking the origin of the camera acquiring the image data as the origin of the ground coordinate system, and the normal vector of the ground plane as the Z-axis of the ground coordinate system, an arbitrary vector not parallel to the normal vector of the ground plane is chosen. The cross product of this vector and the normal vector of the ground plane is calculated. The unit vector of the cross product (e, f, g) is taken as the X-axis of the ground coordinate system. The cross product of vectors (a, b, c) and (e, f, g) is calculated. The unit vector of the cross product (i, h, j) is taken as the Y-axis of the ground coordinate system, thus constructing the ground coordinate system.

[0100] This embodiment acquires first point cloud information of the target object and second point cloud information of the target plane. A second coordinate system is constructed with the target plane as the reference plane, and the first point cloud information is transformed to the second coordinate system. The size information of the target object is then determined in the second coordinate system. Compared with existing technologies, this method eliminates the need to acquire images from multiple angles or at specific angles, simplifying the steps for obtaining object dimensions using a robot.

[0101] like Figure 2 As shown, in the above embodiment, obtaining the first point cloud information of the target object in the first coordinate system and the second point cloud information of the target plane in the first coordinate system includes:

[0102] Step 202: Obtain the first and second images of the target scene;

[0103] The first image is a color image, the second image is a depth image, and the target scene includes the target object and the target plane.

[0104] Step 204: Using the first image, determine the first mask and the second mask. The first mask matches the target object, and the second mask matches the target plane.

[0105] Step 206: Determine the first point cloud information and the second point cloud information based on the first mask image, the second mask image, and the second image.

[0106] In this embodiment, the robot is equipped with an image acquisition device, which can acquire a first image and a second image corresponding to the target scene during the robot's operation. The first image is a color image, and the second image is a depth image.

[0107] It should be noted that the robot acquires image data using the same image acquisition device, which is mounted on the robot and moves with it. During image data acquisition, the entire top surface of the target object must be captured, and the image data must include the target plane.

[0108] During robot operation, a color image (first image) of the target scene and a depth image (second image) of the target scene are acquired via an image acquisition device. The content of the first and second images matches, and both the first and second images include the target object and the target plane.

[0109] For example, the robot acquires a first image and a second image using the same image acquisition device. Since both the first image and the second image come from the same image acquisition device, which is mounted on the robot and moves with it, the content of the images acquired by the image acquisition device is exactly the same. The image acquisition device can be a depth camera.

[0110] It should be noted that during the acquisition of the first and second images, the entire top surface of the target object must be captured, and both the first and second images must include the target plane. The first and second images are the same size. The second image is a single-channel depth image, which stores depth values ​​greater than or equal to 0. If there are areas with a depth value of 0 in the second image, it is determined that the corresponding depth value cannot be obtained for that area.

[0111] In this embodiment, after acquiring the first image, a first mask image corresponding to the target object and a second mask image corresponding to the target plane are determined based on the first image. The first mask image is a mask image obtained by masking the target object in the first image, and the second mask image is a mask image obtained by masking the target plane in the first image. With the first mask image and the second mask image obtained, the first point cloud information of the target object in the depth camera coordinate system and the second point cloud information of the target plane in the depth camera coordinate system can be determined based on the first mask image, the second mask image, and the second image.

[0112] It should be noted that the first mask is a single-channel mask, and the second mask is also a single-channel mask.

[0113] For example, the color value at each position in the first mask and the second mask is either 255 or 0. Taking the first mask as an example, the area with a color value of 255 in the first mask is the area where the target object is located, and the other areas are areas where the target object is not located. Taking the second mask as an example, the area with a color value of 255 in the second mask is the area where the target plane is located, and the other areas are areas where the target plane is not located.

[0114] like Figure 3 As shown, the target object in the color image is a bed. Instance segmentation is performed on the color image to identify various image features, thereby obtaining a first mask and a second mask. The first mask is a bed mask, and the second mask obtained from the color image is a ground mask.

[0115] In this embodiment, by acquiring a first image and a second image of the target scene, and by performing masking processing on the second image, a first mask image corresponding to the target object and a second mask image corresponding to the target plane are obtained. Based on the first mask image, the second mask image, and the second image including depth information, the first point cloud information of the target object and the second point cloud information of the target plane can be determined according to the corresponding depth camera parameters of the second image, without the need for manual operation by the user.

[0116] like Figure 4 As shown, in any of the above embodiments, determining the first mask image and the second mask image using the first image includes:

[0117] Step 402: Identify the first image feature and the second image feature in the first image. The first image feature matches the target object, and the second image feature matches the target plane.

[0118] Step 404: Generate a first mask image using the first image features, and generate a second mask image using the second image features.

[0119] In this embodiment, a first image feature matching a target object in a first image and a second image feature matching a target plane in the first image are used. A first mask image matching the target object is generated based on the first image feature, and a second mask image matching the target object is generated based on the second image feature.

[0120] Specifically, by segmenting the first image using an instance segmentation algorithm, a first mask and a second mask can be obtained. Examples of instance segmentation algorithms include Mask R-CNN and the Yolact algorithm. The first and second masks are the same size, and both are single-channel images.

[0121] This embodiment acquires a first image of a color image and a second image of a depth image using a depth camera, and uses an instance segmentation algorithm to obtain a first mask and a second mask corresponding to the target object and the target plane. This achieves automatic recognition of the first mask and the second mask in the first image and the second image, eliminating the need for the user to manually select the mask in the image and simplifying the required operation.

[0122] like Figure 5As shown, in any of the above embodiments, determining the first point cloud information and the second point cloud information based on the first mask image, the second mask image, and the second image includes:

[0123] Step 502: Based on the depth values ​​in the second image, filter the first coordinate set in the first mask image and the second coordinate set in the second mask image according to preset rules;

[0124] Step 504: Determine the first point cloud information based on the first coordinate set and the depth camera parameters, where the depth camera parameters are the parameters of the depth camera that acquired the second image, and determine the second point cloud information based on the second coordinate set and the depth camera parameters.

[0125] In this embodiment of the application, by filtering the first coordinate set in the first mask image and the second coordinate set in the second mask image, and then calculating the first coordinate set, the second coordinate set, and the depth camera parameters, the corresponding first point cloud information and second point cloud information are obtained.

[0126] Specifically, the first and second mask images are filtered according to the same filtering rules, and the resulting first coordinate set is the coordinate set of the target object in the first mask image, and the second coordinate set is the coordinate set of the target plane in the second mask image.

[0127] Taking the filtering of the first mask image as an example, the first mask image contains only two different color values, 255 and 0. The area with a color value of 255 is determined as the area where the target object is located, and the area with a color value of 0 is determined as the background area. Based on the color value, the first coordinate information in the first mask image is filtered, and the coordinate information with a color value of 255 in the first mask image is determined as the target object.

[0128] Taking the filtering of the second mask image as an example, the second mask image contains only two different color values, 255 and 0. The area with a color value of 255 is determined as the area where the target object is located, and the area with a color value of 0 is determined as the background area. Based on the color values, the second coordinate information in the second mask image is filtered, and the coordinate information with a color value of 255 in the second mask image is determined as the target object.

[0129] In this embodiment, the first coordinate information is the coordinates of the target object in the first mask image, and the second coordinate information is the coordinates of the target plane in the second mask image. At this time, the first coordinate information and the second coordinate information need to be configured into the depth camera coordinate system to obtain the corresponding first point cloud information and second point cloud information.

[0130] It should be noted that the depth camera parameters refer to the depth camera used to acquire image data. When the image data includes a first image and a second image, the depth camera can simultaneously acquire both images. The depth camera parameters are intrinsic parameters of the depth camera, including but not limited to the scale factors of the depth camera in the u-axis and v-axis directions, and the coordinates of the principal point of the depth camera in the image coordinate system.

[0131] For example, point cloud information can be calculated based on coordinate information using the following formula, wherein the coordinate information includes first coordinate information and second coordinate information, the point cloud information includes first point cloud information and second point cloud information, the first coordinate information corresponds to the first point cloud information, and the second coordinate information corresponds to the second point cloud information.

[0132] X=d×(uC x ) / f x ;

[0133] Y = d × (vC) y ) / f y ;

[0134] Z = d;

[0135] Where X, Y, and Z represent point cloud information, d represents depth value, u and v represent coordinate information, and C represents point cloud information. x C y f represents the coordinates of the principal point of the depth camera in the image coordinate system. x f y is the scale factor of the depth camera in the u-axis and v-axis directions.

[0136] In this embodiment, the coordinates of the target object and the target plane in the depth camera calibration system can be calculated using the above formula, and the set of coordinates is used as the corresponding point cloud information.

[0137] It should be noted that the first coordinate information and the second coordinate information refer to the coordinate points in the first and second mask images, i.e., two-dimensional coordinates. The first point cloud information and the second point cloud information refer to the point clouds in the depth camera coordinate system, i.e., three-dimensional coordinates. In the above formula, d is the depth value of the coordinate points in the second image corresponding to the first coordinate information and the second coordinate information.

[0138] This invention can calculate the first point cloud information and the second point cloud information of the target object and the target plane in the depth camera coordinate system by using the first coordinate information, the second coordinate information and the depth camera parameters, so that the user does not need to control the depth camera to collect image data multiple times, which further simplifies the data acquisition process.

[0139] In any of the above embodiments, the depth camera parameters include at least one of the following: the scale factor of the depth camera, and the coordinates of the principal point of the depth camera in the image coordinate system.

[0140] In this embodiment, the parameters of the depth camera are its intrinsic parameters. The size factor of the depth camera includes the scale factor of the depth camera in the u-axis and v-axis directions, that is, the scale factor of the depth camera in the u-axis and v-axis directions in the image coordinate system of the acquired image data. The principal point of the depth camera is defined by its coordinates in the image coordinate system of the acquired image data. The principal point of the depth camera is the sampling point of the depth camera.

[0141] This embodiment uses depth camera parameters to determine the first point cloud information and the second point cloud information of the target object and target plane in the image coordinate system of the image data acquired by the depth camera, based on the two-dimensional coordinate points in the first and second mask images.

[0142] In any of the above embodiments, the preset rules include a depth value greater than a preset depth value and a color value of a preset color value.

[0143] In this embodiment, the first coordinate information in the first mask is filtered by combining the color values ​​of each coordinate point in the first mask and the depth information in the second image, and the second coordinate information in the second mask is filtered by combining the color values ​​of each coordinate point in the second mask and the depth information in the second image.

[0144] In this embodiment, the first coordinate information in the first mask image and the second coordinate information in the second mask image are filtered using the same preset rules.

[0145] For example, the preset depth value is 0 and the preset color value is 255.

[0146] Taking the filtering of the first mask image as an example, the first mask image contains only two different color values, 255 and 0. The coordinate points in the first mask image with a color value of 255 and a depth value greater than 0 are selected as the first coordinate information.

[0147] Taking the filtering of the second mask as an example, the second mask contains only two different color values, 255 and 0. The coordinate points in the second mask with a color value of 255 and a depth value greater than 0 are selected as the second coordinate information.

[0148] It should be noted that for coordinate points with a depth value of 0 in the first and second mask images, it is determined that the depth value of that coordinate position cannot be obtained. Therefore, the corresponding first and second coordinate information for coordinate points with a depth value greater than 0 are obtained.

[0149] This embodiment improves the accuracy of the first and second coordinate information by using the coordinates of the first and second mask images with preset color values ​​and depth values ​​greater than preset depth values ​​as the first coordinate information of the target object and the second coordinate information of the target plane.

[0150] like Figure 6 As shown, in any of the above embodiments, constructing a second coordinate system based on the second point cloud information includes:

[0151] Step 602: Based on the second point cloud information, determine the equation of the first plane, and match the equation of the first plane with the target plane;

[0152] Step 604: Construct a second coordinate system based on the equation of the first plane.

[0153] In this embodiment, the second point cloud information is the point cloud information of the target plane. A second coordinate system is constructed based on the second point cloud information, with the target plane as the reference plane. First, the plane equation of the target plane is obtained by fitting based on the second point cloud information. According to the fitted first plane equation, a coordinate system rotation matrix is ​​generated. The coordinate system rotation matrix is ​​used to rotate and transform the depth camera coordinate system into the plane coordinate system of the target plane.

[0154] Specifically, the equation of the target plane obtained by fitting is ax + by + cz + d = 0, where the plane normal vector is (a, b, c). Taking the first coordinate system as the origin of the second coordinate system and the target plane normal vector as the Z-axis of the second coordinate system, we take any vector that is not parallel to the target plane normal vector and calculate the cross product of this vector and the ground plane normal vector. The result is a vector. The unit vector of the cross product (e, f, g) is the X-axis of the second coordinate system. We calculate the cross product of vectors (a, b, c) and (e, f, g), and the unit vector of the cross product (i, h, j) is the Y-axis of the ground coordinate system, thus constructing the second coordinate system.

[0155] In this embodiment, the plane equation of the target plane is obtained by fitting, and the target plane indicated by the plane equation is used as the reference plane of the second coordinate system to construct the second coordinate system. The second coordinate system is a coordinate system with the target plane as the reference plane, which facilitates the determination of the size information of the target object based on the coordinates of the first point cloud information in the second coordinate system.

[0156] like Figure 7 As shown, in any of the above embodiments, determining the size information of the target object based on the first point cloud information and the second coordinate system includes:

[0157] Step 702: Obtain the coordinate system rotation matrix between the first coordinate system and the second coordinate system;

[0158] Step 704: Determine the third coordinate information of the first point cloud information in the second coordinate system using the coordinate system rotation matrix;

[0159] Step 706: Determine the size information of the target object based on the third coordinate information.

[0160] In this embodiment, after constructing a second coordinate system with the target plane as the reference plane, a coordinate system rotation matrix between the first and second coordinate systems is determined. This rotation matrix maps the first point cloud information in the first coordinate system to the second coordinate system, thereby obtaining the corresponding third coordinate information. The third coordinate information is the set of coordinate points of the target object in the second coordinate system. By calculating the third coordinate information, the size information of the target object can be determined. Since the size information of the target object is calculated in the second coordinate system with the target plane as the reference plane, the accuracy of determining the size information of the target object is improved.

[0161] For example, the target object is indoor furniture products, and the target plane is the indoor floor.

[0162] For example, each point in the first point cloud information is mapped to the second coordinate system using the following formula to obtain the third coordinate information:

[0163]

[0164] Where (e, f, g) is the X-axis of the ground coordinate system, (i, h, j) is the Y-axis of the second coordinate system, (a, b, c) is the normal vector of the target plane, and (x, y, z) are the coordinates of each point in the first point cloud information, (x w y w , z w ) represents the coordinates of each point in the third coordinate information.

[0165] Specifically, the height of the target object can be determined by calculating the distance between the highest point in the third coordinate information and the target plane. The length and width of the target object can be determined by calculating the coordinates of its outer contour in the third coordinate information.

[0166] In this specific embodiment, the first point cloud information of the target object is transformed and projected onto a second coordinate system of the target plane to obtain third coordinate information. The size information of the target object is then determined in the second coordinate system based on the third coordinate information, thus improving the accuracy of determining the target object's size information.

[0167] like Figure 8 As shown, in any of the above embodiments, the size information includes the height value of the target object; determining the size information of the target object based on the third coordinate information includes:

[0168] Step 802: Obtain the distance value between each coordinate point in the third coordinate information and the target plane;

[0169] Step 804: Filter the maximum distance value among multiple distance values ​​and use it as the height value of the target object.

[0170] In this embodiment, the target object is located on the target plane, so the distance between the highest coordinate point in the target object and the target plane is the height value of the target object. The height value of the target object is determined by calculating the distance from each coordinate point in the third coordinate information of the target object to the target plane and then selecting the maximum value among these distances.

[0171] For example, in the calculated third coordinate information, each coordinate point is (x w y w , z w In the case of ), since the target plane is the plane with the Z-axis at 0 in the second coordinate system, i.e., the X0Y plane, the z-axis of each coordinate point is set to 0. w As the distance value, filter for the largest z-value among each coordinate point. w As the height value of the target object.

[0172] For example, the distance between each point and the target plane is calculated using the first point cloud information. Specifically, the distance between each coordinate point in the third coordinate information and the target plane is calculated using the following formula:

[0173]

[0174] Where s is the distance value, a, b, c, d are the parameters of the target plane, and x, y, z are the coordinates in the first point cloud information. The parameters a, b, c, d of the target plane are obtained by solving the plane equation based on the fitting of the plane point cloud.

[0175] This embodiment improves the accuracy of determining the height of the target object by calculating the distance from each coordinate point of the target object to the target plane and determining the maximum distance value among multiple distance values ​​as the height of the target object.

[0176] like Figure 9 As shown, in any of the above embodiments, the size information includes the width and length values ​​of the target object; determining the size information of the target object based on the third coordinate information includes:

[0177] Step 902: Based on the third coordinate information, determine the fourth coordinate information corresponding to the target object. The fourth coordinate information is the coordinate information of the bounding rectangle of the target object.

[0178] Step 904: Determine the width and length of the target object based on the fourth coordinate information.

[0179] In this embodiment, after determining the third coordinate information of the first point cloud information in the second coordinate system, the fourth coordinate information of the bounding rectangle of the target object is obtained. Based on the fourth coordinate information of the bounding rectangle of the target object, the width and length values ​​of the bounding rectangle can be obtained, and the width and length values ​​are determined as the width and length values ​​of the target object.

[0180] The fourth coordinate information is the coordinate information of the minimum bounding rectangle of the target object in the second coordinate system.

[0181] This embodiment determines the minimum bounding rectangle of the target object and its fourth coordinate information in the second coordinate system. Based on this fourth coordinate information, the width and length values ​​of the target object can be accurately calculated. By using the width and length values ​​of the minimum bounding rectangle of the target object as its width and length values, this invention further improves the accuracy of the determined width and length values ​​of the target object.

[0182] In one embodiment according to this application, such as Figure 10 As shown, a size acquisition device 1000 is proposed, which includes:

[0183] The acquisition module 1002 is used to acquire the first point cloud information of the target object in the first coordinate system and the second point cloud information of the target plane in the first coordinate system. The first coordinate system is the camera coordinate system, and the target object is located on the target plane.

[0184] Module 1004 is used to construct a second coordinate system based on the second point cloud information. The second coordinate system is the coordinate system corresponding to the target plane.

[0185] The determination module 1006 is used to determine the size information of the target object based on the first point cloud information and the second coordinate system.

[0186] This embodiment acquires first point cloud information of the target object and second point cloud information of the target plane. A second coordinate system is constructed with the target plane as the reference plane, and the first point cloud information is transformed to the second coordinate system. The size information of the target object is then determined in the second coordinate system. Compared with existing technologies, this method eliminates the need to acquire images from multiple angles or at specific angles, simplifying the steps for obtaining object dimensions using a robot.

[0187] In the above embodiment, the acquisition module 1002 is further configured to acquire a first image and a second image of the target scene, wherein the first image is a color image and the second image is a depth image, and the target scene includes a target object and a target plane;

[0188] The determining module 1006 is further configured to determine a first mask and a second mask based on the first image, wherein the first mask matches the target object and the second mask matches the target plane;

[0189] The determination module 1006 is used to determine the first point cloud information and the second point cloud information based on the first mask image, the second mask image, and the second image.

[0190] In this embodiment, by acquiring a first image and a second image of the target scene, and by performing masking processing on the second image, a first mask image corresponding to the target object and a second mask image corresponding to the target plane are obtained. Based on the first mask image, the second mask image, and the second image including depth information, the first point cloud information of the target object and the second point cloud information of the target plane can be determined according to the corresponding depth camera parameters of the second image, without the need for manual operation by the user.

[0191] In any of the above embodiments, the size acquisition device 1000 includes:

[0192] The recognition module is used to recognize a first image feature and a second image feature in a first image, wherein the first image feature is matched with a target object and the second image feature is matched with a target plane.

[0193] The generation module is used to generate a first mask image using first image features and to generate a second mask image using second image features.

[0194] This embodiment acquires a first image of a color image and a second image of a depth image using a depth camera, and uses an instance segmentation algorithm to obtain a first mask and a second mask corresponding to the target object and the target plane. This achieves automatic recognition of the first mask and the second mask in the first image and the second image, eliminating the need for the user to manually select the mask in the image and simplifying the required operation.

[0195] In any of the above embodiments, the size acquisition device 1000 includes:

[0196] The filtering module is used to filter the first coordinate set in the first mask image and the second coordinate set in the second mask image according to the depth value in the second image and a preset rule.

[0197] The determination module 1006 is used to determine the first point cloud information based on the first coordinate set and the depth camera parameters, wherein the depth camera parameters are the parameters of the depth camera that acquired the second image, and to determine the second point cloud information based on the second coordinate set and the depth camera parameters.

[0198] This invention can calculate the first point cloud information and the second point cloud information of the target object and the target plane in the depth camera coordinate system by using the first coordinate information, the second coordinate information and the depth camera parameters, so that the user does not need to control the depth camera to collect image data multiple times, which further simplifies the data acquisition process.

[0199] In any of the above embodiments, the depth camera parameters include at least one of the following: the scale factor of the depth camera, and the coordinates of the principal point of the depth camera in the image coordinate system of the second image.

[0200] This embodiment uses depth camera parameters to determine the first point cloud information and the second point cloud information of the target object and target plane in the image coordinate system of the image data acquired by the depth camera, based on the two-dimensional coordinate points in the first and second mask images.

[0201] In any of the above embodiments, the preset rules include a depth value greater than a preset depth value and a color value of a preset color value.

[0202] This embodiment improves the accuracy of the first and second coordinate information by using the coordinates of the first and second mask images with preset color values ​​and depth values ​​greater than preset depth values ​​as the first coordinate information of the target object and the second coordinate information of the target plane.

[0203] In any of the above embodiments, the determining module 1006 is used to determine a first plane equation based on the second point cloud information, wherein the first plane equation matches the target plane;

[0204] Module 1004 is used to construct the second coordinate system based on the equation of the first plane.

[0205] In this embodiment, the plane equation of the target plane is obtained by fitting, and the target plane indicated by the plane equation is used as the reference plane of the second coordinate system to construct the second coordinate system. The second coordinate system is a coordinate system with the target plane as the reference plane, which facilitates the determination of the size information of the target object based on the coordinates of the first point cloud information in the second coordinate system.

[0206] In any of the above embodiments, the acquisition module 1002 is used to acquire the coordinate system rotation matrix between the first coordinate system and the second coordinate system;

[0207] The determination module 1006 is used to determine the third coordinate information of the first point cloud information in the second coordinate system through the coordinate system rotation matrix;

[0208] The determination module 1006 is used to determine the size information of the target object based on the third coordinate information.

[0209] In this specific embodiment, the first point cloud information of the target object is transformed and projected onto a second coordinate system of the target plane to obtain third coordinate information. The size information of the target object is then determined in the second coordinate system based on the third coordinate information, thus improving the accuracy of determining the target object's size information.

[0210] In any of the above embodiments, the size information includes the height value of the target object.

[0211] The acquisition module 1002 is used to acquire the distance value between each coordinate point in the third coordinate information and the target plane;

[0212] The filtering module is used to filter the maximum distance value among multiple distance values, and use it as the height value of the target object.

[0213] This embodiment improves the accuracy of determining the height of the target object by calculating the distance from each coordinate point of the target object to the target plane and determining the maximum distance value among multiple distance values ​​as the height of the target object.

[0214] In any of the above embodiments, the determining module 1006 is used to determine the fourth coordinate information corresponding to the target object based on the third coordinate information, wherein the fourth coordinate information is the coordinate information of the bounding rectangle of the target object;

[0215] The determination module 1006 is used to determine the width and length values ​​of the target object based on the fourth coordinate information.

[0216] This embodiment determines the minimum bounding rectangle of the target object and its fourth coordinate information in the second coordinate system. Based on this fourth coordinate information, the width and length values ​​of the target object can be accurately calculated. By using the width and length values ​​of the minimum bounding rectangle of the target object as its width and length values, this invention further improves the accuracy of the determined width and length values ​​of the target object.

[0217] In one embodiment according to this application, such as Figure 11 As shown, a size acquisition device is proposed, including: a processor 1102 and a memory 1104, wherein the memory 1104 stores a program or instructions; the processor 1102 executes the program or instructions stored in the memory 1104 to implement the steps of the size acquisition method as described in any of the technical solutions in the first aspect, and thus has all the beneficial technical effects of the size acquisition method in any of the technical solutions in the first aspect, which will not be elaborated further here.

[0218] In one embodiment of this application, a readable storage medium is provided, on which a program or instructions are stored. When executed by a processor, the program or instructions implement the steps of the dimension acquisition method as described in any of the technical solutions of the first aspect above. Therefore, it possesses all the beneficial technical effects of the dimension acquisition method in any of the technical solutions of the first aspect above, which will not be elaborated further here.

[0219] In one embodiment according to this application, such as Figure 12 As shown, a robot 1200 is proposed, including: a size acquisition device 1100 as defined in the second or third aspect above, and / or a readable storage medium 1202 as defined in the fourth aspect above, thus having all the beneficial technical effects of the size acquisition device 1100 in the second or third aspect above, and / or the readable storage medium 1202 as defined in the fourth aspect above, which will not be elaborated further here.

[0220] In the above technical solution, the robot also includes: a depth camera, used to acquire image data, including color images and / or depth images.

[0221] In this technical solution, a depth camera acquires image data. The depth camera is mounted on the robot and moves with the robot. The image data can include depth images and color images.

[0222] It should be clarified that in the claims, description, and accompanying drawings of this invention, the term "plural" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description process, not to indicate or imply that the device or element referred to must have the described specific orientation, or be constructed and operated in a specific orientation. Therefore, these descriptions should not be construed as limiting the invention. The terms "connection," "installation," "fixing," etc., should be interpreted broadly. For example, "connection" can be a fixed connection between multiple objects, a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects or an indirect connection between multiple objects through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in this invention can be understood based on the specific circumstances of the above data.

[0223] In the claims, description, and accompanying drawings of this invention, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In the claims, description, and accompanying drawings of this invention, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0224] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A size acquisition method characterized by, The method comprises: obtaining first point cloud information of a target object in a first coordinate system and second point cloud information of a target plane in the first coordinate system, the first coordinate system being a camera coordinate system, and the target object being located on the target plane; constructing a second coordinate system according to the second point cloud information, the second coordinate system being a coordinate system corresponding to the target plane; determining size information of the target object according to the first point cloud information and the second coordinate system. The method comprises: obtaining a first image and a second image of a target scene, the first image being a color image, and the second image being a depth image, the target scene comprising the target object and the target plane; determining a first mask image and a second mask image through the first image, the first mask image matching the target object, and the second mask image matching the target plane; determining the first point cloud information and the second point cloud information according to the first mask image, the second mask image and the second image.

2. The size acquisition method according to claim 1, wherein The method comprises: identifying first image features and second image features in the first image, the first image features matching the target object, and the second image features matching the target plane; generating the first mask image through the first image features; and generating the second mask image through the second image features.

3. The size acquisition method of claim 1, wherein The method comprises: screening a first coordinate set in the first mask image and a second coordinate set in the second mask image according to depth values in the second image and a preset rule; determining the first point cloud information according to the first coordinate set and depth camera parameters, the depth camera parameters being parameters of a depth camera used to collect the second image; and determining the second point cloud information according to the second coordinate set and the depth camera parameters.

4. The size acquisition method according to claim 3, wherein The depth camera parameters comprise at least one of a scale factor of the depth camera and a coordinate of a principal point of the depth camera in an image coordinate system.

5. The size acquisition method according to claim 4, wherein The preset rule comprises: a depth value being greater than a preset depth value and a color value being a preset color value.

6. The size acquisition method according to any one of claims 1 to 5, wherein, The method comprises: determining a first plane equation according to the second point cloud information, the first plane equation matching the target plane; constructing the second coordinate system according to the first plane equation.

7. The size acquisition method according to claim 6, wherein The method comprises: obtaining a coordinate system rotation matrix of the first coordinate system and the second coordinate system; determining third coordinate information of the first point cloud information in the second coordinate system through the coordinate system rotation matrix; determining the size information of the target object according to the third coordinate information.

8. The size acquisition method according to claim 7, wherein The size information comprises a height value of the target object. The determining the size information of the target object according to the third coordinate information comprises: obtaining a distance value of each coordinate point in the third coordinate information from the target plane; screening a maximum distance value in the plurality of distance values as the height value of the target object.

9. The sizing method of claim 7, wherein, The size information comprises a width value and a length value of the target object. The determining the size information of the target object according to the third coordinate information comprises: determining fourth coordinate information corresponding to the target object according to the third coordinate information, the fourth coordinate information being coordinate information of a circumscribed rectangle of the target object; determining the width value and the length value of the target object according to the fourth coordinate information.

10. A size acquisition device characterized by, Comprise: an acquisition module, configured to acquire first point cloud information of a target object in a first coordinate system and second point cloud information of a target plane in the first coordinate system, the first coordinate system being a camera coordinate system, and the target object being located on the target plane; a construction module, configured to construct a second coordinate system according to the second point cloud information, the second coordinate system being a coordinate system corresponding to the target plane; a determination module, configured to determine size information of the target object according to the first point cloud information and the second coordinate system; the acquisition module is specifically configured to acquire a first image and a second image of a target scene, the first image being a color image, and the second image being a depth image, the target scene comprising the target object and the target plane; the determination module is configured to determine a first mask image and a second mask image through the first image, the first mask image being matched with the target object, and the second mask image being matched with the target plane; and determine the first point cloud information and the second point cloud information according to the first mask image, the second mask image and the second image.

11. A size acquisition device, characterized by Comprise: a memory having a program or instructions stored thereon; a processor configured to execute the program or instructions to implement the steps of the size acquisition method according to any one of claims 1 to 9.

12. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or instructions are executed by the processor to implement the steps of the size acquisition method according to any one of claims 1 to 9.

13. A robot, characterized in that Comprise: the size acquisition apparatus according to claim 10 or 11; Or the readable storage medium according to claim 12.

14. The robot of claim 13, wherein, Further comprise: a depth camera configured to collect image data, the image data comprising a color image and / or a depth image.

Images