Article height detection method and system and readable storage medium

Abstract

The invention provides an article height detection method and system and a readable storage medium, and relates to the technical field of image processing. According to the application, when the to-be-detected article is stored in the object carrying container, under the condition that the article storage depth map is acquired by using the three-dimensional structured light camera, point cloud conversion processing is performed on the article storage depth map; extracting first target point cloud data of which the corresponding mapping pixel positions are located in the container bottom surface area range, and then determining a target object surface feature point closest to the three-dimensional structured light camera from a global starting point in the first target point cloud data, so as to obtain a target object surface feature point based on a container bottom surface reference plane equation of the object carrying container; and calculating the vertical distance from the surface feature point of the target object to the bottom surface of the container to obtain the actual object height of the object to be detected in the object carrying container, thereby realizing the accurate detection effect of the global object height in the object carrying state of the container.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an article height detection method and system and a readable storage medium. BACKGROUND

[0002] With the rapid development of industrial automation technology, the application of automatic article sorting system is becoming more and more widespread. Generally, a mechanical arm is used to accurately place various articles in a specified container (such as a tray), and to ensure that the actual stacking height of the corresponding articles in the container does not exceed the preset height limit, so as to effectively avoid article dumping or damage during article transportation.

[0003] At present, the height detection scheme for the articles already stored in the container mainly relies on the laser sensor of point laser or line laser, which calculates the height of the object by measuring the distance data on a single point or a single line of laser. However, this height detection scheme is limited to single-point height measurement or single-section height measurement, and it is essentially difficult to effectively detect the overall article height in the container loading range from a global perspective. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an article height detection method and system and a readable storage medium, which can realize accurate detection of the overall article height from a global perspective by combining the full-field depth information acquisition operation and the three-dimensional visual point cloud processing operation under the reference of the container bottom surface, thereby realizing accurate detection of the overall article height.

[0005] In order to achieve the above-mentioned purpose, the technical solutions adopted by the embodiments of the present application are as follows: In a first aspect, the present application provides an article height detection method applied to a height calculation unit in communication connection with a three-dimensional structured light camera, wherein the three-dimensional structured light camera is arranged above a container for loading articles and faces the container bottom surface of the container for loading articles, and the method comprises: acquiring an article storage depth map collected by the three-dimensional structured light camera when the article to be detected is stored in the container for loading articles; performing point cloud conversion processing on the article storage depth map, and extracting first target point cloud data from the converted first point cloud data, wherein the mapping pixel position of the first target point cloud data in the article storage depth map is within the container bottom surface area range of the container for loading articles; determining a target article surface feature point closest to the three-dimensional structured light camera in the first target point cloud data; calculating the vertical distance from the target article surface feature point to the container bottom surface based on the container bottom surface reference plane equation of the container for loading articles, to obtain the actual article height of the article to be detected in the container for loading articles.

[0006] In an optional implementation, the step of determining the target object surface feature point closest to the three-dimensional structured light camera in the first target point cloud data comprises: performing multi-scale extreme point searching on the first target point cloud data according to actual distance values of all spatial points in the first target point cloud data to the three-dimensional structured light camera, to obtain a corresponding local extreme point set; performing noise extreme point removal processing on the local extreme point set to obtain a corresponding target extreme point set; taking a target extreme point closest to the three-dimensional structured light camera in the target extreme point set as the target object surface feature point.

[0007] In an optional implementation, the step of performing multi-scale extreme point searching on the first target point cloud data according to actual distance values of all spatial points in the first target point cloud data to the three-dimensional structured light camera, to obtain a corresponding local extreme point set comprises: performing extreme region division on all spatial points in the first target point cloud data according to actual distance values of all spatial points in the first target point cloud data, to obtain all candidate extreme regions of the first target point cloud data; for each candidate extreme region, constructing a target KD (K-Dimension) tree matching the candidate extreme region, then calling the target KD tree to perform nearest neighbor searching on each spatial point in the candidate extreme region respectively, and recursively reducing the search radius, to obtain a local extreme point in the local extreme point set matching the candidate extreme region.

[0008] In an optional implementation, the step of performing noise extreme point removal processing on the local extreme point set to obtain a corresponding target extreme point set comprises: for each local extreme point in the local extreme point set, detecting whether the local extreme point satisfies a neighborhood point normal vector consistency requirement in the local extreme point set, and detecting whether an actual neighborhood point density of the local extreme point in the local extreme point set is less than a preset point density threshold; if it is detected that the local extreme point does not satisfy the neighborhood point normal vector consistency requirement, and the actual neighborhood point density of the local extreme point is less than the preset point density threshold, the local extreme point is removed as a noise extreme point, otherwise the local extreme point is taken as a target extreme point in the target extreme point set.

[0009] In an optional implementation, the step of, for each local extreme point, detecting whether the local extreme point satisfies a neighborhood point normal vector consistency requirement in the local extreme point set comprises: determining all neighbor local extreme points in a neighborhood of the local extreme point from the set of local extreme points; calculating a normal vector angle between a real normal vector of each of the all neighbor local extreme points and a space normal vector of the local extreme point, and determining a number of real neighbor local extreme points corresponding to a normal vector angle greater than a preset angle threshold; detecting whether the number of real neighbor local extreme points is less than a preset number threshold of local extreme points; if it is detected that the number of real neighbor local extreme points is less than the preset number threshold of local extreme points, determining that the local extreme point meets the neighborhood point normal vector consistency requirement, otherwise determining that the local extreme point does not meet the neighborhood point normal vector consistency requirement.

[0010] In an optional embodiment, the method further comprises: acquiring a container empty depth map collected by the three-dimensional structured light camera when the object container is empty; performing point cloud conversion processing on the container empty depth map, and extracting second target point cloud data from the converted second point cloud data, wherein a mapping pixel position of the second target point cloud data in the container empty depth map is within a container bottom surface area range of the object container; performing multi-model plane fitting processing based on the second target point cloud data to obtain an inlier set of each of a plurality of initial fitting planes at the second target point cloud data; for each inlier set, performing plane parameter optimization on the initial fitting plane corresponding to the inlier set according to an initial distance value of each inlier in the inlier set to the initial fitting plane, to obtain a target optimized plane equation matched with the inlier set; from the plurality of target optimized plane equations, selecting a target optimized plane equation corresponding to a plane normal vector closest to the vertical direction as the container bottom surface reference plane equation.

[0011] In an optional embodiment, for each inlier set, the step of performing plane parameter optimization on the initial fitting plane corresponding to the inlier set according to an initial distance value of each inlier in the inlier set to the initial fitting plane, to obtain a target optimized plane equation matched with the inlier set, comprises: constructing a diagonal weight matrix matched with the inlier set according to the initial distance value of each inlier in the inlier set, wherein each inlier in the inlier set corresponds to an influence weight in the diagonal weight matrix, and the initial distance value of the same inlier is inversely related to the influence weight; based on a weighted least squares optimization principle, performing point set distribution optimization processing on a point set representation matrix of the inlier set according to the diagonal weight matrix to obtain a corresponding target point set matrix; singular value decomposition is performed on the target point set matrix to obtain target plane parameters matched with the minimum singular value of the target point set matrix; A plane equation is constructed according to the target plane parameters to obtain a target optimization plane equation matched with the inner point set.

[0012] In an optional embodiment, a point set distribution optimization relationship among a target point set matrix corresponding to the i-th inner point set, a diagonal weight matrix and a point set representation matrix of the i-th inner point set is characterized by the following formula: ; wherein, is used to represent a target point set matrix corresponding to the i-th inner point set, is used to represent a diagonal weight matrix corresponding to the i-th inner point set, is used to represent a point set representation matrix corresponding to the i-th inner point set.

[0013] In a second aspect, the present application provides an article height detection system, which comprises a three-dimensional structured light camera, an article container and a height calculation unit; The three-dimensional structured light camera is arranged above the article container, and a depth information collection direction of the three-dimensional structured light camera is directly opposite to a container bottom surface of the article container; The height calculation unit is in communication connection with the three-dimensional structured light camera, and is used to acquire a depth map actually collected by the three-dimensional structured light camera; The height calculation unit is pre-configured with a computer program or a hardware circuit module, and performs the article height detection method in any one of the foregoing embodiments by driving the hardware circuit module or running the computer program.

[0014] In a third aspect, the present application provides a readable storage medium, which stores a computer program, and when the computer program is executed by a height calculation unit in communication connection with a three-dimensional structured light camera, the article height detection method in any one of the foregoing embodiments is realized, wherein the three-dimensional structured light camera is arranged above an article container and directly opposite to a container bottom surface of the article container.

[0015] In this case, the beneficial effects of the embodiments of the present application can include the following contents: ​​​​This application, when an item to be detected is stored in a container, uses a 3D structured light camera to acquire an item storage depth map. It then performs point cloud conversion processing on this depth map and extracts first target point cloud data from the converted first point cloud data. The target point cloud data is located within the container's bottom surface region (which characterizes the container's carrying capacity). Subsequently, the application globally determines the target item surface feature point closest to the 3D structured light camera from the first target point cloud data. Based on the container's bottom reference plane equation, it calculates the vertical distance from this target item surface feature point to the container's bottom surface, thus obtaining the actual height of the item within the container. This allows for the organic combination of full-field depth information acquisition and 3D visual point cloud processing operations under the container's bottom reference, enabling the global detection of the overall height of items within the container's carrying capacity, achieving accurate detection of the overall item height.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram illustrating the system deployment of the item height detection system provided in this application embodiment in either an empty container state or a container loaded with goods. Figure 2 One of the flowcharts for the article height detection method provided in the embodiments of this application; Figure 3 for Figure 2 A flowchart illustrating the sub-steps included in step S230; Figure 4 A second schematic flowchart illustrating the article height detection method provided in this application embodiment; Figure 5 for Figure 4 A flowchart illustrating the sub-steps included in step S340.

[0019] Icons: 10-Object height detection system; 11-Height calculation unit; 12-3D structured light camera; 13-Container. Detailed Implementation

[0020] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. The components of the embodiments of the present application generally described and shown in the drawings herein can be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0022] It should be noted that: similar reference numerals and letters indicate similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0023] In the description of the present application, it should be understood that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship commonly understood by those skilled in the art, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0024] In the description of the present application, it should also be noted that unless otherwise explicitly specified and limited, the terms "set", "mount", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0025] Moreover, in the description of the present application, it can be understood that the relational terms such as "first" and "second" and the like are used only to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features can be combined with each other unless otherwise stated.

[0026] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features can be combined with each other unless otherwise stated.

[0027] Please refer to Figure 1 , Figure 1 is a schematic diagram of the system deployment of the article height detection system 10 in the container empty state or the container loaded state. In the embodiments of the present application, the article height detection system 10 can include a height calculation unit 11, a three-dimensional structured light camera 12, and a loaded container 13. The three-dimensional structured light camera 12 is configured to realize the function of collecting depth information, and the loaded container 13 is configured to store the to-be-detected articles (which can be single articles or effective stacked structures of multiple articles) whose heights need to be detected. Figure 1 For example, in (a), when the loaded container 13 does not store any articles, the article height detection system 10 is in the container empty state. Figure 1 For example, in (b), when the loaded container 13 stores to-be-detected articles (which can be single articles or effective stacked structures of multiple articles), the article height detection system 10 is in the container loaded state.

[0028] In the embodiments of the present application, the three-dimensional structured light camera 12 is arranged above the loaded container 13, and the depth information collection direction (i.e., the direction indicated by the blue filled arrow in (a) or (b)) of the three-dimensional structured light camera 12 is directly opposite to the bottom surface of the loaded container 13, so as to collect the full-field depth information of the container working area where the loaded container is located. Figure 1

[0029] ​In the embodiment of the present application, the height calculation unit 11 can be in communication connection with the three-dimensional structured light camera 12, for controlling the actual working state of the three-dimensional structured light camera 12 and acquiring the depth map actually collected by the three-dimensional structured light camera 12. The height calculation unit 11 can be, but is not limited to, a notebook computer, a smart phone, a personal computer, etc.

[0030] In the embodiment of the present application, the height calculation unit 11 is pre-configured with a specific computer program or a hardware circuit module related to the article height detection function, and can, when the article height detection system 10 is in the container loading state, realize the precise detection effect of the global article height of the to-be-detected article in the loading container 13 by means of the organic combination of the full-field depth information collection operation and the three-dimensional visual point cloud processing operation in the container loading state, and globally considering the overall height of the articles in the loading range of the container under the reference action of the container bottom surface of the loading container 13, by running the aforementioned specific computer program or driving the aforementioned hardware circuit module.

[0031] It can be understood that, Figure 1 The block diagram shown is only one constituent schematic diagram of the article height detection system 10, and the article height detection system 10 can further include more or less components than those shown in Figure 1 or have a different configuration than that shown in Figure 1 . Figure 1 The components shown in the embodiment of the present application can be realized by hardware, software or a combination thereof.

[0032] In the present application, in order to ensure that the height calculation unit 11 in communication connection with the three-dimensional structured light camera 12 can accurately detect the global article height of the to-be-detected article in the container loading state, the embodiment of the present application realizes the aforementioned purpose by providing an article height detection method applied to the aforementioned height calculation unit 11, and the article height detection method provided by the present application is described in detail as follows.

[0033] Please refer to Figure 2 , Figure 2 is one of the flowcharts of the article height detection method provided by the embodiment of the present application. In the embodiment of the present application, Figure 2 The article height detection method shown in

[0034] Step S210: acquiring the article storage depth map collected by the three-dimensional structured light camera when the to-be-detected article is stored in the loading container.

[0035] In the embodiment, when the to-be-detected article is stored in the carrier container 13, the article height detection system 10 enters the carrier container state, at which time the height calculation unit 11 can control the three-dimensional structured light camera 12 to face the container bottom of the carrier container 13 to perform full-field depth information collection processing, so as to obtain the article storage depth map of the to-be-detected article in the carrier container.

[0036] In step S220, the article storage depth map is subjected to point cloud conversion processing, and first target point cloud data is extracted from the converted first point cloud data.

[0037] In the embodiment, after obtaining the article storage depth map, the height calculation unit 11 can obtain first point cloud data converted from the article storage depth map through a point cloud conversion operation in a world coordinate system (which can be constructed on the lens of the three-dimensional structured light camera 12, and the vertical axis of the world coordinate system is consistent with the direction of the lens of the three-dimensional structured light camera 12) in which the three-dimensional structured light camera 12 is located. Then, the height calculation unit 11 can perform point cloud preprocessing (including voxel downsampling operation, statistical filtering denoising operation, etc.) on the first point cloud data to eliminate outliers in the first point cloud data and reduce the subsequent calculation complexity. Then, the height calculation unit 11 can perform regional clustering segmentation processing on the first point cloud data after point cloud preprocessing based on the regional growth algorithm of Euclidean distance, so as to extract first target point cloud data directly associated with the container bottom area range of the carrier container 13 (which is used to represent the carrier range of the corresponding carrier container 13) from the first point cloud data. In the first target point cloud data, the mapping pixel positions of all spatial points in the article storage depth map are in the container bottom area range of the carrier container in the article storage depth map.

[0038] In step S230, a target article surface feature point closest to the three-dimensional structured light camera is determined in the first target point cloud data.

[0039] In the embodiment, the height calculation unit 11 can use a spatial extreme value search algorithm to filter out the article surface spatial point (i.e., the target article surface feature point) of the to-be-detected article closest to the three-dimensional structured light camera 12 from the first target point cloud data.

[0040] Optionally, please refer to Figure 3 , Figure 3 is Figure 2A flowchart of sub-steps included in step S230 is shown. In the embodiment of the present application, step S230 can include sub-step S231 to sub-step S233 to globally consider the effective distance conditions of each position on the to-be-detected article to the three-dimensional structured light camera 12, and screen out target article surface feature points that best represent the overall article height of the to-be-detected article.

[0041] In sub-step S231, multi-scale extreme point searching is performed on the first target point cloud data according to the actual distance values of all spatial points in the first target point cloud data to the three-dimensional structured light camera, to obtain a corresponding local extreme point set.

[0042] In the embodiment, all spatial points in the first target point cloud data can be subjected to extreme region division according to the actual distance values of all spatial points in the first target point cloud data, to obtain all candidate extreme regions of the first target point cloud data (wherein the actual distance values of all spatial points in a single candidate extreme region are in the same distance value interval, all spatial points in a single candidate extreme region are mutually adjacent, and the neighborhood point density of each spatial point in a same candidate extreme region is greater than a preset density threshold), and then a target KD tree matched with each candidate extreme region is constructed, and then the target KD tree is called to perform nearest neighbor searching on each spatial point in the candidate extreme region, and the search radius is recursively reduced, to obtain a local extreme point in the local extreme point set that matches the candidate extreme region.

[0043] In sub-step S232, noise extreme point removal processing is performed on the local extreme point set, to obtain a corresponding target extreme point set.

[0044] In the embodiment, for each local extreme point in the local extreme point set, whether the local extreme point satisfies the neighborhood point normal vector consistency requirement in the local extreme point set can be detected, and whether the actual neighborhood point density of the local extreme point in the local extreme point set is less than a preset point density threshold can be detected, and then in a case where it is detected that the local extreme point does not satisfy the neighborhood point normal vector consistency requirement and the actual neighborhood point density of the local extreme point is less than the preset point density threshold, the local extreme point is removed as a noise extreme point, otherwise the local extreme point is taken as a target extreme point in the target extreme point set.

[0045] For each local extreme point, the step of detecting whether the local extreme point satisfies the neighborhood point normal vector consistency requirement in the local extreme point set includes: all neighbor extreme points in the extreme point neighborhood of the local extreme point are determined in the local extreme point set; Calculate the normal vector angle between the actual normal vector of each of the all neighbor extreme points and the space normal vector of the local extreme point, and determine the number of actual neighbor extreme points whose corresponding normal vector angle is greater than a preset angle threshold value; Detect whether the number of actual neighbor extreme points is less than a preset extreme point number threshold value; If it is detected that the number of actual neighbor extreme points is less than the preset extreme point number threshold value, it is determined that the local extreme point meets the neighborhood point normal vector consistency requirement, otherwise it is determined that the local extreme point does not meet the neighborhood point normal vector consistency requirement.

[0046] Therefore, the present application can effectively confirm and eliminate the noise extreme points of the local extreme point set by executing the specific steps of the above-mentioned sub-step S232, so as to complete the local extreme point set after the noise extreme point elimination processing as the target extreme point set.

[0047] Sub-step S233, the target extreme point closest to the three-dimensional structured light camera in the target extreme point set is taken as the target object surface feature point.

[0048] Therefore, the present application can consider the effective distance conditions of each position on the to-be-detected object to the three-dimensional structured light camera 12 globally by executing the above-mentioned sub-step S231 to sub-step S233, and screen out the target object surface feature point which can best represent the overall object height of the to-be-detected object.

[0049] Step S240, based on the container bottom surface reference plane equation of the object container, the vertical distance of the target object surface feature point to the container bottom surface is calculated to obtain the actual object height of the to-be-detected object in the object container.

[0050] In the present embodiment, the vertical distance (which can be represented by “ ”) of any one space point (which can be represented by “ ”) to a certain specific plane (which can be represented by the plane equation “ ”, wherein , , , are the plane parameters of the plane equation) can be calculated by the formula “ ”.

[0051] Therefore, the application can realize the precise detection effect of the global article height of the to-be-detected article in the container 13 by performing the above steps S210-S240, detecting the overall height of the article in the container loading range under the global consideration of the container bottom surface reference of the container 13 by using the organic combination of the full-field depth information acquisition operation and the three-dimensional visual point cloud processing operation in the container loading state, and realizing the precise detection effect of the global article height of the to-be-detected article in the container 13.

[0052] Optionally, please refer to Figure 4 , Figure 4 is a flowchart of the article height detection method provided by the embodiment of the application. In the embodiment of the application, compared with the article height detection method shown in Figure 2 , the article height detection method shown in Figure 4 may further include steps S310-S350 to realize the dynamic modeling of the reference plane equation of the container bottom surface in the container empty state, so as to eliminate the system measurement error caused by the camera installation angle error, the container placement tilt and other external factors.

[0053] In step S310, the container empty depth map collected by the three-dimensional structured light camera when the container is idle is obtained.

[0054] In the embodiment, when there is no article stored in the container 13 (that is, the container 13 is idle), the article height detection system 10 enters the container empty state, and at this time, the height calculation unit 11 can control the three-dimensional structured light camera 12 to face the container bottom of the container 13 to collect and process the full-field depth information, so as to obtain the container empty depth map of the container when there is no article stored.

[0055] In step S320, the point cloud conversion processing is performed on the container empty depth map, and the second target point cloud data is extracted from the converted second point cloud data.

[0056] In this embodiment, after obtaining the container vacancy depth map, the height calculation unit 11 can obtain second point cloud data converted from the container vacancy depth map through a point cloud conversion operation in the world coordinate system where the three-dimensional structured light camera 12 is located. Then, the height calculation unit 11 can perform point cloud preprocessing on the second point cloud data to remove outliers in the second point cloud data, thereby reducing the subsequent calculation complexity. Then, the height calculation unit 11 can perform regional clustering segmentation processing on the second point cloud data after point cloud preprocessing based on the regional growth algorithm of the Euclidean distance, so as to extract second target point cloud data directly related to the container bottom surface region range of the object container 13 from the second point cloud data. In the second target point cloud data, the mapping pixel positions of all spatial points in the container vacancy depth map are within the container bottom surface region range of the object container in the container vacancy depth map.

[0057] In step S330, multi-model plane fitting processing is performed based on the second target point cloud data to obtain an inlier set of each of the plurality of initial fitting planes at the second target point cloud data.

[0058] In this embodiment, the multi-model plane fitting algorithm based on RANSAC (RANdom SAmple Consensus) can be used to perform multi-model plane fitting processing on the second target point cloud data to obtain an initial plane equation of each of the plurality of initial fitting planes related to the second target point cloud data, and an inlier set used in the construction of the initial plane equation of each initial fitting plane. In any inlier set corresponding to an initial fitting plane, the initial distance value of each inlier in the inlier set to the corresponding initial fitting plane is less than a preset distance value.

[0059] In step S340, for each inlier set, the initial distance value of each inlier in the inlier set to the corresponding initial fitting plane is used to optimize the plane parameters of the initial fitting plane corresponding to the inlier set to obtain a target optimized plane equation matched with the inlier set.

[0060] In this embodiment, the height calculation unit 11 can implement the optimization effect of the initial plane equation of the corresponding initial fitting plane based on the principle of weighted least squares optimization, and amplify the contribution of the inliers in the corresponding inlier set close to the initial fitting plane while weakening the influence of the inliers in the corresponding inlier set on the initial fitting plane during the plane equation optimization process, so as to ensure that the target optimized plane equation matched with a single inlier set can effectively reflect the real placement status of the related container.

[0061] Optionally, please refer to Figure 5 , Figure 5 is Figure 4A flowchart illustrating the sub-steps included in step S340. In this embodiment, step S340 may include sub-steps S341 to S344 to ensure that the final target optimization plane equation can effectively reflect the actual placement of the container in the corresponding interior point set.

[0062] Sub-step S341: Based on the initial distance values ​​of all interior points in the interior point set, construct a diagonal weight matrix that matches the interior point set.

[0063] In this embodiment, each inlier in a single inlier set has a separate influence weight at the diagonal weight matrix, and the initial distance value of the same inlier is inversely correlated with the influence weight.

[0064] Sub-step S342, based on the principle of weighted least squares optimization, performs point set distribution optimization on the point set representation matrix of the interior point set according to the diagonal weight matrix to obtain the corresponding target point set matrix.

[0065] In this embodiment, the second target point cloud data... The point set distribution optimization relationship between the target point set matrix, diagonal weight matrix, and point set representation matrix corresponding to each internal point set is represented by the following formula: ; in, Used to indicate the first The target point set matrix corresponding to each interior point set Used to indicate the first The diagonal weight matrix corresponding to each interior point set Used to indicate the first The point set representation matrix corresponding to each interior point set.

[0066] And for the first For an interior point set, if the interior point set includes... If there are interior points, then the point set representation matrix of the interior point set can be represented as: ; in, Used to indicate the first The set of interior points, the first The three-dimensional spatial coordinates of an interior point in the world coordinate system.

[0067] Sub-step S343: Perform singular value decomposition on the target point set matrix to obtain the target plane parameters that match the minimum singular value of the target point set matrix.

[0068] In this embodiment, when the first After the target point set matrix corresponding to the inner point set is obtained, singular value decomposition processing is performed on the target point set matrix to obtain the right singular vector of the target point set matrix when the minimum singular value is maintained, and then the right singular vector is taken as the first target plane parameter corresponding to the inner point set which can be represented by "a".

[0069] In the sub-step S244, a plane equation is constructed according to the target plane parameter to obtain a target optimization plane equation matched with the inner point set.

[0070] Therefore, the initial plane equation optimization effect of the initial fitting plane can be realized based on the weighted least square optimization principle by executing the above-mentioned sub-step S341 to sub-step S344, and it is ensured that the target optimization plane equation finally obtained and matched with the single inner point set can effectively reflect the real placement state of the container.

[0071] In step S350, among the plurality of target optimization plane equations, a target optimization plane equation corresponding to a plane normal vector closest to the vertical direction is selected as a container bottom surface reference plane equation.

[0072] Therefore, by executing the above-mentioned step S310 to step S350, the reference plane equation of the container bottom surface can be dynamically modeled in the empty state of the container to eliminate the system measurement error caused by the camera installation angle error, the container placement tilt and other external factors.

[0073] In the embodiments provided by the present application, it should be understood that the disclosed device and method can also be implemented by other ways. The device embodiments described above are only schematic, and for example, the flowcharts and block diagrams in the attached drawings illustrate the possible implementation architectures, functions and operation of the device, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the attached drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0074] ​In addition, each function module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. When the functions are realized in the form of a software function module and sold or used as an independent product, the software function module can be stored in a readable storage medium. Based on this understanding, the technical solutions of the present application essentially or say the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium and includes a plurality of instructions for causing the height measurement unit 11 in communication connection with the three-dimensional structured light camera 12 to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0075] The above is only various embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting the height of an object, characterized in that, A height measurement unit is applied to a three-dimensional structured light camera, wherein the three-dimensional structured light camera is positioned above a container and directly facing the bottom surface of the container. The method includes: Obtain the object storage depth map captured by the three-dimensional structured light camera when the object to be detected is stored in the container; The depth map of the stored items is processed by point cloud transformation, and first target point cloud data is extracted from the transformed first point cloud data, wherein the mapped pixel position of the first target point cloud data in the depth map of the stored items is within the bottom surface area of ​​the container of the carrying container. In the first target point cloud data, determine the target object surface feature point that is closest to the three-dimensional structured light camera; Based on the reference plane equation of the container bottom surface, the vertical distance from the surface feature points of the target item to the bottom surface of the container is calculated to obtain the actual height of the item to be detected inside the container.

2. The method according to claim 1, characterized in that, The step of determining the target object surface feature point closest to the 3D structured light camera in the first target point cloud data includes: Based on the actual distance values ​​from each spatial point in the first target point cloud data to the three-dimensional structured light camera, a multi-scale extreme point search is performed on the first target point cloud data to obtain the corresponding local extreme point set. The local extreme point set is subjected to noise extreme point removal processing to obtain the corresponding target extreme point set; The target extreme points are concentrated at the target extreme points closest to the three-dimensional structured light camera, and these are used as the surface feature points of the target object.

3. The method according to claim 2, characterized in that, The step of performing a multi-scale extreme point search on the first target point cloud data based on the actual distance values ​​from each spatial point in the first target point cloud data to the three-dimensional structured light camera, to obtain the corresponding local extreme point set, includes: Based on the actual distance values ​​corresponding to each spatial point in the first target point cloud data, extreme value regions are divided for all spatial points in the first target point cloud data to obtain all candidate extreme value regions of the first target point cloud data. For each candidate extreme value region, a target KD tree matching the candidate extreme value region is constructed. Then, the target KD tree is called to perform nearest neighbor search on each spatial point in the candidate extreme value region, and the search radius is recursively reduced to obtain the local extreme value points in the local extreme value point set that match the candidate extreme value region.

4. The method according to claim 2, characterized in that, The step of removing noise extrema from the local extrema set to obtain the corresponding target extrema set includes: For each local extreme point in the local extreme point set, it is detected whether the local extreme point satisfies the requirement of consistency of the normal vector of the neighboring point in the local extreme point set, and it is detected whether the actual density of the neighboring point of the local extreme point in the local extreme point set is less than a preset point density threshold. If the local extremum point does not meet the requirement of consistency of the normal vector of the neighboring points, and the actual density of the neighboring points of the local extremum point is less than the preset point density threshold, then the local extremum point is removed as a noise extremum point; otherwise, the local extremum point is taken as a target extremum point in the target extremum point set.

5. The method according to claim 4, characterized in that, For each local extremum point, the step of detecting whether the local extremum point satisfies the consistency requirement of the normal vector of its neighborhood points at the local extremum point set includes: In the set of local extreme points, determine all neighboring extreme points within the neighborhood of the extreme point that is in the local extreme point's extreme point neighborhood; Calculate the angle between the actual normal vector of each of the neighboring extreme points and the spatial normal vector of the local extreme point, and determine the number of actual neighboring extreme points whose normal vector angle is greater than a preset angle threshold; Detect whether the actual number of neighboring extreme points is less than a preset extreme point number threshold; If the number of actual neighboring extreme points is less than the preset threshold number of extreme points, then the local extreme point is determined to meet the neighboring point normal vector consistency requirement; otherwise, the local extreme point is determined not to meet the neighboring point normal vector consistency requirement.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Obtain the container vacancy depth map captured by the three-dimensional structured light camera when the container is idle; The container vacancy depth map is processed by point cloud transformation, and second target point cloud data is extracted from the transformed second point cloud data, wherein the mapped pixel position of the second target point cloud data in the container vacancy depth map is within the container bottom area of ​​the cargo container; Based on the second target point cloud data, a multi-model plane fitting process is performed to obtain the interior point set of each of the multiple initial fitting planes at the second target point cloud data. For each set of interior points, based on the initial distance values ​​from each interior point in the set to the corresponding initial fitting plane, the plane parameters of the initial fitting plane corresponding to the set of interior points are optimized to obtain the target optimized plane equation that matches the set of interior points. Among multiple target optimization plane equations, the target optimization plane equation whose corresponding plane normal vector is closest to the vertical direction is selected as the reference plane equation for the bottom surface of the container.

7. The method according to claim 6, characterized in that, For each set of interior points, the step of optimizing the plane parameters of the initial fitting plane corresponding to the set of interior points based on the initial distance values ​​from each interior point in the set to the corresponding initial fitting plane, to obtain the target optimization plane equation matching the set of interior points, includes: Based on the initial distance values ​​of all inliers in the inlier set, a diagonal weight matrix matching the inlier set is constructed, wherein each inlier in the inlier set has a separate influence weight at the diagonal weight matrix, and the initial distance value of the same inlier is inversely correlated with the influence weight. Based on the principle of weighted least squares optimization, the point set distribution of the internal point set is optimized according to the diagonal weight matrix to obtain the corresponding target point set matrix. Singular value decomposition is performed on the target point set matrix to obtain the target plane parameters that match the minimum singular value of the target point set matrix; The plane equation is constructed according to the target plane parameters to obtain the target optimized plane equation that matches the interior point set.

8. The method according to claim 7, characterized in that, No. The point set distribution optimization relationship between the target point set matrix, diagonal weight matrix, and point set representation matrix corresponding to each internal point set is represented by the following formula: ； in, Used to indicate the first The target point set matrix corresponding to each interior point set Used to indicate the first The diagonal weight matrix corresponding to each interior point set Used to indicate the first The matrix representing the point set corresponding to each interior point set.

9. An object height detection system, characterized in that, The system includes a three-dimensional structured light camera, a cargo container, and a height measurement unit; The three-dimensional structured light camera is positioned above the container, and the depth information acquisition direction of the three-dimensional structured light camera is facing the bottom surface of the container. The height calculation unit is communicatively connected to the three-dimensional structured light camera and is used to acquire the depth map actually collected by the three-dimensional structured light camera. The height measurement unit is pre-configured with a computer program or hardware circuit module, and executes the object height detection method according to any one of claims 1-8 by driving the hardware circuit module or running the computer program.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the height measurement unit that is communicatively connected to the three-dimensional structured light camera, it implements the object height detection method according to any one of claims 1-8, wherein the three-dimensional structured light camera is disposed above the container and directly facing the bottom surface of the container.

Images