Shelf detection method, electronic device and computer readable storage medium

Abstract

The present application discloses a shelf detection method, an electronic device and a computer-readable storage medium, the shelf detection method comprising: obtaining point cloud data scanned by a mobile robot, determining the shelf leg point cloud and its matching shelf model based on the point cloud data; obtaining the first shelf center target pose based on the shelf leg point cloud and the shelf model; classifying the shelf leg point cloud based on the first shelf center target pose and the shelf model to obtain the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud; obtaining the second shelf center target pose based on the first direction point cloud, the second direction point cloud and the shelf model; wherein the second shelf center target pose is used to generate the target position of the mobile robot. The above scheme can improve the accuracy of detecting the center position of the shelf.

Description

Technical Field

[0001] The present application relates to the technical field of automatic shelf handling, and in particular to a shelf detection method, an electronic device and a computer-readable storage medium. Background Art

[0002] In the field of logistics and industry, mobile robots generally refer to automated guided vehicles (AGVs), which are transport vehicles equipped with electromagnetic or optical automatic guidance devices, capable of traveling along a prescribed guidance path, and with safety protection and transfer functions. They can replace manual lifting and handling of containers such as shelves. Mobile robots need to move accurately to the bottom of the shelves to perform shelf handling tasks. In some cases, the shelves are not placed accurately or the positioning information of the mobile robot is not accurate enough, making it impossible for the mobile robot to accurately reach the bottom of the shelves. Therefore, how to improve the accuracy of detecting the center position of the shelf has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a shelf detection method, an electronic device and a computer-readable storage medium, which can improve the accuracy of detecting the center position of the shelf.

[0004] To solve the above technical problems, the first aspect of the present application provides a shelf detection method, including: acquiring point cloud data scanned by a mobile robot, and determining a shelf leg point cloud and a matching shelf model based on the point cloud data; obtaining a first shelf center target pose based on the shelf leg point cloud and the shelf model; classifying the shelf leg point cloud based on the first shelf center target pose and the shelf model to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud; obtaining a second shelf center target pose based on the first direction point cloud, the second direction point cloud and the shelf model; wherein the second shelf center target pose is used to generate the target position of the mobile robot.

[0005] To solve the above technical problem, the second aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the method described in the first aspect above.

[0007] In the above scheme, the electronic device obtains the point cloud data scanned by the mobile robot, determines the shelf leg point cloud and the shelf model matching the shelf leg point cloud based on the point cloud data, obtains the first shelf center target pose based on the shelf leg point cloud and the shelf model matching it, classifies the shelf leg point cloud based on the first shelf center target pose and the shelf model, obtains the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud, obtains the second shelf center target pose for generating the target position of the mobile robot based on the first direction point cloud, the second direction point cloud and the shelf model, classifies the shelf leg point cloud, and performs constraint calculation using the classified point cloud to make the second shelf center target pose more accurate, thereby being able to detect the accuracy of the shelf center position. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a structural schematic diagram of an embodiment of a mobile robot provided by the present application;

[0010] Figure 2 This application provides a two-dimensional grid map Figure 1 Schematic diagram of implementation method;

[0011] Figure 3 It is a schematic diagram of an implementation method of a shelf model provided by the present application;

[0012] Figure 4 It is a flow chart of an implementation method of a shelf detection method of the present application;

[0013] Figure 5 It is a schematic diagram of the overall process of the shelf detection method provided by this application;

[0014] Figure 6 yes Figure 4 Step S402 corresponds to a flowchart of an implementation method;

[0015] Figure 7 It is a schematic diagram of calculating the coordinate position of the center of the shelf provided in this application;

[0016] Figure 8 yes Figure 4 Step S403 corresponds to a flowchart of an implementation method;

[0017] Fig. 9is a schematic diagram of classifying the shelf leg point cloud provided by the present application;

[0018] Fig.10 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0019] Fig.11 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0022] This application estimates the posture of the mobile robot based on the odometer, laser sensor and prior map. When docking with the shelf, the laser sensor is first used to obtain the laser point cloud directly in front of the mobile robot. After performing ROI, clustering, filtering and other operations on the point cloud, a combination of the shelf model and the shelf leg point cloud is screened out, and the shelf leg point cloud is segmented to obtain point clouds belonging to the two sides of the L shape, and the vertex coordinates of the L-shaped shelf legs are obtained, and then the initial coordinates of the center of the shelf leg are obtained in combination with the shelf model; then the shelf leg point cloud data is used as observation to construct the error equation, and a nonlinear optimization solution is performed to calculate the posture of the shelf leg; after obtaining the postures of the four shelf legs, the center posture of the shelf is calculated, and its posture is converted to the world coordinate system; when the mobile robot enters under the shelf, a secondary check is performed to check whether the relative position of the mobile robot and the shelf is as expected, and finally to ensure that the mobile robot reaches the position directly under the shelf, so that the mobile robot can successfully lift the shelf.

[0023] See also Figure 1-Figure 3 , Figure 1 is a structural schematic diagram of an embodiment of a mobile robot provided by the present application, Figure 2 This application provides a two-dimensional grid map Figure 1 Schematic diagram of implementation method, Figure 3 This is a schematic diagram of an implementation of a shelf model provided by this application. The mobile robot involved in this application includes a mobile chassis (including a motion controller, a motor, a battery, an embedded computer, an odometer, etc.), a laser radar sensor, etc. Its basic functions are described as follows:

[0024] The mobile robot generates a two-dimensional grid map based on the storage environment, such as Figure 2 The mobile robot uses the odometer to estimate the change in the movement of the mobile robot, and uses the lidar sensor to scan the contour of the environment. Before the mobile robot enters the shelf, it will perform shelf detection at the shelf detection preparation point, detect the center of the shelf, move to the bottom of the shelf and successfully lift the shelf. The corresponding shelf model is as follows Figure 3 shown.

[0025] The following first introduces the shelf detection process in the above process. Figure 4 and Figure 5 , Figure 4 It is a flowchart of an implementation method of the shelf detection method of the present application. Figure 5 : is a schematic diagram of the overall process of the shelf detection method provided by the present application. The shelf detection method includes:

[0026] S401: Acquire point cloud data scanned by a mobile robot, and determine the shelf leg point cloud and its matching shelf model based on the point cloud data.

[0027] Specifically, the electronic device obtains point cloud data scanned by the mobile robot, and based on the point cloud data, determines a shelf leg point cloud and a shelf model matching the shelf leg point cloud.

[0028] In one application, an electronic device obtains point cloud data scanned by a mobile robot using a laser sensor, divides the point cloud data into multiple clusters using a clustering algorithm, determines the shelf leg point cloud corresponding to the shelf by analyzing the shape, size, position and other features of the clusters, and traverses the shelf leg point cloud and multiple pre-saved shelf model combinations to determine a matching shelf leg point cloud and shelf model combination.

[0029] In an application scenario, before the electronic device performs shelf detection, it is necessary to pre-process the point cloud data obtained by the mobile robot using a laser sensor to scan. The electronic device will first exclude invalid point cloud data, such as no data points or outlier points. Then, the electronic device will convert the laser point cloud from the laser coordinate system to the robot body coordinate system, so that it can be processed and judged in the robot body coordinate system. Finally, the electronic device cuts the point cloud according to the pre-set target area range so that only the point cloud within the target area is retained. For example, the present application can set the range of +0.5m to +3m before and after the mobile robot, -1.5m to +1.5m left and right, and 0m to +1.5m up and down as the recognition ROI, and all point clouds outside this range are eliminated. Moreover, in most scenarios, the shelf legs are perpendicular to the ground, and the electronic device can project the shelf leg point cloud to a two-dimensional plane for processing, perform Euclidean clustering on the pre-processed point cloud, and eliminate clusters that do not meet the point number requirements. The shelf leg point cloud corresponding to the shelf is determined based on the clustered point cloud data, and the shelf leg point cloud and multiple pre-saved shelf model combinations are traversed to determine the matching shelf leg point cloud and shelf model combination.

[0030] It should be noted that in the robot body coordinate system, the x-axis is the direction of the body pointing to the shelf, and the y-axis is the perpendicular direction of the x-axis.

[0031] In another application method, an electronic device obtains point cloud data scanned by a mobile robot using a laser sensor, uses a pre-trained deep learning model to identify the point cloud data, determines the shelf leg point cloud corresponding to the shelf, and traverses the shelf leg point cloud and multiple pre-saved shelf model combinations to determine a matching shelf leg point cloud and shelf model combination.

[0032] S402: Based on the shelf leg point cloud and the shelf model, obtain the first shelf center target pose.

[0033] Specifically, based on the shelf leg point cloud and its matched shelf model, the target pose of the center of the first shelf is obtained.

[0034] In one application, stable feature points are extracted from the shelf leg point cloud, the correspondence between the shelf legs is found by matching these feature points, and the target pose of the center of the first shelf is calculated based on the correspondence and the shelf model.

[0035] In one application scenario, stable feature points are extracted from the shelf leg point cloud, and the extracted feature points are matched to find the correspondence between the shelf legs. Based on the matched feature point pairs, the relative pose between the shelf legs is calculated using a three-dimensional reconstruction or pose estimation algorithm. The calculated shelf leg pose is combined with the shelf model to further optimize the pose, and the first shelf center target pose is calculated based on the optimized shelf leg pose.

[0036] It should be noted that the first shelf center target pose is a roughly calculated shelf center pose.

[0037] In another application, according to the shape and layout of the shelf leg point cloud and in combination with the shelf model to set geometric constraints, the target position of the center of the first shelf is calculated through an optimization algorithm.

[0038] S403: Based on the first shelf center target pose and the shelf model, the shelf leg point cloud is classified to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud.

[0039] Specifically, the shelf leg point cloud is an L-shaped point cloud. Based on the first shelf center target pose and the shelf model, the shelf leg point cloud is classified to obtain the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud, wherein the first direction point cloud corresponds to the horizontal point cloud in the L-shaped point cloud, and the second direction point cloud corresponds to the longitudinal point cloud in the L-shaped point cloud.

[0040] In one application, PCA (principal components analysis) is performed on the shelf leg point cloud based on the first shelf center target posture to determine the first direction of the shelf leg point cloud and the second direction of the shelf leg point cloud (usually perpendicular to the first direction), and the shelf leg point cloud is classified in combination with the shelf model to obtain the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud.

[0041] In another application, a pre-trained classifier is used to classify the shelf leg point cloud based on the first shelf center target pose and the shelf model to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud.

[0042] S404: Based on the first direction point cloud, the second direction point cloud and the shelf model, obtain a second shelf center target pose; wherein the second shelf center target pose is used to generate a target position of the mobile robot.

[0043] Specifically, based on the first direction point cloud, the second direction point cloud and the shelf model, a second shelf center target pose for generating a target position of the mobile robot is obtained.

[0044] In one application, the first direction point cloud and the second direction point cloud are used as observation point clouds, and non-linear optimization is performed based on these observation point clouds in combination with the shelf model to obtain the target position of the center of the second shelf.

[0045] In another application, the points in the first direction point cloud and the second direction point cloud are fitted to a plane, and then the intersection of the normal vector of the plane and the shelf leg point cloud is calculated, and the target position of the center of the second shelf is obtained in combination with the shelf model.

[0046] In the above scheme, the electronic device obtains the point cloud data scanned by the mobile robot, determines the shelf leg point cloud and the shelf model matching the shelf leg point cloud based on the point cloud data, obtains the first shelf center target pose based on the shelf leg point cloud and the shelf model matching it, classifies the shelf leg point cloud based on the first shelf center target pose and the shelf model, obtains the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud, obtains the second shelf center target pose for generating the target position of the mobile robot based on the first direction point cloud, the second direction point cloud and the shelf model, classifies the shelf leg point cloud, and performs constraint calculation using the classified point cloud to make the second shelf center target pose more accurate, thereby being able to detect the accuracy of the shelf center position.

[0047] In one embodiment, see Figure 6 , Figure 6 This application Figure 4 Step S402 in the flowchart corresponds to an implementation method. Step S402 specifically includes:

[0048] S601: Filter the shelf leg point cloud based on the shelf size corresponding to the shelf model.

[0049] Specifically, based on the shelf size corresponding to the shelf model, the point cloud of each cluster of candidate shelf legs is filtered.

[0050] In one implementation scenario, the cluster center of each cluster of shelf leg point clouds is taken as the center of the circle, the length of the hypotenuse of the rectangular shelf leg is taken as the diameter, and the point clouds falling outside the circle are considered to be breakfast and need to be filtered out.

[0051] S602: Based on the filtered shelf leg point cloud, obtain the first shelf leg pose and the first shelf center initial pose.

[0052] Specifically, based on the filtered shelf leg point cloud, the first shelf leg pose and the first shelf center initial pose are obtained.

[0053] In an implementation scenario, based on the filtered shelf leg point cloud, the first shelf leg pose of each cluster of shelf leg point clouds is roughly determined, the mean center coordinates m1, m2, m3, and m4 of the four shelf leg point clouds are obtained, and the initial pose C1 of the first shelf center is calculated. The specific calculation process is as follows:

[0054] C1=(m1+m2+m3+m4) / 4(1)

[0055] S603: Obtain a target pose of the center of the first shelf based on the pose of the legs of the first shelf, the initial pose of the center of the first shelf, and the shelf size corresponding to the shelf model.

[0056] Specifically, based on the first shelf leg posture, the first shelf center initial posture and the shelf size corresponding to the shelf model, the first shelf center target posture is calculated, and the accuracy of the first shelf center target posture is improved by calculating through the filtered shelf leg point cloud.

[0057] In one implementation scenario, see Figure 7 , Figure 7 This is a schematic diagram of calculating the coordinate pose of the center of the shelf provided in this application. The error equation is constructed using the pose of the first shelf leg, and the target pose of the center of the first shelf is obtained by nonlinear optimization solution in combination with the initial pose of the center of the first shelf and the shelf size corresponding to the shelf model, wherein the target pose of the center of the first shelf is a roughly calculated pose of the center of the shelf. The specific calculation process is as follows:

[0058]

[0059]

[0060] ε i =ε′ i +ε″ i +ε″′ i (5)

[0061] Among them, d and w are the depth and width of the shelf size corresponding to the shelf model, ω′ i ,ω″ i ,ω″′ i are the weights of each shelf leg point cloud in the residual term, p i (x), p i (y) are the x and y axis coordinate values ​​of each shelf leg point cloud in the vehicle coordinate system, C(x), C(y), and C(θ) are the target poses of the center of the first shelf, which are the coordinates of the center of the shelf and the angular orientation of the shelf, respectively.

[0062] See also Figure 8 , Figure 8 This application Figure 4 Step S403 in the flowchart corresponds to an implementation method. Step S403 specifically includes:

[0063] S801: Based on the first shelf center target pose, obtain a set of unit feature vector bases.

[0064] Specifically, based on the target pose at the center of the first shelf, a set of unit feature vector bases {β1, β2} are constructed, where β1 is the main direction vector and β2 is β1 rotated 90° counterclockwise, forming a right-handed coordinate system.

[0065] S802: Obtain the mean center point of the shelf leg point cloud, classify the shelf leg point cloud based on the mean center point, the unit eigenvector basis and the shelf model, and obtain the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud.

[0066] Specifically, the mean center point of each cluster of shelf leg point clouds is obtained, and the shelf leg point clouds are classified based on the mean center point of each cluster of shelf leg point clouds, a set of constructed unit eigenvector bases and the shelf model to obtain the first direction point cloud and the second direction point cloud corresponding to the shelf leg point cloud. By reducing the dimensionality of the shelf leg point cloud, the quality of the point cloud data can be improved, thereby improving the accuracy of point cloud classification.

[0067] In one embodiment, in step S802, the shelf leg point cloud is classified based on the mean center point, the unit eigenvector basis and the shelf model, specifically including: constructing a sample matrix based on the mean center point and the first shelf leg posture, projecting the sample matrix to the unit eigenvector basis, and obtaining a one-dimensional vector group corresponding to the shelf leg point cloud; classifying the shelf leg point cloud based on the mean center point, the one-dimensional vector group corresponding to the shelf leg point cloud and the shelf model.

[0068] Specifically, for a cluster of candidate shelf leg point clouds Z, whose mean center is m, construct the mean-removed sample matrix The sample matrix is ​​projected onto two eigenvectors β1 and β2 to achieve dimensionality reduction, and the corresponding one-dimensional vector groups α1 and α2 of the shelf leg point cloud after dimensionality reduction are obtained. The shelf leg point cloud is classified based on the mean center point m of the shelf leg point cloud, the one-dimensional vector groups α1 and α2 corresponding to the shelf leg point cloud, and the shelf model. Among them, the mobile robot has a first pose in the coordinate system of the mean center point m of the shelf leg point cloud. The first pose and the one-dimensional vector groups α1 and α2 are used to determine the coordinate position of the shelf leg vertex in the coordinate system of the mean center point m of the shelf leg point cloud, and then the second pose of the mobile robot in the coordinate system of the shelf leg vertex can be determined. The second pose and the shelf model are used to determine the coordinate position of the shelf leg center in the coordinate system of the shelf leg vertex, so that the shelf leg point cloud can be classified according to the coordinate position of the shelf leg center, the shelf leg vertex, and the shelf leg point cloud.

[0069] In one implementation scenario, the steps of classifying the shelf leg point cloud based on the mean center point, the one-dimensional vector group corresponding to the shelf leg point cloud and the shelf model specifically include: obtaining the first pose of the laser sensor on the mobile robot relative to the mean center point; wherein the laser sensor is used to obtain point cloud data, and based on the first pose and the one-dimensional vector group corresponding to the shelf leg point cloud, determine the coordinate position of the shelf leg vertex in the shelf leg point cloud relative to the mean center point; obtain the second pose of the laser sensor relative to the shelf leg vertex; based on the second pose and the shelf leg size corresponding to the shelf model, obtain the coordinate position of the shelf leg center relative to the shelf leg vertex; based on the coordinate position of the shelf leg center relative to the shelf leg vertex and the coordinate position of the shelf leg point cloud relative to the shelf leg vertex, classify the shelf leg point cloud.

[0070] Specifically, the laser sensor has a first pose relative to the mean center point of the shelf leg point cloud. The coordinate positions of the shelf leg vertices in the shelf leg point cloud relative to the mean center point are determined through the first pose and the one-dimensional vector group corresponding to the shelf leg point cloud, thereby obtaining a second pose of the laser sensor relative to the shelf leg vertices. The coordinate positions of the shelf leg center relative to the shelf leg vertices are obtained through the second pose and the shelf leg size corresponding to the shelf model. The shelf leg point cloud is then classified based on the coordinate positions of the shelf leg center relative to the shelf leg vertices and the coordinate positions of the shelf leg point cloud relative to the shelf leg vertices.

[0071] In a specific implementation scenario, please refer to Fig. 9 , Fig. 9 : is a schematic diagram of the classification of the shelf leg point cloud provided by the present application. The shelf leg point cloud is an L-shaped point cloud. According to the first position T of the laser sensor in the coordinate system of the mean center point m of the shelf leg point cloud (in the same direction as {β1, β2}), L m , calculate the coordinates v(x, y) of the shelf leg vertex v in the shelf leg point cloud in the coordinate system of the mean center point m (that is, T v m ), the calculation process is as follows:

[0072]

[0073] Among them, v1 is the coordinate value corresponding to the one-dimensional vector α1 after the dimensionality reduction of the shelf leg point cloud, and v2 is the coordinate value corresponding to the one-dimensional vector α2 after the dimensionality reduction of the shelf leg point cloud.

[0074] Furthermore, according to the second position T of the laser sensor in the coordinate system of the shelf leg vertex v (in the same direction as {β1, β2}), L v , combined with the shelf leg size corresponding to the shelf model, including the shelf leg width w leg and shelf leg depth d leg, calculate the coordinates c(x, y) of the shelf leg center c in the coordinate system of the shelf leg vertex v (that is, T c v ), the calculation process is as follows:

[0075]

[0076] Furthermore, according to the position T of the shelf leg center c in the coordinate system of the shelf leg vertex v c v , segment the L-shaped point cloud of the shelf leg, the segmentation line is the straight line formed by point c and point v, and classify the points in the shelf leg point cloud Z. The specific process is as follows:

[0077]

[0078] Among them, p i ∈Z means p i is any point in the point cloud of the shelf leg; represents the vector from point c to point v, From point c to point p i It should be noted that the vector This is to take into account that when the laser sensor observes the shelf legs from different angles, The horizontal and vertical sides corresponding to the directions are different. The classified shelf leg point clouds belong to the vertical side (class=1), i.e., the first direction point cloud, and the horizontal side (class=2), i.e., the second direction point cloud, of the L-shaped point cloud.

[0079] Optionally, the point cloud near the shelf leg vertex v may not be accurately classified due to fluctuations and noise, and can be discarded. Therefore, in order to reduce interference, the point cloud within a certain threshold range of the shelf leg vertex v can be filtered, thereby further improving the accuracy of the subsequent calculated second shelf center target pose.

[0080] In one embodiment, step S404 specifically includes: obtaining the second shelf leg posture and the second shelf center initial posture based on the first direction point cloud, the second direction point cloud and the shelf leg size corresponding to the shelf model; obtaining the second shelf center target posture based on the second shelf leg posture, the second shelf center initial posture and the shelf size corresponding to the shelf model.

[0081] Specifically, based on the classified first-direction point cloud, second-direction point cloud and the shelf leg size corresponding to the shelf model, the second shelf leg pose and the second shelf center initial pose are obtained; based on the second shelf leg pose, the second shelf center initial pose and the shelf size corresponding to the shelf model, the second shelf center target pose is calculated.

[0082] In one implementation scenario, the classified first direction point cloud and second direction point cloud are used as observations, combined with the shelf leg size corresponding to the shelf model, to construct an error equation, and the center coordinates c(x', y') of the shelf leg are obtained through nonlinear optimization, and the angle direction C(θ) is set as a constant. The specific calculation process is as follows:

[0083]

[0084] Among them, p i ∈(class=1 / 2) means p i Belongs to a category in the first direction point cloud or the second direction point cloud, d leg and w leg are the shelf leg depth and shelf leg width, p i (x) and p i (y) represents the coordinate value of each shelf leg point cloud, c(x′), c(y′) and C(θ) are the coordinates of the center of the shelf leg and the angular orientation of the shelf respectively (the default angular orientation of the shelf leg is the same as the angular orientation of the shelf).

[0085] Furthermore, through the optimized shelf leg center coordinates, the second shelf leg postures corresponding to the four shelf leg point clouds are obtained in turn, namely p1(c1(x), c1(x), C(θ)), p2(c2(x), c2(x), C(θ)), p3(c3(x), c3(x), C(θ)), p4(c4(x), c4(x), C(θ)), then the initial posture of the second shelf center is C2 = (p1+p2+p3+p4 / 4). The error equation is then constructed using the second shelf leg posture, and combined with the initial posture of the second shelf center and the shelf size corresponding to the shelf model, the target posture of the second shelf center is obtained through nonlinear optimization. The specific calculation process can be found in formulas (2) to (4), which will not be repeated here.

[0086] The above scheme segments and classifies the point clouds on the two sides of the L-shaped point cloud, and constructs residuals for the observations of the two types of point clouds respectively, so that the constraints are more accurate, thereby improving the accuracy of the calculated target position of the center of the second shelf.

[0087] In one embodiment, after step S404, it also includes: controlling the mobile robot to move to the target position, using the mobile robot to scan at the target position to obtain the current position of the mobile robot; obtaining a relative position based on the target position and the current position; and determining that the mobile robot is located at the target position based on the relative position.

[0088] Specifically, after obtaining the target position of the center of the second shelf, the target position of the mobile robot is generated, the mobile robot is controlled to move to the target position, and the laser sensor on the mobile robot is used to perform a second shelf recognition at the target position to obtain the current position of the mobile robot. Based on the target position and the current position, the relative position of the current position relative to the target position is obtained, and the relative position is used to determine whether the mobile robot has reached the target position. Through the laser secondary verification method, after the mobile robot reaches the target position, there is no need to add other sensors or other camera equipment for inspection, which effectively reduces the cost of use and the difficulty of implementation.

[0089] In one implementation scenario, based on the relative position, the step of determining whether the mobile robot is located at the target position specifically includes: judging whether the position parameters and angle parameters in the relative position meet the preset threshold conditions; if so, determining that the mobile robot is located at the target position.

[0090] Specifically, the relative position includes multiple parameters, namely position parameters and angle parameters. When the position parameters and angle parameters meet the preset threshold conditions, it is determined that the mobile robot has successfully reached the target position and the next action can be executed. Otherwise, check whether the status of the mobile robot is abnormal.

[0091] In a specific implementation scenario, the target position is denoted by T t S , the current position of the mobile robot is recorded as T S B , then the relative position T B t =(T t S ) -1 *(T S B ) -1 , when the relative position T B t When the position parameters (x, y) and angle parameters yaw in are both less than the threshold, it is considered that the mobile robot has successfully reached the corresponding target position directly below the shelf.

[0092] See also Fig.10 , Fig.101 is a schematic diagram of the structure of an embodiment of an electronic device of the present application, wherein the electronic device 100 comprises a memory 1000 and a processor 1002 coupled to each other, wherein the memory 1000 stores program data (not shown), and the processor 1002 calls the program data to implement the method in any of the above embodiments, and the description of the relevant contents can be found in the detailed description of the above method embodiments, which will not be repeated here. Specifically, the electronic device 100 includes but is not limited to: a desktop computer, a laptop computer, a tablet computer, a server, etc., which are not limited here. In addition, the processor 1002 can also be referred to as a CPU (Center Processing Unit). The processor 1002 may be an integrated circuit chip with signal processing capabilities. The processor 1002 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 1002 may be implemented by an integrated circuit chip.

[0093] See also Fig.11 , Fig.11 It is a structural diagram of an implementation method of a computer-readable storage medium of the present application. The computer-readable storage medium 110 stores program data 1100. When the program data 1100 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0094] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0097] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A shelf detection method, characterized in that: include: Acquire point cloud data scanned by a mobile robot, and determine a shelf leg point cloud and a matching shelf model based on the point cloud data; Based on the shelf leg point cloud and the shelf model, obtaining a first shelf center target pose; Based on the first shelf center target pose and the shelf model, the shelf leg point cloud is classified to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud; Based on the first direction point cloud, the second direction point cloud and the shelf model, a second shelf center target pose is obtained; wherein the second shelf center target pose is used to generate a target position of the mobile robot.

2. The method according to claim 1, characterized in that The step of obtaining a first shelf center target pose based on the shelf leg point cloud and the shelf model includes: Filtering the shelf leg point cloud based on the shelf size corresponding to the shelf model; Based on the filtered shelf leg point cloud, obtaining the first shelf leg pose and the first shelf center initial pose; The first shelf center target posture is obtained based on the first shelf leg posture, the first shelf center initial posture and the shelf size corresponding to the shelf model.

3. The method according to claim 1, characterized in that The step of classifying the shelf leg point cloud based on the first shelf center target pose and the shelf model to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud includes: Based on the first shelf center target pose, a set of unit feature vector bases are obtained; The mean center point of the shelf leg point cloud is obtained, and based on the mean center point, the unit eigenvector basis and the shelf model, the shelf leg point cloud is classified to obtain a first direction point cloud and a second direction point cloud corresponding to the shelf leg point cloud.

4. The method according to claim 3, characterized in that The classifying the shelf leg point cloud based on the mean center point, the unit feature vector basis and the shelf model comprises: Based on the mean center point and the first shelf leg pose corresponding to the shelf leg point cloud, construct a sample matrix, project the sample matrix to the unit eigenvector basis, and obtain a one-dimensional vector group corresponding to the shelf leg point cloud; The shelf leg point cloud is classified based on the mean center point, the one-dimensional vector group corresponding to the shelf leg point cloud and the shelf model.

5. The method according to claim 4, characterized in that The classifying the shelf leg point cloud based on the mean center point, the one-dimensional vector group corresponding to the shelf leg point cloud and the shelf model includes: Obtaining the first position of the laser sensor on the mobile robot relative to the mean center point; wherein the laser sensor is used to obtain the point cloud data; Determine the coordinate positions of the shelf leg vertices in the shelf leg point cloud relative to the mean center point based on the first posture and the one-dimensional vector group corresponding to the shelf leg point cloud; Acquire a second position of the laser sensor relative to the vertex of the shelf leg; Based on the second posture and the shelf leg size corresponding to the shelf model, obtaining the coordinate position of the center of the shelf leg relative to the vertex of the shelf leg; The shelf leg point cloud is classified based on the coordinate position of the shelf leg center relative to the shelf leg vertex and the coordinate position of the shelf leg point cloud relative to the shelf leg vertex.

6. The method according to claim 1, characterized in that The obtaining the second shelf center target pose based on the first direction point cloud, the second direction point cloud and the shelf model includes: Based on the first direction point cloud, the second direction point cloud and the shelf leg size corresponding to the shelf model, obtain the second shelf leg posture and the second shelf center initial posture; Based on the second shelf leg posture, the second shelf center initial posture and the shelf size corresponding to the shelf model, the second shelf center target posture is obtained.

7. The method according to claim 1, characterized in that After obtaining the second shelf center target pose based on the first direction point cloud and the second direction point cloud, the method further includes: Controlling the mobile robot to move to the target position, and using the mobile robot to scan at the target position to obtain the current position of the mobile robot; Based on the target position and the current position, a relative position is obtained; Based on the relative position, it is determined that the mobile robot is located at the target position.

8. The method according to claim 7, characterized in that The step of determining that the mobile robot is located at the target position based on the relative position includes: It is determined whether the position parameter and the angle parameter in the relative position meet a preset threshold condition. If so, it is determined that the mobile robot is located at the target position.

9. An electronic device, characterized in that: The method comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the method according to any one of claims 1 to 8.

Images