Shelf identification method based on laser radar and related equipment
Through single-line lidar combined with data processing of shelf parameters, the problem of cumbersome and high cost of shelf identification in the existing technology is solved, and the path planning of AGV accurately enters the shelf center is realized, reducing the complexity of equipment and calculations.
Patent Information
- Application Number
- CN202510600977.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-26
AI Technical Summary
When identifying shelves, especially in large industrial sites, pasting reflective strips or QR codes is cumbersome and costly, while using depth cameras or binocular cameras is expensive and has high computing complexity, resulting in the AGVs being unable to accurately enter the center of the shelf or collide.
Single-line lidar scans the shelves, and data segmentation processing and screening are performed by obtaining distance values and combining shelf parameters, determine the position information and leg positions of the shelf, and realize path planning, avoiding identification pasting and complex algorithms.
With no identification or auxiliary information, accurate access to the center of the bottom of the shelf reduces costs and simplifies the identification process.
Smart Images
Figure CN120539698A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of laser radar data processing technology, and in particular to a shelf recognition method and related equipment based on laser radar. Background Art
[0002] With the continuous improvement of industrial automation, more and more industrial manufacturing companies are gradually introducing autonomous mobile equipment such as automated guided vehicles (AGVs) to complete transportation tasks. In actual production, AGVs need to follow a specified movement path to a specified location to complete the pickup operation. However, when the placement of the goods is offset or the angle is offset, the specified path does not change with the offset position or angle. This may cause the AGV to be unable to accurately enter the center of the bottom of the shelf, or even cause the AGV to collide with the shelf.
[0003] In related technologies, shelf recognition and terminal path replanning are usually used to control the AGV to enter the center position of the bottom of the shelf more accurately. Common shelf recognition solutions are mainly the following:
[0004] (1) Reflective strips are attached to the legs of the shelves. The reflection intensity of the laser radar on the reflective strips is much higher than that on other areas, so the position information of the center of the shelf is determined.
[0005] (2) Paste a QR code on the shelf leg, collect image data with the help of a camera, and combine it with image processing technology to determine the position information of the shelf center.
[0006] (3) Install a depth camera or binocular camera on the AVG to collect images or videos, and combine it with deep learning and other related algorithms to determine the position information of the center of the shelf.
[0007] However, while reflective strips or QR codes offer lower equipment costs, they require labeling each shelf, making this a cumbersome task when dealing with numerous shelves in large industrial sites. Depth cameras or binocular cameras, while capable of capturing richer environmental information, are more expensive, and the recognition algorithms are more complex and computationally intensive. Summary of the Invention
[0008] The embodiments of the present application provide a shelf recognition method and related equipment based on laser radar, which is used to apply the calculated shelf posture information and shelf leg position information without the aid of any identification or auxiliary information, thereby ensuring that the mobile main device accurately enters the shelf area, which is simple to implement and saves costs.
[0009] In a first aspect, an embodiment of the present application provides a shelf recognition method based on laser radar, wherein each shelf is composed of four shelf legs and a shelf surface, and the four shelf legs form a rectangle. The method includes:
[0010] Acquire multiple distance values obtained when a laser radar scans at least one object in a scanning area; wherein the object is a shelf leg or other object; the multiple distance values correspond to multiple scanning angles, and the distance values and the scanning angles correspond one to one;
[0011] Segmenting the plurality of distance values to determine a data segment for each object; wherein a data segment is a data set including at least one distance value for an object;
[0012] Filtering the plurality of data segments based on shelf parameters to determine a plurality of data segment pairs; wherein each data segment pair includes at least three data segments respectively representing shelf legs belonging to the same shelf;
[0013] Among multiple data segment pairs, the target data segment pair closest to the laser radar is selected, and the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf are calculated based on the target data segment pair; wherein the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf instruct the mobile device where the laser radar is located to perform path planning.
[0014] In a second aspect, an embodiment of the present application provides a shelf identification device based on laser radar, wherein each shelf is composed of four shelf legs and a shelf surface, and the four shelf legs form a rectangle. The device includes:
[0015] A data acquisition unit is configured to: acquire a plurality of distance values obtained when the laser radar scans at least one object in the scanning area; wherein the object is a shelf leg or other object; the plurality of distance values correspond to a plurality of scanning angles, and the distance values and the scanning angles correspond one to one;
[0016] A data processing unit, configured to: segment the plurality of distance values to determine a data segment for each object; wherein a data segment is a data set including at least one distance value for an object;
[0017] The shelf identification unit is used to: screen the plurality of data segments based on the shelf parameters of the shelf to determine a plurality of data segment pairs; wherein each data segment pair includes at least three data segments respectively representing shelf legs belonging to the same shelf;
[0018] The shelf identification unit is also used to: select the target data segment pair closest to the laser radar from multiple data segment pairs, and calculate the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf based on the target data segment pair; wherein the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf instruct the mobile device where the laser radar is located to perform path planning.
[0019] In a third aspect, an embodiment of the present application provides a mobile device, including a laser radar, a memory, and a processor, wherein the laser radar executes the step of obtaining multiple distance values when scanning at least one object in the scanning area, the memory stores a computer program that can be run on the processor, and the processor implements the steps of any of the above-mentioned shelf recognition methods when executing the computer program.
[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above-mentioned shelf identification methods when executed by a processor.
[0021] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned shelf identification methods when executed by a processor.
[0022] In an embodiment of the present application, a single-line laser radar is used to scan at least one object in a scanning area to obtain multiple distance values. The multiple distance values are then segmented to determine a data segment for each object. A data segment is a data set that includes at least one distance value for the object, thereby separating the distance values of different objects. The multiple data segments are then filtered based on the shelf parameters to determine multiple data segment pairs, each of which includes at least three data segments representing shelf legs belonging to the same shelf. In this way, the distance values belonging to the same shelf can be filtered out. From the multiple data segment pairs obtained, the target data segment pair closest to the laser radar is selected to determine the target shelf. The target shelf's position information and the position information of the four shelf legs that constitute the target shelf are then calculated based on the target data segment pair. The target shelf's position information and the position information of the four shelf legs that constitute the target shelf instruct the mobile device where the laser radar is located to perform path planning. This design, without the aid of any identification or auxiliary information, uses the calculated shelf's position information and shelf leg position information to ensure that the mobile host device accurately enters the center position at the bottom of the shelf, saving costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0024] Figure 1a A schematic diagram of an application scenario of a shelf recognition method based on laser radar provided in an embodiment of the present application;
[0025] Figure 1b A schematic diagram of a cross section of a shelf leg provided in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of a process flow of a shelf recognition method based on laser radar provided in one embodiment of the present application;
[0027] Figure 3 A schematic diagram of calculating the center position of the bottom of a shelf provided in one embodiment of the present application;
[0028] Figure 4 A schematic diagram of calculating the center of mass of the fourth shelf leg provided in one embodiment of the present application;
[0029] Figure 5 A schematic structural diagram of a shelf recognition device based on laser radar provided in one embodiment of the present application;
[0030] Figure 6 A schematic structural diagram of a mobile device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0032] Any number of elements in the drawings is for illustration and not limitation, and any naming is for distinction only and does not have any limiting meaning.
[0033] In practice, while reflective strips or QR codes offer lower equipment costs, they require labeling each shelf, making this a cumbersome task when dealing with numerous shelves in large industrial sites. Depth cameras or binocular cameras, while capable of capturing richer environmental information, are more expensive, and the recognition algorithms are more complex and computationally intensive.
[0034] To this end, this application provides a method for identifying shelves using a single-line laser radar. In this method, the distance value collected by the single-line laser radar (the distance measurement between the laser radar emission point and the nearest detected obstacle) is combined with the dimensional parameters and geometric relationships of the shelf legs to determine the position information of the shelf center. This position information instructs the mobile device equipped with the laser radar to perform path planning so that the mobile device can accurately reach the center of the bottom of the shelf for cargo handling, which is low-cost.
[0035] After introducing the design concepts of the embodiments of this application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of this application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of this application and are not limiting. In specific implementations, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.
[0036] refer to Figure 1a , which is a schematic diagram of an application scenario of a shelf recognition method based on laser radar provided in an embodiment of the present application. In which, each shelf is composed of four shelf legs and a shelf surface, and the four shelf legs form a rectangle. Figure 1a In the figure, points A, B, C, and D are the horizontal mapping points of the center of mass of the four rack legs, and point O is the horizontal mapping point of the rack center. Rack parameters include shelf length, shelf width, and the length of the shelf diagonal. The shelf diagonal is formed by connecting the horizontal mapping points of two non-adjacent rack legs.
[0037] In addition, the shelf parameters also include the length of the diagonal of the shelf legs, which can be used to perform operations such as filtering distance values. Figure 1b A schematic diagram of a cross-section of a shelf leg is provided in an embodiment of the present application. In this diagram, the cross-section of the shelf leg is a square. a, b, c, and d are the projections of the four vertices of shelf leg A onto a horizontal plane. ab is the side length of the square, and ac is the length of the shelf leg diagonal. In this embodiment of the present application, a distinction is made between the shelf diagonal and the shelf leg diagonal. In actual applications, the cross-section of the shelf leg may also be a rectangle. This is merely an example and does not constitute a specific limitation.
[0038] In the embodiment of the present application, a single-line laser radar is used to reduce structural complexity and reduce costs. The laser radar can scan in the horizontal plane, and the area to be scanned may include shelves or other objects. During the scanning process, a distance value in each ranging data corresponds to a scanning angle. The scanning angle and scanning angle range here can be determined according to the relevant parameters of the laser radar to ensure that all shelves in the area to be scanned can be scanned. The ranging data obtained for each scan may include N distance values under N scanning angles. The embodiment of the present application is the data processing process during one scanning process. The data processing process in other scanning processes is the same and will not be repeated here.
[0039] Of course, the method provided in the embodiment of the present application is not limited to Figure 1a The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1a The functions that can be implemented by each device in the application scenario shown will be described in subsequent method embodiments and will not be described in detail here.
[0040] To further illustrate the technical solutions provided by the embodiments of the present application, the following is a detailed description of the technical solutions in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present application provide the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative work. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application.
[0041] The following combination Figure 1a The application scenario shown illustrates the technical solution provided by the embodiment of this application.
[0042] refer to Figure 2 The embodiment of the present application provides a shelf recognition method based on laser radar, comprising the following steps:
[0043] S201: Acquire multiple distance values obtained when the laser radar scans at least one object in the scanning area.
[0044] S202: Segment the multiple distance values to determine a data segment for each object.
[0045] S203: Filtering the plurality of data segments based on the shelf parameters of the shelf to determine a plurality of data segment pairs.
[0046] S204: Select a target data segment pair closest to the laser radar from among multiple data segment pairs, and calculate the pose information of the target shelf and the position information of the four shelf legs constituting the target shelf based on the target data segment pair.
[0047] In an embodiment of the present application, a single-line laser radar is used to scan at least one object in a scanning area to obtain multiple distance values. The multiple distance values are then segmented to determine a data segment for each object. A data segment is a data set that includes at least one distance value for the object, thereby separating the distance values of different objects. The multiple data segments are then filtered based on the shelf parameters to determine multiple data segment pairs, each of which includes at least three data segments representing shelf legs belonging to the same shelf. In this way, the distance values belonging to the same shelf can be filtered out. From the multiple data segment pairs obtained, the target data segment pair closest to the laser radar is selected to determine the target shelf. The target shelf's position information and the position information of the four shelf legs that constitute the target shelf are then calculated based on the target data segment pair. The target shelf's position information and the position information of the four shelf legs that constitute the target shelf instruct the mobile device where the laser radar is located to perform path planning. This design, without the aid of any identification or auxiliary information, uses the calculated shelf's position information and shelf leg position information to ensure that the mobile host device accurately enters the center position at the bottom of the shelf, saving costs.
[0048] Regarding S201, the scanning angle interval can be determined based on the relevant parameters of the laser radar, so that multiple distance values obtained when the laser radar scans at least one object in the scanning area can be obtained during each scanning process.
[0049] After obtaining multiple distance values, these distance values can also be optimized. The optimization process can be achieved through steps A1-A2:
[0050] A1: Filter out some distance values that meet the set distance conditions from multiple distance values.
[0051] Optionally, considering that the detection distance of the laser radar (for example, 0.1m to 10m or 0.1m to 30m) is much greater than the distance between the laser radar and the shelf during identification (for example, 0.1m to 5m), in order to reduce the amount of calculation and interference, a set distance condition can be determined based on the detection distance of the laser radar, for example, a distance value within 0.1m to 5m is considered to meet the set distance condition. In this way, at least one distance value that meets the set distance condition can be screened out, and distance values that do not meet the set distance condition can be set to zero.
[0052] Such a design can reduce the amount of calculation and interference.
[0053] A2: Filter the filtered distance values.
[0054] Among them, among the multiple distance values after screening, the distance values that do not meet the set distance conditions are set to zero. In the embodiment of the present application, the multiple distance values after screening can be filtered, for example, by using median filtering.
[0055] Such a design can reduce interference such as outliers and noise points.
[0056] Regarding S202, considering that the distance values of the same object are roughly the same during the scanning process of the laser radar, the mutation data in the distance value (the distance value of the step point or mutation point) can be filtered out, and then the distance value can be segmented according to the mutation data to separate the data segments of different objects.
[0057] Optionally, the mutation data of the mutation point and the data segment of each object can be determined through steps B1-B2:
[0058] B1: Determine multiple mutation data among multiple distance values.
[0059] The mutation data corresponds to a first scanning angle, and the absolute value of the difference between the distance value corresponding to the second scanning angle adjacent to the first scanning angle and the mutation data is greater than a first length threshold. Optionally, the first length threshold is determined based on the size of a shelf leg, for example, the length of the diagonal of the shelf leg, such as Figure 1b The length of ac in .
[0060] B2: For each mutation data in the plurality of mutation data, determine whether the mutation data and the distance value between the mutation data and the previous mutation data constitute a data segment of an object.
[0061] Optionally, if there is no other mutation data before the mutation data, the mutation data and the distance value before it constitute a data segment of an object; if there is other mutation data before the mutation data, the distance value between the mutation data and the previous mutation data constitutes a data segment of an object; if there is no other mutation data after it, the mutation data and the distance value after it constitute a data segment of an object.
[0062] This design allows for the determination of a data segment for each object, where each data segment is a data set containing at least one distance value for the object. In other words, a set of distance values corresponding to each object can be determined. The objects here can be shelf legs or other objects.
[0063] After obtaining the data segments of each object, you can also filter the data segments of each object through steps C1-C3:
[0064] C1: For each object, determine a first number of distance values included in the data segment of the object.
[0065] Considering that the distance value is expressed in polar coordinates, the origin of the polar coordinates is the position of the laser radar, and the polar axis can be set according to relevant parameters of the laser radar and is not limited here. Taking an object as an example, the first number of distance values included in the data segment of the object is determined, and is represented by L.
[0066] C2: Calculate a reference arc length of the data segment of the object based on the first number, the arc resolution of the laser radar, and the average value of the distance values included in the data segment of the object.
[0067] Optionally, based on the first quantity and the arc resolution r of the lidar, the approximate arc of the data segment can be calculated as L*r, and combined with the average value d of the distance values included in the data segment, the reference arc length of the data segment can be calculated as L*r*d.
[0068] C3: Based on the reference arc length corresponding to each object, filter out data segments that are less than a second length threshold.
[0069] By applying the above method, the reference arc length corresponding to each data segment of each object can be determined. At this time, the data segments whose length is less than the second length threshold can be filtered out. The second length threshold here is determined according to the size of a shelf leg. The second length threshold can be twice the length of the diagonal of the shelf leg. Figure 1b As shown in FIG. , the second length threshold may be 2*ac.
[0070] In an embodiment of the present application, the size of a single shelf leg is taken into consideration, and the data segments can be further filtered. The filtered data segments can represent a single shelf leg or other objects with a similar size to the shelf leg, thereby avoiding interference from the distance values of other irrelevant objects, reducing the amount of data processing, and improving calculation accuracy.
[0071] In step S203 , multiple data segments are screened based on shelf parameters of the shelf to determine multiple data segment pairs.
[0072] The shelf parameters include shelf length, shelf width, and shelf diagonal length, and each data segment pair includes at least three data segments representing shelf legs belonging to the same shelf. Optionally, each data segment pair includes three data segments, and the three data segments represent three shelf legs.
[0073] In practice, considering occlusion during LiDAR scanning, it can be assumed that the shelf is identified if three data segments, corresponding to the length, width, and diagonal length of the shelf legs, are distributed in a right triangle. Thus, the three centroids of each selected centroid group satisfy the geometric relationship between the shelf legs and can represent the three legs of a single shelf.
[0074] Optionally, the process of determining multiple data segment pairs may be implemented through steps D1-D4:
[0075] D1: For each object, calculate the object's center of mass based on the distance value included in the data segment corresponding to the object.
[0076] Among them, considering the convenience of distance calculation and coordinate calculation, the distance value stored in the data segment in polar coordinate form can be converted into a rectangular coordinate system. The origin of the rectangular coordinate system here is the position of the laser radar, and the horizontal axis and the vertical axis can be determined according to the conversion relationship from the polar coordinate system to the rectangular coordinate system determined by the relevant parameters of the laser radar.
[0077] For each object, a clustering algorithm, such as a K-means clustering algorithm, can be used to calculate the center of mass of each data segment based on the distance value included in the data segment corresponding to the object in a rectangular coordinate system. The center of mass of the data segment is the center of mass of the object.
[0078] In this embodiment of the application, each data segment is clustered to obtain a centroid, which is used to represent the center of the object represented by the data segment. This provides more accurate coordinate parameters for subsequent calculations of distance relationships between objects (centroids) and perpendicular relationships between line segments connecting objects (centroids).
[0079] D2: Calculate the distance between any two centroids in the centroid of each object.
[0080] Among them, each object corresponds to a center of mass, so the center of mass distance between any two centers of mass can be calculated.
[0081] D3: Among the multiple centroid distances, determine multiple first data segment pairs based on the shelf length, the shelf width, and the length of the shelf diagonal.
[0082] Among them, each first data segment pair corresponds to a centroid group, which includes the centroids corresponding to three objects respectively. The lengths of the three line segments formed by the three centroids in the centroid group corresponding to the first data segment pair respectively meet the set length conditions with the shelf length, shelf width and shelf diagonal length.
[0083] Optionally, the length condition can be set such that the length of line segment 1 formed by centroids A and B is equal to the shelf width, the length of line segment 2 formed by centroids B and C is equal to the shelf length, and the length of line segment 3 formed by centroids B and C is equal to the length of the shelf diagonal. In actual applications, appropriate size tolerances can be set. For example, in setting the length condition, the tolerance for shelf length can be the shelf leg length, the tolerance for shelf width can be the shelf leg width, and the tolerance for shelf diagonal length can be the length of the shelf leg diagonal.
[0084] The first data segment pair selected in this way satisfies the condition of forming the shelf legs of the shelf in terms of length.
[0085] D4: Among the plurality of first data segment pairs, select first data segment pairs whose included angles meet a set angle condition to form a plurality of data segment pairs.
[0086] The included angle is the angle between two shorter line segments among three line segments calculated from the three centroids in the centroid group corresponding to the first data segment.
[0087] Optionally, to further determine whether the shelf legs corresponding to the three centroids constitute a single shelf, an angle condition may be set. The angle condition may be that the angle between the two shorter line segments of the three line segments calculated for the three centroids in the corresponding centroid group by the first data segment is 90 degrees. In actual applications, a suitable angle tolerance may be set, such as 3°.
[0088] In an embodiment of the present application, the geometric relationship between the shelf legs is comprehensively considered to filter out data segment pairs that meet the conditions. The filtered data segment pairs include three data segments, representing the three shelf legs of a single shelf.
[0089] Regarding S204, among multiple data segment pairs, theoretically, the three shelf legs corresponding to each data segment pair can constitute a shelf. To more accurately determine the target shelf, factors such as distance from the LiDAR can be considered. The target data segment pair closest to the LiDAR can be selected from the multiple data segment pairs. Based on the target data segment pair, the target shelf's pose information and the position information of the four shelf legs that constitute the target shelf are calculated. The target shelf's pose information includes both its position and posture information.
[0090] Optionally, the target data pair can be determined through steps E1-E2:
[0091] E1: Calculate the first center point corresponding to each data segment pair respectively.
[0092] Among them, the first center point is the midpoint of the longest line segment formed by the three centroids in the centroid group corresponding to the data segment (the projections on the horizontal plane are A, B, and C respectively), that is, the midpoint of the hypotenuse (AC) of the right triangle. Figure 3 A schematic diagram of calculating the center position of the bottom of a shelf provided in an embodiment of the present application, wherein point O is the first center point, that is, the center position of the bottom of the shelf.
[0093] E2: Determine the distance between each first center point and the laser radar, and the data segment pair corresponding to the first center point with the shortest distance is the target data segment pair.
[0094] Optionally, considering the situation in which multiple shelves are densely placed in actual scenarios, the data segment pair corresponding to the first center point closest to the location of the lidar is screened out as the target data segment pair, that is, the three shelf legs represented by the target data segment pair constitute the target shelf.
[0095] Optionally, the position information of the target shelf and the position information of the four shelf legs constituting the target shelf can be determined through steps F1-F3:
[0096] F1: Determine the first center point with the shortest distance to the LiDAR as the center position of the target shelf.
[0097] The center position of the target shelf is the location information of the target shelf, that is, the center of the bottom of the shelf, see Figure 3 Point O in .
[0098] F2: Determine the centroid of the fourth shelf leg of the target shelf; wherein the target shelf is composed of the centroids of the three shelf legs represented by the target data segment pairs.
[0099] The method for calculating the centroid of the fourth shelf leg can be based on the right triangle characteristics or the rectangle characteristics. In this design, the positions of the centroids of the four shelf legs are the position information of the four shelf legs of the target shelf. Figure 4 A schematic diagram of calculating the center of mass of the fourth shelf leg is provided in an embodiment of the present application, wherein the mapping point of the center of mass of the fourth shelf leg on the horizontal plane is represented by D.
[0100] F3: In the three line segments formed by the three centroids of the target data segment pair, the angle between the first direction and the second direction is determined as the posture information of the target shelf.
[0101] Among them, the first direction is the direction of the line connecting the midpoint of the target line segment and the center position of the target shelf, and the second direction is the polar axis direction of the polar coordinate system of the laser radar; the target line segment is the line segment formed by connecting the two centroids closest to the laser radar among the four centroids of the target shelf.
[0102] This design, after obtaining the target shelf's pose information and the positions of its four legs, can use this calibrated information to assist the mobile device, which uses the LiDAR, in path planning, ensuring it precisely reaches the center of the shelf's bottom according to the planned path. Even if the shelf is offset, the mobile device can still accurately reach the center of the shelf's bottom without the aid of any markings or auxiliary information, significantly saving costs.
[0103] In summary, the embodiments of the present application have the following beneficial effects:
[0104] First, by obtaining distance values through lidar and performing data processing, data filtering, data segmentation, etc., the amount of calculation can be reduced and interference such as outliers and noise points can be reduced.
[0105] Secondly, only the size parameters of a single shelf leg are considered to screen out potential data segments (each data segment represents a single shelf leg, or other interfering objects with a similar size to the shelf leg); the geometric relationship between the shelf legs is comprehensively considered to screen out potential data segment pairs (each data segment pair contains three data segments, representing the three shelf legs of a single shelf); in this way, when the scene information that can be obtained by a single-line lidar is relatively limited, the geometric features of the objects to be identified in the area to be identified can be fully utilized to assist target recognition.
[0106] Finally, the data segment pair closest to the LiDAR is identified, the position data for the fourth shelf leg is completed, and the pose information for the shelf center is calculated, ensuring accurate navigation of the mobile device based on the calibrated pose information. In this way, replacing the rectangle with a fitted right triangle and completing the final right triangle into a rectangle solves the problem of individual shelf legs being obscured during LiDAR scanning, while effectively reducing the complexity and computational effort of the method without affecting the recognition results.
[0107] like Figure 5 As shown, based on the same inventive concept as the above-mentioned shelf recognition method based on laser radar, an embodiment of the present application also provides a shelf recognition device based on laser radar, which includes a data acquisition unit 501, a data processing unit 502 and a shelf recognition unit 503.
[0108] The data acquisition unit 501 is configured to: acquire multiple distance values obtained when the laser radar scans at least one object in the scanning area; wherein the object is a shelf leg or other object; the multiple distance values correspond to multiple scanning angles, and the distance values and the scanning angles correspond one to one;
[0109] The data processing unit 502 is configured to segment the multiple distance values to determine a data segment for each object; wherein the multiple distance values constitute a piece of ranging data, and a data segment is a data set including at least one distance value of an object;
[0110] The shelf identification unit 503 is configured to screen the plurality of data segments based on the shelf parameters to determine a plurality of data segment pairs, wherein each data segment pair includes at least three data segments representing shelf legs belonging to the same shelf;
[0111] The shelf identification unit 503 is also used to: select the target data segment pair closest to the laser radar from multiple data segment pairs, and calculate the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf based on the target data segment pair; wherein the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf instruct the mobile device where the laser radar is located to perform path planning.
[0112] In an optional embodiment, the shelf identification unit 503 is specifically configured to:
[0113] Determining a plurality of mutation data from a plurality of distance values; wherein the mutation data corresponds to a first scanning angle, and an absolute value of a difference between a distance value corresponding to a second scanning angle adjacent to the first scanning angle and the mutation data is greater than a first length threshold; the first length threshold is determined based on the size of a shelf leg;
[0114] For each mutation data in the plurality of mutation data, it is determined that the mutation data and a distance value between the mutation data and the previous mutation data constitute a data segment of an object.
[0115] In an optional embodiment, the shelf parameters include shelf length, shelf width, and shelf diagonal length;
[0116] The shelf identification unit 503 is specifically used for:
[0117] For each object, calculate the object's center of mass based on the distance value included in the data segment corresponding to the object;
[0118] Calculate the distance between any two centroids in the centroids corresponding to each object;
[0119] Among the multiple centroid distances, multiple first data segment pairs are determined based on the shelf length, shelf width, and shelf diagonal length; wherein each first data segment pair corresponds to a centroid group, the centroid group including the centroids corresponding to three objects, and the lengths of three line segments formed by the three centroids in the centroid group corresponding to the first data segment pair respectively satisfy set length conditions with the shelf length, shelf width, and shelf diagonal length;
[0120] Among multiple first data segment pairs, first data segment pairs whose included angles satisfy a set angle condition are screened to form multiple data segment pairs; wherein the included angle is the angle between two shorter line segments of three line segments calculated from three centroids in a centroid group corresponding to the first data segment pairs.
[0121] In an optional embodiment, the shelf identification unit 503 is further configured to:
[0122] Calculate the first center point corresponding to each data segment pair respectively; wherein the first center point is the midpoint of the longest line segment formed by the three centroids in the centroid group corresponding to the data segment;
[0123] When determining the distances between each first center point and the laser radar, the data segment pair corresponding to the first center point with the shortest distance is the target data segment pair.
[0124] In an optional embodiment, the position information of the target shelf includes the position information and posture information of the target shelf, and the shelf identification unit 503 is specifically configured to:
[0125] Determine the first center point with the shortest distance from the laser radar as the center of the target shelf; wherein the position information of the center of the target shelf is the position information of the target shelf;
[0126] Determine the center of mass of each of the three shelf legs represented by the target data segment and calculate the center of mass of the fourth shelf leg constituting the target shelf; wherein the positions of the center of mass of the four shelf legs are the position information of the four shelf legs of the target shelf;
[0127] In the three line segments formed by the three centroids of the target data segment pair, the angle between the first direction and the second direction is determined as the posture information of the target shelf; wherein the first direction is the direction of the line connecting the midpoint of the target line segment and the center position of the target shelf, and the second direction is the polar axis direction of the polar coordinate system of the laser radar; the target line segment is a line segment connecting the two centroids closest to the laser radar among the four centroids of the target shelf.
[0128] In an optional implementation, the data processing unit 502 is further configured to perform segmentation processing on the multiple distance values, and after determining the data segment of each object:
[0129] For each object, determining a first number of distance values included in the data segment for the object;
[0130] Calculating a reference arc length of the data segment of the object based on the first number, the arc resolution of the laser radar, and an average of the distance values included in the data segment of the object;
[0131] Based on the reference arc length corresponding to each object, data segments smaller than a second length threshold are screened out; wherein the second length threshold is determined based on the size of a shelf leg.
[0132] In an optional implementation, the data processing unit 502 is further configured to, before performing segmentation processing on the multiple distance values:
[0133] Filtering out some distance values that meet a set distance condition from among the multiple distance values; wherein the set distance condition is determined based on the detection distance of the laser radar;
[0134] Filtering is performed on the filtered multiple distance values.
[0135] The laser radar-based shelf recognition device proposed in the embodiment of the present application adopts the same inventive concept as the above-mentioned laser radar-based shelf recognition method and can achieve the same beneficial effects, which will not be repeated here.
[0136] Based on the same inventive concept as the above-mentioned shelf recognition method based on laser radar, the embodiment of the present application also provides a mobile device, which can be an automatic guided vehicle, etc. Figure 6 As shown, the mobile device may include a laser radar 600 , a processor 601 and a memory 602 .
[0137] The laser radar 600 executes the step of obtaining multiple distance values when scanning at least one object in the scanning area.
[0138] The processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.
[0139] Memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory can include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. Memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0140] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The above-mentioned computer storage medium may be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid-state drives (SSDs)), and other media that can store program codes.
[0141] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM), a magnetic storage device (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage device (such as a CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as a ROM, EPROM, EEPROM, a non-volatile memory (NAND FLASH), a solid-state drive (SSD)) and other various media that can store program code.
[0142] Based on the same inventive concept, embodiments of the present application further provide a computer program product, comprising computer program code. When executed on a computer, the computer program code causes the computer to execute any of the hard disk data reading methods discussed above. Because the principles underlying the problems solved by the computer program product are similar to those of the hard disk data reading method, the implementation of the computer program product can be referenced to the implementation of the method, and any repetitions will not be repeated.
[0143] The above embodiments are only used to provide a detailed introduction to the technical solutions of the present application. However, the description of the above embodiments is only used to help understand the methods of the embodiments of the present application and should not be understood as limiting the embodiments of the present application. Any changes or substitutions that can be easily conceived by those skilled in the art should be included in the scope of protection of the embodiments of the present application.
Claims
1. A shelf recognition method based on laser radar, characterized in that: Each shelf is composed of four shelf legs and a shelf surface, wherein the four shelf legs form a rectangle, and the method includes: Acquire multiple distance values obtained when the laser radar scans at least one object in the scanning area; wherein the multiple distance values are a piece of ranging data, and the object is a shelf leg or other object; the multiple distance values correspond to multiple scanning angles, and the distance values and the scanning angles correspond one to one; Segmenting the plurality of distance values to determine a data segment for each object; wherein the data segment is a data set including at least one distance value for the object; Filtering multiple data segments based on the shelf parameters of the shelf to determine multiple data segment pairs; wherein each data segment pair includes at least three data segments respectively representing shelf legs belonging to the same shelf; Among the multiple data segment pairs, the target data segment pair closest to the laser radar is selected, and the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf are calculated based on the target data segment pair; wherein the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf instruct the mobile device where the laser radar is located to perform path planning.
2. The method according to claim 1, characterized in that The segmenting of the plurality of distance values to determine the data segment of each object includes: Determine a plurality of mutation data from the plurality of distance values; wherein the mutation data corresponds to a first scanning angle, and an absolute value of a difference between a distance value corresponding to a second scanning angle adjacent to the first scanning angle and the mutation data is greater than a first length threshold; the first length threshold is determined based on the size of a shelf leg; For each mutation data in the plurality of mutation data, it is determined that the mutation data and a distance value between the mutation data and the previous mutation data constitute a data segment of an object.
3. The method according to claim 1, characterized in that The shelf parameters include shelf length, shelf width and shelf diagonal length; The screening of the plurality of data segments based on the shelf parameters of the shelf to determine a plurality of data segment pairs includes: For each object, calculating the center of mass of the object according to the distance value included in the data segment corresponding to the object; Calculate the distance between any two centroids in the centroids corresponding to each object; Among the multiple centroid distances, a plurality of first data segment pairs are determined based on the shelf length, the shelf width, and the length of the shelf diagonal; wherein each first data segment pair corresponds to a centroid group, the centroid group including the centroids corresponding to three objects, and the lengths of three line segments formed by the three centroids in the centroid group corresponding to the first data segment pair respectively satisfy set length conditions with the shelf length, the shelf width, and the length of the shelf diagonal; Among the multiple first data segment pairs, the first data segment pairs whose angles meet the set angle conditions are screened to form the multiple data segment pairs; wherein the angle is the angle between the two shorter line segments of the three line segments calculated from the three centroids in the centroid group corresponding to the first data segment pairs.
4. The method according to claim 3, characterized in that The method further comprises: Calculate the first center point corresponding to each pair of data segments respectively; wherein the first center point is the midpoint of the longest line segment formed by the three centroids in the centroid group corresponding to the data segment; Among the distances between each of the first center points and the laser radar, the data segment pair corresponding to the first center point with the shortest distance is determined as the target data segment pair.
5. The method according to claim 4, characterized in that The position information of the target shelf includes the position information and posture information of the target shelf; The method of selecting a target data segment pair closest to the laser radar from the plurality of data segment pairs and calculating the pose information of the target shelf and the position information of the four shelf legs constituting the target shelf according to the target data segment pair includes: Determine the first center point with the shortest distance from the laser radar as the center of the target shelf; wherein the position information of the center of the target shelf is the position information of the target shelf; Determine the centroid of the fourth shelf leg of the target shelf; wherein the target shelf is composed of the centroids of the three shelf legs represented by the target data segment pairs; the positions of the centroids of the four shelf legs are the position information of the four shelf legs of the target shelf; In the three line segments formed by the three centroids of the target data segment pair, the angle between the first direction and the second direction is determined as the posture information of the target shelf; wherein the first direction is the direction of the line connecting the midpoint of the target line segment and the center position of the target shelf, and the second direction is the polar axis direction of the polar coordinate system of the laser radar; the target line segment is a line segment formed by connecting the two centroids closest to the laser radar among the four centroids of the target shelf.
6. The method according to claim 1, characterized in that After segmenting the multiple distance values to determine the data segment of each object, the method further includes: determining, for each object, a first number of distance values included in the data segment for the object; Calculating a reference arc length of the data segment of the object based on the first number, the arc resolution of the laser radar, and an average of distance values included in the data segment of the object; Based on the reference arc length corresponding to each object, data segments smaller than a second length threshold are screened out; wherein the second length threshold is determined according to the size of a shelf leg.
7. The method according to any one of claims 1 to 6, characterized in that Before segmenting the multiple distance values, the method further includes: Filtering out some distance values that meet a set distance condition from the multiple distance values; wherein the set distance condition is determined based on the detection distance of the laser radar; The filtered distance values are filtered.
8. A shelf recognition device based on laser radar, characterized in that: Each shelf is composed of four shelf legs and a shelf surface, wherein the four shelf legs form a rectangle, and the device comprises: a data acquisition unit, configured to: acquire a plurality of distance values obtained when the laser radar scans at least one object in a scanning area; wherein the object is a shelf leg or other object; the plurality of distance values correspond to a plurality of scanning angles, and the distance values and the scanning angles correspond one to one; a data processing unit, configured to: segment the plurality of distance values to determine a data segment for each object; wherein the data segment is a data set including at least one distance value of the object; A shelf identification unit is used to: screen a plurality of data segments based on shelf parameters of the shelf to determine a plurality of data segment pairs; wherein each data segment pair includes at least three data segments respectively representing shelf legs belonging to the same shelf; The shelf identification unit is further used to: select the target data segment pair closest to the laser radar from the multiple data segment pairs, and calculate the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf based on the target data segment pair; wherein the posture information of the target shelf and the position information of the four shelf legs constituting the target shelf instruct the mobile device where the laser radar is located to perform path planning.
9. A mobile device comprising a laser radar, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the shelf identification method according to any one of claims 1 to 7 are implemented, and the laser radar executes the step of obtaining multiple distance values when scanning at least one object in the scanning area.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the shelf identification method according to any one of claims 1 to 7 are implemented.