Method, device, computer equipment and storage medium for detecting pickup height

Abstract

The present application relates to a method, apparatus, computer device, and storage medium for detecting pickup height. The method comprises: obtaining original point cloud data of a target pickup area; the target pickup area includes a pallet for storing and retrieving goods; performing image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object used to represent the pallet; determining the positioning contour edge of the pallet object from the target image; the positioning contour edge is the edge of the pallet object's outline used to locate the pickup height; determining target point cloud data in the original point cloud data that matches the positioning contour edge; and determining the target pickup height based on the target point cloud data. The present application can still normally calculate the target pickup height even when the original point cloud data is incomplete, thereby improving the accuracy of the target pickup height.

Description

Technical Field

[0001] The present application relates to the field of logistics application technology, and in particular to a method, device, computer equipment and storage medium for detecting pickup height. Background Art

[0002] With the development of the logistics industry, the amount of goods is increasing, and the demand for picking up goods is also increasing. At present, the picking up of goods is mainly done by handling equipment, such as unmanned forklifts, according to a predetermined picking height.

[0003] Typically, the pickup height is calculated directly from the point cloud data on the pallet. However, if the pallet is damaged or obstructed, the point cloud data on the pallet may be incomplete, affecting the accuracy of the pickup height. Therefore, improving the accuracy of the pickup height has become a technical problem that those skilled in the art need to solve. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for detecting the pickup height that can improve the accuracy of the pickup height in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting the height of a pickup. The method comprises:

[0006] Acquire original point cloud data of a target pickup area; the target pickup area includes a pallet for storing and retrieving goods;

[0007] Performing image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object for representing the pallet;

[0008] Determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is an edge in the contour of the pallet object used to locate the pickup height;

[0009] Determining target point cloud data in the original point cloud data that matches the positioning contour edge;

[0010] A target pickup height is determined based on the target point cloud data.

[0011] In a second aspect, the present application also provides a device for detecting the height of a pickup. The device comprises:

[0012] A data acquisition module is used to acquire original point cloud data of a target pickup area; the target pickup area includes a pallet for storing and retrieving goods;

[0013] An image conversion module is used to perform image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object used to represent the pallet;

[0014] An edge determination module is used to determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is an edge in the outline of the pallet object used to locate the pickup height;

[0015] A data matching module is used to determine target point cloud data that matches the positioning contour edge in the original point cloud data;

[0016] A height determination module is used to determine a target pickup height based on the target point cloud data.

[0017] In some embodiments, the image conversion module includes a screening unit and a projection unit. The screening unit is configured to screen point cloud data located on a vertical section in the laser coordinate system from the raw point cloud data to obtain reference point cloud data. The projection unit is configured to project the reference point cloud data to obtain the target image. The data matching module is further configured to determine target point cloud data that matches the positioning contour edge from the reference point cloud data.

[0018] In some embodiments, the screening unit is also used to preliminarily screen preliminary point cloud data that meets preset conditions from the original point cloud data; divide the preliminary point cloud data into voxel grids; and screen the target points on the vertical section under the laser coordinate system from each voxel grid to obtain the reference point cloud data.

[0019] In some embodiments, the edge determination module is further configured to preprocess the target image to obtain a preprocessed image; perform contour recognition on the preprocessed image to obtain a recognized contour; and determine the positioning contour edge of the pallet object from the recognized contour.

[0020] In some embodiments, each of the voxel grids corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to characterize whether the corresponding voxel grid has point cloud data. The edge determination module is also used to determine the voxel index corresponding to each pixel coordinate point in the pixel coordinate system of the target image; the voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system; according to the voxel index corresponding to the pixel coordinate point, the target image is grayscale processed to obtain a grayscale image; and the grayscale image is subjected to expansion and corrosion operations to obtain the preprocessed image.

[0021] In some embodiments, the data matching module is further used to determine the target voxel index corresponding to the pixel coordinate point of the positioning contour edge; from the reference point cloud data, determine the point in the target voxel grid pointed to by the target voxel index to obtain the target point cloud data.

[0022] In some embodiments, the height determination module is further used to perform straight line fitting on the target point cloud data to obtain a fitting line segment; and determine the target pickup height based on the midpoint value of the fitting line segment.

[0023] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method for detecting the pickup height when executing the computer program.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-mentioned method for detecting the pickup height.

[0025] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the above-mentioned method for detecting the pickup height.

[0026] The above-mentioned method, device, computer equipment and storage medium for detecting the pickup height obtain the original point cloud data of the target pickup area; the target pickup area includes a pallet for storing and retrieving goods; an image conversion is performed based on the original point cloud data to obtain a target image; the target image includes a pallet object for representing the pallet; the positioning contour edge of the pallet object is determined from the target image; the positioning contour edge is the edge in the contour of the pallet object used to locate the pickup height; the target point cloud data that matches the positioning contour edge in the original point cloud data is determined; and the target pickup height is determined based on the target point cloud data. The present application obtains the original point cloud data of the target pickup area, performs an image conversion on the original point cloud data to obtain the target image, and only determines the target point cloud data that matches the positioning contour edge in the original point cloud data for the positioning contour edge in the target image, and determines the target pickup height based on the target point cloud data. In the case where the original point cloud data is incomplete, the target pickup height can still be calculated normally, thereby improving the accuracy of the target pickup height. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of a flow chart of a method for detecting the pickup height in some embodiments;

[0028] Figure 2is a schematic diagram of a tray in some embodiments;

[0029] Figure 3 is a schematic diagram of a laser coordinate system in some embodiments;

[0030] Figure 4 is a schematic diagram of a voxel coordinate system in some embodiments;

[0031] Figure 5 is a schematic diagram of a pixel coordinate system in some embodiments;

[0032] Figure 6 is a structural block diagram of a device for detecting the pickup height in some embodiments;

[0033] Figure 7 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0035] In some embodiments, as Figure 1 As shown, a method for detecting the height of picking up goods is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to handling equipment, and can also be applied to a system including handling equipment and a server, and is implemented through the interaction between the handling equipment and the server. Among them, the handling equipment refers to a transportation equipment used to carry goods. The handling equipment can be, but is not limited to, at least one of an automated guided vehicle (AGV) and a forklift, wherein the forklift can be, but is not limited to, an unmanned forklift. In this embodiment, the method includes the following steps:

[0036] Step 102: Obtain original point cloud data of the target pickup area.

[0037] The target pickup area includes one or more pallets for storing and retrieving goods.

[0038] A pallet is a means of transporting goods in groups.

[0039] Point cloud data refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually expressed in the form of X, Y, and Z three-dimensional coordinates and are generally used to represent the outer surface shape of an object.

[0040] Specifically, a laser device scans the target pickup area to generate corresponding raw point cloud data. A laser device refers to a device capable of emitting laser light. The server then uses this raw point cloud data to calculate the target pickup height for the handling equipment.

[0041] In some embodiments, the laser device can be fixed on the handling device, and the specific fixed position of the laser device is not limited, as long as the fixed laser device can accurately scan the point cloud data.

[0042] In some embodiments, the laser device can be fixed at the midpoint between the bases of two clamping arms used to clamp or hold cargo on the handling equipment. This ensures that the laser device is not blocked by the cargo during laser scanning, thereby improving laser scanning accuracy. The base of the clamping arm refers to the end of the clamping arm on the handling equipment that faces the handling equipment body.

[0043] In some embodiments, after the laser device is mounted on the transporting device, the server controls the transporting device to move to the pickup location, where the laser device performs a laser scan of the target pickup area to generate raw point cloud data. The pickup location is the approximate pickup location pre-set for the transporting device.

[0044] In other embodiments, after the server controls the transport device to travel to the pickup location, the transport device feeds the pre-target location of the pickup pallet to the perception algorithm module. The perception algorithm module then uses a laser device mounted on the transport device to capture point cloud data in front of the transport device. Based on the pre-target location transmitted by the transport device, the perception algorithm module extracts the portion of point cloud data associated with the pre-target location from the point cloud data in front of the transport device as the raw point cloud data.

[0045] The waiting pallet refers to the pallet in the target pickup area that needs to be picked up by the handling equipment. The perception algorithm module is a module on the server used to extract point cloud data.

[0046] The pre-target position of the pallet to be picked up refers to the coordinate position of the pallet to be picked up relative to the coordinate system where the handling equipment is located, specifically including but not limited to the longitudinal width of the pallet to be picked up in the X-axis direction relative to the coordinate system where the handling equipment is located, the lateral length of the pallet to be picked up in the Y-axis direction relative to the coordinate system where the handling equipment is located, the vertical height of the pallet to be picked up in the Z-axis direction relative to the coordinate system where the handling equipment is located, and at least one of the angles of the pallet plane of the pallet to be picked up compared to the coordinate system where the handling equipment is located in the Y-axis direction.

[0047] The point cloud data in front of the handling equipment includes the point cloud data of the pallet to be picked up and the point cloud data of the environment surrounding the pallet to be picked up.

[0048] In some embodiments, the process of the perception algorithm module extracting point cloud data of interest from the point cloud data in front of the handling equipment according to the pre-target position is as follows: the perception algorithm module extracts part of the point cloud data that matches the pre-target position of the pallet to be picked up from the point cloud data in front of the handling equipment according to the pre-calibrated laser external parameters and the pre-configured pallet size parameters of the pallet to be picked up, as the original point cloud data.

[0049] The laser external parameter refers to the coordinate system conversion relationship between the laser device and other coordinate systems, such as the coordinate system of the handling equipment. The pallet size parameters include but are not limited to at least one of the pallet length, pallet width, and pallet height.

[0050] In some embodiments, the perception algorithm module may also extract, based on the pallet size parameter, point cloud data that falls within a preset size range and matches the pallet to be picked up as raw point cloud data. The preset size range is set based on the pallet size parameter and a preset value. Specifically, the difference between the pallet size parameter and the preset value is used as the minimum value of the preset size range, and the sum of the pallet size parameter and the preset value is used as the maximum value of the preset size range. The preset value can be set based on actual needs, for example, it can be set to 0.3 cm.

[0051] In other embodiments, the laser equipment can also be set separately from the handling equipment. Specifically, the laser equipment can be fixedly installed at a position where laser scanning can be performed on the target pickup area. When the original point cloud data of the target pickup area needs to be collected, laser scanning is performed directly through the laser equipment.

[0052] Step 104: Perform image conversion based on the original point cloud data to obtain a target image.

[0053] The target image includes a pallet object for representing a pallet.

[0054] In some embodiments, the server can directly convert the raw point cloud data into an image to obtain a target image. In other embodiments, the server can also filter the raw point cloud data and convert the filtered point cloud data into an image to obtain the target image. This application can improve the accuracy of the target image by filtering out a portion of the raw point cloud data before image conversion.

[0055] Step 106: Determine the positioning contour edge of the pallet object from the target image.

[0056] The positioning contour edge is the edge within the positioning contour of the pallet object used to determine the pickup height. In practical applications, the pallet object's contour includes an inner contour and an outer contour. The positioning contour of the pallet object is the inner contour of the pallet object. The positioning contour edge of the pallet object includes, but is not limited to, at least one of the upper edge of the inner contour of the pallet object and the lower edge of the inner contour of the pallet object.

[0057] The positions of the inner contour of the pallet object, the outer contour of the pallet object, the upper edge line of the inner contour of the pallet object, and the lower edge line of the inner contour of the pallet object can be specifically referred to as shown in FIG2 .

[0058] In some embodiments, the server can directly extract the positioning contour of the pallet object, i.e., the inner contour, from the target image, and extract the corresponding positioning contour edge based on the inner contour of the pallet object, i.e., the upper edge line of the inner contour of the pallet object or the lower edge line of the inner contour of the pallet object.

[0059] In other embodiments, the server may also first extract the outer contour of the pallet object from the target image, and identify the positioning contour within the contour range of the outer contour, that is, the inner contour, and then extract the corresponding positioning contour edge based on the inner contour of the pallet object, that is, the upper edge line of the inner contour of the pallet object or the lower edge line of the inner contour of the pallet object.

[0060] Step 108 : Determine target point cloud data in the original point cloud data that matches the positioning contour edge.

[0061] Specifically, the server determines point cloud data that matches the positioning contour edge from the original point cloud data based on the positioning contour edge determined in the target image, as the target point cloud data.

[0062] In some embodiments, the server may further filter the original point cloud data, and determine point cloud data that matches the positioning contour edge from the filtered point cloud data as target point cloud data.

[0063] Step 110: Determine a target pickup height based on the target point cloud data.

[0064] The target pickup height refers to the height at which the handling equipment picks up the pallet to be picked up.

[0065] Specifically, after obtaining the target point cloud data that matches the positioning contour edge, the server fits the target point cloud data and determines the target pickup height based on the fitting result.

[0066] In some embodiments, the server may further determine a pickup plane for the pallet to be picked up based on the target point cloud data, and determine a target pickup height based on the pickup plane. The pickup plane refers to the plane on which the handling equipment needs to pick up the pallet to be picked up.

[0067] In the above-mentioned method for detecting the pickup height, the original point cloud data of the target pickup area is obtained; the target pickup area includes a pallet for storing and retrieving goods; an image conversion is performed based on the original point cloud data to obtain a target image; the target image includes a pallet object for representing the pallet; the positioning contour edge of the pallet object is determined from the target image; the positioning contour edge is the edge in the contour of the pallet object used to locate the pickup height; the target point cloud data that matches the positioning contour edge in the original point cloud data is determined; and the target pickup height is determined based on the target point cloud data. This application obtains the original point cloud data of the target pickup area, performs an image conversion on the original point cloud data to obtain a target image, and only determines the target point cloud data that matches the positioning contour edge in the original point cloud data for the positioning contour edge in the target image, and determines the target pickup height based on the target point cloud data. This application can normally calculate the target pickup height even when the original point cloud data is incomplete, thereby improving the accuracy of the target pickup height.

[0068] In some embodiments, step 104 specifically includes but is not limited to: filtering point cloud data on a vertical section in the laser coordinate system where the laser device is located from the original point cloud data to obtain reference point cloud data; and projecting the reference point cloud data to obtain a target image.

[0069] The laser coordinate system refers to a three-dimensional coordinate system constructed based on the laser device, and the vertical section in the laser coordinate system refers to a two-dimensional plane in the vertical direction of the laser coordinate system, specifically the xoy plane, that is, the plane where z=0 in the laser coordinate system. Figure 3 .

[0070] Specifically, the server selects the two-dimensional plane located in the vertical section of the laser coordinate system from the original point cloud data, that is, the point cloud data of the XOY plane at Z = 0 in the laser coordinate system, to obtain the reference point cloud data. The reference point cloud data is projected into an image to obtain the target image.

[0071] In some embodiments, the server may further filter out the point cloud data of the two-dimensional plane located in the vertical direction of the laser coordinate system from the original point cloud data, and then filter out the point cloud data of the two-dimensional plane located in the vertical direction of the laser coordinate system again to obtain reference point cloud data. The reference point cloud data is projected into an image to obtain a target image.

[0072] In some embodiments, step 108 specifically includes but is not limited to: determining target point cloud data that matches the positioning contour edge from the reference point cloud data.

[0073] Specifically, the server determines point cloud data that matches the positioning contour edge from the reference point cloud data as the target point cloud data.

[0074] In some embodiments, the step of "filtering point cloud data on the vertical section under the laser coordinate system from the original point cloud data to obtain reference point cloud data" specifically includes but is not limited to: preliminarily filtering preliminary point cloud data that meets preset conditions from the original point cloud data; dividing the preliminary point cloud data into voxel grids; and filtering target points on the vertical section under the laser coordinate system from each voxel grid to obtain reference point cloud data.

[0075] The preset condition refers to a preset distance range from each point of the original point cloud data after plane fitting is performed on the XOY plane (i.e., the plane with Z=0 in the laser coordinate system) in the laser coordinate system where the laser device is located, to the fitting plane. The fitting plane is a plane obtained after plane fitting is performed on the original point cloud data, and the preset distance range can be set to 2 cm to 3 cm.

[0076] In practical applications, the Random Sample Consensus (RANSAC) algorithm can be used to perform plane fitting on raw point cloud data. The RANSAC algorithm calculates mathematical model parameters based on a set of sample data containing abnormal data to obtain valid sample data.

[0077] Specifically, after the server performs plane fitting on the raw point cloud data and obtains the fitted plane, it filters out point cloud data that meets a preset distance range, for example, filtering out points within a range of 2 to 3 centimeters from the fitted plane, to form preliminary point cloud data. Next, the server performs voxel filtering on the preliminary point cloud data. Specifically, the server divides the preliminary point cloud data into voxel grids, and after filtering the target points on the vertical section of the laser coordinate system from each voxel grid, it replaces all points in the corresponding voxel grid with the target points filtered out from each voxel grid to obtain the reference point cloud data.

[0078] The voxel grid may be a cubic grid, specifically a cube, and the side length of the voxel grid may be 1 cm. The voxel grid may also be a plane grid, specifically a square, and the corresponding side length may also be 1 cm on a vertical section in the laser coordinate system.

[0079] The target point is a point selected from each voxel grid based on preset criteria. Generally, the point with the largest Y coordinate (i.e., the vertical coordinate in the laser coordinate system) in each voxel grid is used as the target point. For ease of description, the vertical coordinate in the laser coordinate system will be referred to as the Y coordinate.

[0080] In other embodiments, the server may first filter the point cloud data on the vertical section under the laser coordinate system from the preliminary point cloud data, and after dividing the point cloud data on the vertical section under the laser coordinate system into voxel grids, directly filter the target points from each voxel grid, and replace all points in the corresponding voxel grid with the target points filtered out from each voxel grid to obtain reference point cloud data.

[0081] In some embodiments, step 106 specifically includes but is not limited to: preprocessing the target image to obtain a preprocessed image; performing contour recognition on the preprocessed image to obtain a recognized contour; and determining the positioning contour edge of the pallet object from the recognized contour.

[0082] Specifically, the server preprocesses the target image, performing grayscale processing and erosion and dilation operations to improve image accuracy. The server then identifies the pallet object's outline from the preprocessed image and compares it with the actual pallet's dimensions. It then extracts the outline that matches the actual pallet's size as a candidate outline and identifies the positioning outline edge of the pallet object from these candidate outlines.

[0083] The size of the actual pallet includes but is not limited to at least one of the length and width of the actual pallet.

[0084] In some embodiments, the specific process of identifying the positioning contour edge of the pallet object from the alternative contours is: for each alternative contour, search from bottom to bottom or from bottom to top for pixel points with a grayscale value of a preset value, for example, search for pixel points with a grayscale value of 255, so as to find all the positioning contour edges on the pallet object.

[0085] In some embodiments, after the step of "replacing all points in the corresponding voxel grid with the target points screened out in each voxel grid to obtain reference point cloud data", a voxel coordinate system can also be established based on the reference point cloud data, and an index array corresponding to the reference point cloud data can be set based on the established voxel coordinate system. For the sake of convenience of description, the index array corresponding to the reference point cloud data is referred to as the voxel grid index array.

[0086] Each pair of voxel grid index arrays is the index array of the target point in each voxel grid. In other words, [voxel index of each voxel grid] = (voxel index of each target point with the largest Y coordinate in the voxel grid). If there is no point in the voxel grid, the array value of [voxel index of the voxel grid] is set to a fixed value, such as negative 1.

[0087] The specific form of the voxel coordinate system can be referred to Figure 4 ,exist Figure 4 The 0, 1, 2, etc. shown in the coordinate system are all voxel indices. Figure 4(x_min, y_min) represents the coordinate point with the minimum X coordinate and the minimum Y coordinate in the laser coordinate system, which is also the origin of the voxel coordinate system. Figure 4 Where (x_max, y_min) represents the coordinate point with the maximum X coordinate and the minimum Y coordinate in the laser coordinate system, (x_min, y_max) represents the coordinate point with the minimum X coordinate and the maximum Y coordinate in the laser coordinate system, and (x_max, y_max) represents the coordinate point with the maximum X coordinate and the maximum Y coordinate in the laser coordinate system. The X coordinate in the laser coordinate system is the horizontal coordinate obtained in the laser coordinate system, and the Y coordinate in the laser coordinate system is the vertical coordinate obtained in the laser coordinate system. For ease of description, the horizontal coordinate in the laser coordinate system is referred to as the X coordinate, and the vertical coordinate in the laser coordinate system is referred to as the Y coordinate.

[0088] It should be noted that (x_min, y_min) is used as the origin of the voxel coordinate system for the convenience of calculation. In practical applications, points at other locations in the reference point cloud data can also be used as the origin of the voxel coordinate system, and this application does not limit this.

[0089] In some embodiments, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to characterize whether the corresponding voxel grid has point cloud data. Specifically, whether the corresponding voxel grid has point cloud data is determined based on the value corresponding to the voxel index. If the array value corresponding to the voxel index is negative 1, it means that the corresponding voxel grid has no point cloud data. If the array value corresponding to the voxel index is not negative 1, it means that the corresponding voxel grid has point cloud data.

[0090] In some embodiments, the target image can also be obtained by projecting the points in the voxel grid corresponding to all voxel indices in the voxel coordinate system. Specifically, the points in the voxel grid are all located on the XOY plane of the laser coordinate system where the laser device is located. Therefore, the points in the voxel grid need to be projected parallel to the XOY plane to obtain the target image.

[0091] In some embodiments, after obtaining the target image, the pixel coordinate point corresponding to the voxel index in the pixel coordinate system established based on the target image can be determined according to the position of the point in the voxel grid corresponding to the voxel index in the voxel coordinate system.

[0092] In some embodiments, the step of "preprocessing the target image to obtain a preprocessed image" specifically includes but is not limited to: determining the voxel index corresponding to each pixel coordinate point in the pixel coordinate system of the target image; performing grayscale processing on the target image according to the voxel index corresponding to the pixel coordinate point in the voxel coordinate system to obtain a grayscale image; performing dilation and erosion operations on the grayscale image to obtain a preprocessed image.

[0093] The pixel coordinate system refers to the pixel coordinate system established based on the target image. The origin of this pixel coordinate system must be consistent with the origin of the voxel coordinate system. The voxel index corresponding to a pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system.

[0094] In some embodiments, the specific form of the pixel coordinate system can be referred to Figure 5 , with the voxel coordinate system origin (x_min, y_min) as the origin of the pixel coordinate system of the target image, (x_min, y_min) represents the pixel with the smallest X coordinate and the smallest Y coordinate in the pixel coordinate system. Figure 5 Where (x_max, y_min) represents the pixel with the maximum X coordinate and the minimum Y coordinate in the pixel coordinate system, (x_min, y_max) represents the pixel with the minimum X coordinate and the maximum Y coordinate in the pixel coordinate system, and (x_max, y_max) represents the pixel with the maximum X coordinate and the maximum Y coordinate in the pixel coordinate system.

[0095] Specifically, for each pixel coordinate point in the pixel coordinate system of the target image, the server determines the voxel index in the voxel coordinate system that corresponds one-to-one to the pixel coordinate point. The server obtains the corresponding array value from the voxel grid index array according to the voxel index, and sets the pixel coordinate point corresponding to the voxel index with different array values ​​to different grayscale values, thereby obtaining a grayscale image. Next, the server performs an expansion and corrosion operation on the grayscale image. Specifically, the server can perform a closing operation on the grayscale image to fill the small holes in the grayscale image and fit the cracks in the grayscale image to ensure that the overall position and shape of the grayscale image remain unchanged. Performing a closing operation on the grayscale image can prevent holes in the point cloud data obtained by scanning the pallet to be picked up due to obstructions on the pallet to be picked up or other factors.

[0096] In practical applications, the step of "setting the pixel coordinate points corresponding to the voxel indices having different array values ​​to different grayscale values ​​to obtain a grayscale image" specifically includes but is not limited to: if the array value corresponding to the voxel index is greater than a fixed value (the fixed value may be set to negative 1), then the grayscale value of the pixel coordinate point corresponding to the voxel index having an array value greater than the fixed value is set to a preset value, for example, 255. If the array value corresponding to the voxel index is equal to or less than the fixed value, then the grayscale value of the pixel coordinate point corresponding to the voxel index having an array value equal to or less than negative 1 is set to another preset value, for example, 0.

[0097] In some embodiments, the step of "determining target point cloud data that matches the positioning contour edge from the reference point cloud data" specifically includes but is not limited to: determining the target voxel index corresponding to the pixel coordinate point of the positioning contour edge; determining the point in the target voxel grid pointed to by the target voxel index from the reference point cloud data to obtain the target point cloud data.

[0098] Specifically, the server determines the pixel coordinates of the contour edge in the target image in the pixel coordinate system and determines the voxel index corresponding to the pixel coordinates of the contour edge from the voxel index as the target voxel index. The server then determines the voxel grid pointed to by the target voxel index from the reference point cloud data, i.e., the target voxel grid, and extracts the corresponding target point from the target voxel grid to obtain the target point cloud data.

[0099] In some embodiments, step 110 specifically includes but is not limited to: performing straight line fitting on the target point cloud data to obtain a fitting line segment; and determining a target pickup height based on the midpoint value of the fitting line segment.

[0100] Specifically, the server performs a straight line fit on the target point cloud data, such as a least squares linear fit, to obtain a fitted line. The server obtains the fitted line segment formed by the target point cloud data on the fitted line and determines the target pickup height of the handling equipment based on the midpoint value of the segment corresponding to the midpoint of the fitted line segment. The midpoint value of the segment refers to the Y value of the midpoint of the fitted line segment in the laser coordinate system.

[0101] It should be noted that any straight line can be expressed as a straight line equation, i.e., y = bx + a, where b is the slope and a is the intercept. For N points in the target point cloud data, an infinite number of straight lines can be used to fit the objective function using the least squares method to find the optimal solution of the objective function, i.e., the optimal slope and intercept in the straight line equation. Based on these optimal slope and intercept, the fitted straight line can be determined.

[0102] In some embodiments, the steps of using least squares straight line fitting to obtain the optimal slope and the optimal intercept in the straight line equation are as follows, and the formulas involved include formula (1) to formula (7).

[0103] Step 1: Establish the objective function based on the straight line equation.

[0104]

[0105] Step 2: Derivative the objective function of formula (1) to obtain the linear equation expression.

[0106]

[0107]

[0108] Step 3: After sorting out the linear equation expressions obtained by formula (2) and formula (3), the sorted linear equation expressions are obtained.

[0109] aN+b∑x i =∑y i (4)

[0110] a∑x i +b∑x i 2 =∑x i y i (5)

[0111] Step 4: Solve the rearranged straight line equation to get the best estimate of the straight line parameters a and b, and thus get the fitted straight line.

[0112]

[0113]

[0114] Among them, x i and y i It is the X coordinate value (i.e. the horizontal coordinate value of each point in the laser coordinate system) and the Y coordinate value (i.e. the vertical coordinate value of each point in the laser coordinate system) of the target point cloud data. a and b represent the slope and intercept in the straight line equation respectively. and It represents the optimal slope and the optimal intercept in the straight line equation, and N represents the number of points in the target point cloud data.

[0115] In other embodiments, to prevent collisions with handling equipment during the pickup process, the target pickup height of the present application can also be determined based on the Y value (i.e., the midpoint of the fitted line segment) and the height of the pallet footrests located at the bottom of the pallet. Specifically, the target pickup height = Y value + half the height of the pallet footrests. The pallet footrests, also known as pallet pads, pallet blocks, and pallet squares, are located at the bottom of the pallet and are used to protect the cargo from being squeezed or collided with by handling equipment during transportation.

[0116] In some embodiments, the method for detecting the pickup height of the present application further includes, but is not limited to, the following steps:

[0117] First, a laser device is fixed on the transport equipment, and after the transport equipment is controlled to travel to the location where the goods are to be picked up, the original point cloud data of the target area is obtained through the laser device.

[0118] Secondly, the server obtains the original point cloud data collected by the laser equipment, preliminarily screens the preliminary point cloud data that meets the preset conditions from the original point cloud data, divides the preliminary point cloud data into voxel grids, and screens the target points on the vertical section under the laser coordinate system from each voxel grid to obtain reference point cloud data.

[0119] Next, the server establishes a voxel coordinate system for the reference point cloud data and determines the voxel index corresponding to the voxel coordinate system. The server projects the reference point cloud data to obtain a target image and preprocesses the target image to obtain a preprocessed image.

[0120] In some embodiments, the process of preprocessing the target image includes: establishing a pixel coordinate system for the target image; determining the voxel index corresponding to each pixel coordinate point in the pixel coordinate system of the target image; performing grayscale processing on the target image based on the voxel index corresponding to the pixel coordinate point to obtain a grayscale image; and performing dilation and erosion operations on the grayscale image to obtain a preprocessed image. Each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to indicate whether the corresponding voxel grid has point cloud data. The voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system.

[0121] Subsequently, the server performs contour recognition on the pre-processed image to obtain a recognized contour, and then determines the positioning contour edge of the pallet object from the recognized contour. Specifically, the server first recognizes all contours in the pre-processed image to obtain a recognized contour, and then determines the positioning contour edge of the pallet object from the recognized contour.

[0122] Next, the server determines the target voxel index corresponding to the pixel coordinate point of the positioning contour edge, and determines the point in the target voxel grid pointed to by the target voxel index from the reference point cloud data to obtain the target point cloud data.

[0123] Finally, the server performs straight line fitting on the target point cloud data to obtain a fitting line segment, and determines the target pickup height based on the midpoint value of the fitting line segment.

[0124] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0125] Based on the same inventive concept, the present application also provides a device for detecting the pickup height of a product for implementing the aforementioned method for detecting the pickup height. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of the one or more embodiments of the device for detecting the pickup height provided below can be found in the above-mentioned limitations of the method for detecting the pickup height, and will not be repeated here.

[0126] In some embodiments, as Figure 6 As shown, a device for detecting the pickup height is provided, comprising: a data acquisition module 602, an image conversion module 604, an edge determination module 606, a data matching module 608, and a height determination module 610, wherein:

[0127] The data acquisition module 602 is used to acquire the original point cloud data of the target pickup area; the target pickup area includes the pallets used for storing and retrieving goods;

[0128] An image conversion module 604 is used to perform image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object used to represent the pallet;

[0129] The edge determination module 606 is used to determine the positioning contour edge of the pallet object from the target image; the positioning contour edge is the edge of the pallet object's contour used to locate the pickup height;

[0130] A data matching module 608 is used to determine target point cloud data that matches the positioning contour edge in the original point cloud data;

[0131] The height determination module 610 is used to determine the target pickup height based on the target point cloud data.

[0132] The present application uses the above-mentioned pickup height detection device to obtain the original point cloud data of the target pickup area; the target pickup area includes a pallet for storing and retrieving goods; based on the original point cloud data, an image conversion is performed to obtain a target image; the target image includes a pallet object used to characterize the pallet; the positioning contour edge of the pallet object is determined from the target image; the positioning contour edge is the edge in the contour of the pallet object used to locate the pickup height; the target point cloud data that matches the positioning contour edge in the original point cloud data is determined; and the target pickup height is determined based on the target point cloud data. By obtaining the original point cloud data of the target pickup area, performing an image conversion on the original point cloud data to obtain the target image, and only focusing on the positioning contour edge in the target image, the target point cloud data that matches the positioning contour edge in the original point cloud data is determined, and the target pickup height is determined based on the target point cloud data. In the case where the original point cloud data is incomplete, the target pickup height can still be calculated normally, thereby improving the accuracy of the target pickup height.

[0133] In some embodiments, the image conversion module 604 includes a screening unit and a projection unit. The screening unit is configured to screen the point cloud data on the vertical section of the laser coordinate system from the raw point cloud data to obtain reference point cloud data. The projection unit is configured to project the reference point cloud data to obtain a target image. The data matching module 608 is further configured to determine target point cloud data that matches the positioning contour edge from the reference point cloud data.

[0134] In some embodiments, the screening unit is also used to preliminarily screen preliminary point cloud data that meets preset conditions from the original point cloud data; divide the preliminary point cloud data into voxel grids; and screen the target points on the vertical section under the laser coordinate system from each voxel grid to obtain reference point cloud data.

[0135] In some embodiments, the edge determination module 606 is further configured to pre-process the target image to obtain a pre-processed image; perform contour recognition on the pre-processed image to obtain a recognized contour; and determine the positioning contour edge of the pallet object from the recognized contour.

[0136] In some embodiments, each voxel grid corresponds to a voxel index in the voxel coordinate system, and the voxel index is used to characterize whether the corresponding voxel grid has point cloud data. The edge determination module 606 is also used to determine the voxel index corresponding to each pixel coordinate point in the pixel coordinate system of the target image; the voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system; according to the voxel index corresponding to the pixel coordinate point, the target image is grayscale processed to obtain a grayscale image; and the grayscale image is dilated and eroded to obtain a preprocessed image.

[0137] In some embodiments, the data matching module 608 is further used to determine the target voxel index corresponding to the pixel coordinate point of the positioning contour edge; determine the point in the target voxel grid pointed to by the target voxel index from the reference point cloud data to obtain the target point cloud data.

[0138] In some embodiments, the height determination module 610 is further used to perform straight line fitting on the target point cloud data to obtain a fitting line segment; and determine the target pickup height based on the midpoint value of the fitting line segment.

[0139] Each module in the aforementioned pickup height detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0140] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store point cloud data and images. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting a pickup height is implemented.

[0141] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0142] In some embodiments, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0143] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0144] In some embodiments, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0145] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0146] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting the height of picking up goods, characterized in that: The method comprises: Acquire original point cloud data of a target pickup area; the target pickup area includes a pallet for storing and retrieving goods; Performing image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object for representing the pallet; Determining the positioning contour edge of the pallet object from the target image includes: extracting the outer contour of the pallet object from the target image, identifying the inner contour within the contour range of the outer contour, and extracting the corresponding positioning contour edge based on the inner contour; the positioning contour edge is an edge in the contour of the pallet object used to locate the height of the pickup; wherein the contour of the pallet object includes an inner contour and an outer contour, the positioning contour of the pallet object is the inner contour of the pallet object, and the positioning contour edge includes at least one of an upper edge line of the inner contour of the pallet object and a lower edge line of the inner contour of the pallet object; Determining target point cloud data that matches the positioning contour edge in the original point cloud data, including: determining a target voxel index corresponding to a pixel coordinate point of the positioning contour edge, and determining, from reference point cloud data obtained by screening the original point cloud data, a point located in a target voxel grid pointed to by the target voxel index to obtain the target point cloud data; A target pickup height is determined based on the target point cloud data.

2. The method according to claim 1, characterized in that The performing image conversion based on the original point cloud data to obtain a target image includes: Filtering point cloud data on a vertical section in a laser coordinate system from the original point cloud data to obtain reference point cloud data; Performing projection processing on the reference point cloud data to obtain the target image; The determining target point cloud data in the original point cloud data that matches the positioning contour edge includes: Target point cloud data matching the positioning contour edge is determined from the reference point cloud data.

3. The method according to claim 2, characterized in that The step of screening point cloud data on a vertical section in a laser coordinate system from the original point cloud data to obtain reference point cloud data includes: Preliminarily screening preliminary point cloud data that meets preset conditions from the original point cloud data; Dividing the preliminary point cloud data into voxel grids; Target points on the vertical section of the laser coordinate system are respectively selected from each voxel grid to obtain the reference point cloud data.

4. The method according to claim 3, characterized in that The determining the positioning contour edge of the pallet object from the target image includes: Preprocessing the target image to obtain a preprocessed image; Performing contour recognition on the preprocessed image to obtain a recognized contour; The positioning contour edge of the pallet object is determined from the identified contour.

5. The method according to claim 4, characterized in that Each voxel grid corresponds to a voxel index in a voxel coordinate system, and the voxel index is used to indicate whether the corresponding voxel grid has point cloud data; The preprocessing of the target image to obtain a preprocessed image includes: For each pixel coordinate point in the pixel coordinate system of the target image, determine a voxel index corresponding to the pixel coordinate point; the voxel index corresponding to the pixel coordinate point is the voxel index of the voxel grid corresponding to the pixel coordinate point in the voxel coordinate system; Performing grayscale processing on the target image according to the voxel index corresponding to the pixel coordinate point to obtain a grayscale image; Performing dilation and erosion operations on the grayscale image to obtain the preprocessed image.

6. The method according to any one of claims 1 to 5, characterized in that The determining a target pickup height based on the target point cloud data includes: Performing straight line fitting on the target point cloud data to obtain a fitting line segment; The target pickup height is determined based on the midpoint value of the fitting line segment.

7. A device for detecting the height of picking up goods, characterized in that: The device comprises: A data acquisition module is used to acquire original point cloud data of a target pickup area; the target pickup area includes a pallet for storing and retrieving goods; An image conversion module is used to perform image conversion based on the original point cloud data to obtain a target image; the target image includes a pallet object used to represent the pallet; an edge determination module, configured to determine the positioning contour edge of the pallet object from the target image, comprising: extracting the outer contour of the pallet object from the target image, identifying the inner contour within the contour range of the outer contour, and extracting the corresponding positioning contour edge based on the inner contour; the positioning contour edge is an edge in the contour of the pallet object used to locate the pickup height; wherein the contour of the pallet object includes an inner contour and an outer contour, the positioning contour of the pallet object is the inner contour of the pallet object, and the positioning contour edge includes at least one of an upper edge line of the inner contour of the pallet object and a lower edge line of the inner contour of the pallet object; a data matching module, configured to determine target point cloud data in the original point cloud data that matches the positioning contour edge, comprising: determining a target voxel index corresponding to a pixel coordinate point of the positioning contour edge, and determining, from reference point cloud data screened from the original point cloud data, a point located in a target voxel grid pointed to by the target voxel index, to obtain the target point cloud data; A height determination module is used to determine a target pickup height based on the target point cloud data.

8. The device for detecting the pickup height according to claim 7, characterized in that: The image conversion module includes a screening unit and a projection unit; The screening unit is used to screen the point cloud data on the vertical section in the laser coordinate system from the original point cloud data to obtain reference point cloud data; The projection unit is configured to perform projection processing on the reference point cloud data to obtain the target image; The data matching module is further configured to determine target point cloud data that matches the positioning contour edge from the reference point cloud data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images