Pallet hole height measurement method, device, equipment, storage medium and program product

Abstract

The present invention provides a pallet hole height measurement method, device, equipment, storage medium and program product, which relates to the field of image processing technology. The method comprises: according to the pallet contour, calculating the minimum circumscribed rectangular envelope of the pallet contour, and determining the center point of the pallet and the rotation angle of the rectangular envelope; constructing a straight line segment based on the center point of the pallet and the rectangular envelope; the straight line segment passes through the center point of the pallet, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are respectively the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope; based on the pallet contour, determining the convex hull contour; determining the two intersection points of the straight line segment and the convex hull contour; and obtaining the pallet hole height according to the point cloud coordinates corresponding to the two intersection points. The present invention realizes the automatic measurement of the pallet hole height.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, storage medium and program product for measuring the height of a pallet hole. Background Art

[0002] With the rapid development of computer and robot technology, warehousing and logistics technology is developing towards automation and intelligence. However, the logistics industry has not yet fully achieved standardization and automation. Before standardization and automation are fully realized, unmanned forklifts need to adapt to the current non-standardized scenarios.

[0003] In non-standardized logistics scenarios, pallets come in a variety of sizes and pallet hole heights vary. For pallets that are too short, unmanned forklifts have a low tolerance for forking and cannot be picked up smoothly. Therefore, it is necessary to measure the height of the pallet holes.

[0004] The existing technology has conducted a lot of research on the identification and positioning of pallets, but most of them are limited to the measurement of the center point of the pallet, while the height of the pallet hole is mostly measured manually. Summary of the invention

[0005] The present invention provides a pallet hole height measurement method, device, equipment, storage medium and program product, which are used to solve the defect that the pallet hole height cannot be automatically measured in the prior art, and realize the automatic measurement of the pallet hole height.

[0006] The present invention provides a method for measuring the height of a pallet hole, comprising:

[0007] According to the pallet outline, the minimum circumscribed rectangular envelope of the pallet outline is calculated, and the rotation angle of the pallet center point and the rectangular envelope is determined;

[0008] Based on the center point of the tray and the rectangular envelope, a straight line segment is constructed; the straight line segment passes through the center point of the tray, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope respectively;

[0009] Based on the tray contour, determining a convex hull contour;

[0010] Determine two intersection points of the straight line segment and the convex hull contour;

[0011] The height of the tray hole is obtained according to the point cloud coordinates corresponding to the two intersection points.

[0012] In some embodiments, determining two intersection points of the straight line segment and the convex hull contour includes:

[0013] Traversing two adjacent points in the convex hull contour;

[0014] When the two adjacent points are located on both sides of the straight line segment or on the straight line segment, an intersection point between the contour edge formed by the two adjacent points and the straight line segment is determined.

[0015] In some embodiments, the method further comprises:

[0016] Capture images of the pallet based on the depth camera to obtain a depth image;

[0017] Inputting the depth image into a trained semantic segmentation network to obtain a segmentation result;

[0018] The segmentation result is subjected to morphological dilation processing to obtain the tray outline.

[0019] In some embodiments, before inputting the depth image into the trained semantic segmentation network to obtain the segmentation result, the method further includes:

[0020] Manually annotate the collected pallet depth images to obtain the pallet segmentation dataset;

[0021] performing enhancement processing on the pallet segmentation dataset;

[0022] The enhanced pallet segmentation data set is input into the semantic segmentation network for training to obtain the trained semantic segmentation network.

[0023] In some embodiments, after the depth camera is used to capture an image of the pallet and obtain a depth image, the method further includes:

[0024] The depth image is filtered and mapped to a point cloud image to obtain a point cloud image.

[0025] In some embodiments, the method further comprises:

[0026] Reporting the pallet hole height to a forklift dispatching system;

[0027] When the height of the pallet hole is greater than a preset threshold, receiving a forklift instruction issued by the forklift dispatching system;

[0028] When the height of the pallet hole is less than or equal to a preset threshold, an abandonment forking instruction issued by the forklift dispatching system is received.

[0029] The present invention also provides a pallet hole height measuring device, comprising:

[0030] A calculation module, used to calculate the minimum circumscribed rectangular envelope of the pallet outline according to the pallet outline, and determine the rotation angle of the pallet center point and the rectangular envelope;

[0031] A construction module, configured to construct a straight line segment based on the center point of the tray and the rectangular envelope; the straight line segment passes through the center point of the tray, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are respectively the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope;

[0032] A first determination module, configured to determine a convex hull contour based on the tray contour;

[0033] A second determination module is used to determine two intersection points of the straight line segment and the convex hull contour;

[0034] The first acquisition module is used to obtain the height of the tray hole according to the point cloud coordinates corresponding to the two intersection points.

[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned methods for measuring the height of a tray hole is implemented.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for measuring the height of a tray hole as described above is implemented.

[0037] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for measuring the height of a tray hole as described above is implemented.

[0038] The present invention provides a method, device, equipment, storage medium and program product for measuring the height of a pallet hole. The method calculates the minimum circumscribed rectangular envelope of the pallet contour, constructs a straight line segment based on the center point of the pallet and the rectangular envelope, determines the convex hull contour based on the pallet contour, and obtains the pallet hole height according to the two intersection points of the straight line segment and the convex hull contour, thereby realizing automated measurement of the pallet hole height. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is one of the flow charts of the pallet hole height measurement method provided by the present invention;

[0041] Figure 2 is a schematic diagram of a straight line segment structure provided by the present invention;

[0042] Figure 3 is a schematic diagram of the intersection of a straight line segment and a convex hull contour provided by the present invention;

[0043] Figure 4 This is the second flow chart of the pallet hole height measurement method provided by the present invention;

[0044] Figure 5 It is a structural schematic diagram of a pallet hole height measuring device provided by the present invention;

[0045] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Figure 1 FIG. 1 is one of the flow charts of the method for measuring the height of a tray hole provided by the present invention. Figure 1 As shown, the present invention provides a method for measuring the height of a pallet hole, comprising the following steps:

[0048] Step 110, calculating the minimum circumscribed rectangular envelope of the pallet outline according to the pallet outline, and determining the center point of the pallet and the rotation angle of the rectangular envelope.

[0049] Specifically, according to the pallet outline, the minimum circumscribed rectangular envelope of the pallet outline is calculated, and the rotation angle of the rectangular envelope is determined, and the centroid of the rectangular envelope is determined as the center point of the pallet.

[0050] Step 120, construct a straight line segment based on the center point of the pallet and the rectangular envelope; the straight line segment passes through the center point of the pallet, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope.

[0051] Specifically, Figure 2 is a schematic diagram of the straight line segment structure provided by the present invention, such as Figure 2 As shown, a straight line segment perpendicular to the long side of the rectangular envelope is constructed through the center point of the tray, that is, the straight line segment is parallel to the short side of the rectangular envelope, that is, the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope. Figure 2 As shown in , the rotation angle of the rectangular envelope is , then the slope of the straight line segment is The row coordinates of the two end points of the straight line segment are the maximum and minimum row coordinates of the vertices of the rectangular envelope, which limits the length of the straight line segment.

[0052] For imaging tilted trays caused by perspective effects and other reasons, when constructing straight line segments, considering the rotation angle can better fit the height direction, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope.

[0053] The tray hole height calculation does not use simple vertical measurement, but takes into account the slight rotation angle of the tray in the imaging, making the tray hole height more accurate.

[0054] Step 130 : determining a convex hull contour based on the tray contour.

[0055] Specifically, based on the tray contour, a convex hull contour corresponding to the tray contour is determined, and the convex hull contour contains all pixel points inside the contour.

[0056] The convex hull contour can reduce the amount of calculation and improve the fault tolerance of the pallet semantic segmentation results. The center point and height of the pallet hole can be accurately found even when the pallet is slightly occluded.

[0057] Step 140, determining two intersection points of the straight line segment and the convex hull contour.

[0058] Specifically, Figure 3 is a schematic diagram of the intersection of the straight line segment and the convex hull contour provided by the present invention, such as Figure 3 As shown, the straight line segment intersects with the convex hull contour at two points, and the two intersection points of the straight line segment and the convex hull contour are determined.

[0059] Step 150, obtaining the height of the tray hole according to the point cloud coordinates corresponding to the two intersection points.

[0060] Specifically, according to the point cloud coordinates corresponding to the two intersection points, the spatial distance between the two intersection points is calculated, that is, the height of the tray hole is obtained.

[0061] The method for measuring the height of a pallet hole provided by the present invention calculates the minimum circumscribed rectangular envelope of the pallet contour, constructs a straight line segment based on the center point of the pallet and the rectangular envelope, determines the convex hull contour based on the pallet contour, and obtains the height of the pallet hole according to the two intersection points of the straight line segment and the convex hull contour, thereby realizing automatic measurement of the height of the pallet hole.

[0062] In some embodiments, determining two intersection points of the straight line segment and the convex hull contour includes:

[0063] Traverse two adjacent points in the convex hull contour;

[0064] When two adjacent points are on both sides of a straight line segment or on the straight line segment, the intersection point of the contour edge formed by the two adjacent points and the straight line segment is determined.

[0065] Specifically, it is analyzed whether two adjacent points in the convex hull contour are on both sides of the straight line segment or on the straight line segment. If so, it indicates that the contour edge formed by the two adjacent points has an intersection with the straight line segment, and then the intersection of the contour edge formed by the two adjacent points and the straight line segment is determined.

[0066] Traverse two adjacent points in the convex hull contour and determine the two intersection points of the straight line segment and the convex hull contour.

[0067] In the pallet hole height measurement method provided by the present invention, when calculating the intersection, it is first determined whether there is an intersection, and then the intersection is calculated, which helps to reduce the amount of calculation.

[0068] In some embodiments, the pallet hole height measurement method provided by the present invention further includes:

[0069] Capture images of the pallet based on the depth camera to obtain a depth image;

[0070] Input the depth image into the trained semantic segmentation network to obtain the segmentation result;

[0071] The segmentation results are subjected to morphological dilation processing to obtain the tray outline.

[0072] Specifically, the forklift moves to the front of the pallet, turns on the depth camera installed on it to capture images of the pallet, and obtains a depth image. The depth image is input into a trained semantic segmentation network (such as DDRNet) to obtain the segmentation result.

[0073] The entire pallet is divided in the width direction, and only the pallet holes are divided in the height direction. This is different from the solution of dividing the entire pallet in the prior art, so that the center point of the pallet is closer to the center of the pallet hole in the height direction, and the height of the pallet hole is closer to the height of the pallet hole.

[0074] Since the edge of the segmentation result may be uneven, the segmentation result is subjected to morphological dilation processing to achieve edge smoothing, thereby obtaining the tray contour. The coordinate values ​​of the tray contour are recorded and arranged in coordinate order as a closed line.

[0075] The pallet hole height measurement method provided by the present invention acquires a depth image by collecting images of the pallet based on a depth camera; inputs the depth image into a trained semantic segmentation network to obtain a segmentation result; and performs morphological expansion processing on the segmentation result to achieve accurate acquisition of the pallet contour.

[0076] In some embodiments, before inputting the depth image into the trained semantic segmentation network and obtaining the segmentation result, the method further includes:

[0077] Manually annotate the collected pallet depth images to obtain the pallet segmentation dataset;

[0078] Enhance the pallet segmentation dataset;

[0079] The enhanced pallet segmentation dataset is input into the semantic segmentation network for training to obtain a trained semantic segmentation network.

[0080] Specifically, depth images of pallets in different postures, different lights, and different scenes are collected, and the collected depth images of pallets are manually annotated. Only the pallet holes are annotated in the height direction of the pallet holes, and the upper edge of the pallet is not annotated to obtain a pallet segmentation dataset.

[0081] The pallet segmentation dataset is subjected to enhancement processing such as rotation, translation, and scaling to expand the number of samples and improve the diversity of samples. The enhanced pallet segmentation dataset is then input into the semantic segmentation network for training to obtain a trained semantic segmentation network.

[0082] The pallet hole height measurement method provided by the present invention performs enhancement processing on the manually labeled pallet segmentation data set to expand the number of samples and improve the diversity of samples. The enhanced pallet segmentation data set is input into the semantic segmentation network for training to improve the segmentation accuracy of the semantic segmentation network.

[0083] In some embodiments, after the depth camera is used to capture an image of the pallet and obtain the depth image, the method further includes:

[0084] The depth image is filtered and mapped to a point cloud image to obtain a point cloud image.

[0085] Specifically, after the depth image is acquired, the depth image is filtered to remove noise points, and is mapped to a point cloud image to acquire a point cloud image for subsequent calculation of the height of the pallet hole.

[0086] In some embodiments, after acquiring the depth image, the acquired depth data is corrected according to the intrinsic parameters of the depth camera.

[0087] In some embodiments, the pallet hole height measurement method provided by the present invention further includes:

[0088] Report the pallet hole height to the forklift dispatch system;

[0089] When the height of the pallet hole is greater than the preset threshold, the forklift dispatching system receives the forklift command;

[0090] When the height of the pallet hole is less than or equal to the preset threshold, an abandon forking instruction issued by the forklift dispatching system is received.

[0091] Specifically, the pallet hole height is reported to the forklift dispatching system, and the back-end of the forklift dispatching system makes an over-limit judgment, that is, compares the size relationship between the pallet hole height and the preset threshold. When the pallet hole height is lower than or equal to the preset threshold, it is reported to the front-end of the forklift dispatching system to notify manual processing.

[0092] When the height of the pallet hole is greater than the preset threshold, the pallet can be forked smoothly and the forklift dispatching system issues a fork-picking instruction. Otherwise, the forklift dispatching system issues an abandonment fork-picking instruction and the forklift goes to identify the next pallet.

[0093] The preset threshold is related to the thickness of the fork arm and the flatness of the ground. The thinner the fork arm and the flatter the ground, the smaller the preset threshold, and vice versa.

[0094] The pallet hole height measurement method provided by the present invention reports the pallet hole height to the forklift dispatching system for forking decision, actively abandons the pallet that is too short, and reports it for manual processing, thereby facilitating human-machine collaborative operation in non-standard pallet scenarios.

[0095] Figure 4 FIG. 2 is a flow chart of the method for measuring the height of a tray hole provided by the present invention. Figure 4 As shown, the present invention provides a method for measuring the height of a pallet hole, comprising the following steps:

[0096] (1) Image acquisition: The forklift moves to the front of the pallet and turns on its installed depth camera to obtain a depth image.

[0097] (2) Pallet segmentation: The acquired depth image is input into the trained semantic segmentation network to obtain the segmentation result.

[0098] (3) Contour detection: Perform morphological dilation on the segmentation results to detect the contour of the tray.

[0099] (4) Minimum rectangular envelope calculation: Calculate the minimum circumscribed rectangular envelope of the pallet contour to obtain the pixel coordinates of the rectangular centroid, i.e., the center point of the pallet, and the rotation angle of the rectangular envelope.

[0100] (5) Straight line segment construction: Construct a straight line segment perpendicular to the long side of the rectangular envelope through the center point of the pallet. The row coordinates of the two end points of the straight line segment are the maximum and minimum row coordinates of the vertices of the rectangular envelope, respectively. The slope of the straight line segment is determined by the rotation angle of the rectangular envelope.

[0101] (6) Calculate the convex hull contour: Calculate the convex hull contour based on the pallet contour, which contains all the pixels inside the contour.

[0102] (7) Intersection calculation: Calculate the intersection of the convex hull contour and the straight line segment. Traverse two adjacent points in the convex hull contour and analyze whether the two points are on both sides of the straight line segment or on the straight line segment. If so, the contour edge formed by the two points has an intersection with the straight line segment. Then calculate the intersection of the contour edge and the straight line segment.

[0103] (8) Calculate the spatial distance between two intersection points: Use the intersection pixel coordinates to index into the point cloud coordinates, and calculate the spatial distance between the two intersection points based on the point cloud coordinates corresponding to the two intersection points.

[0104] (9) Calculation of pallet hole height: The spatial distance between the two intersection points is determined as the pallet hole height.

[0105] (10) Pallet hole height reporting: Report the pallet hole height to the forklift dispatch system.

[0106] (11) Judgment: Determine whether the height of the tray hole is less than or equal to a preset threshold.

[0107] (12) Receiving a fork pick-up instruction: When the height of the pallet hole is greater than a preset threshold, receiving a fork pick-up instruction.

[0108] (13) Receiving an abandonment forking instruction: When the height of the tray hole is less than or equal to a preset threshold, receiving an abandonment forking instruction.

[0109] The pallet hole height measuring device provided by the present invention is described below. The pallet hole height measuring device described below and the pallet hole height measuring method described above can correspond to each other.

[0110] Figure 5 Schematic diagram of the structure of the pallet hole height measuring device provided by the present invention. Figure 5 As shown, the present invention provides a pallet hole height measuring device, comprising:

[0111] A calculation module 510 is used to calculate the minimum circumscribed rectangular envelope of the pallet outline according to the pallet outline, and determine the rotation angle of the pallet center point and the rectangular envelope;

[0112] A construction module 520 is used to construct a straight line segment based on the center point of the tray and the rectangular envelope; the straight line segment passes through the center point of the tray, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope respectively;

[0113] A first determination module 530, configured to determine a convex hull contour based on the tray contour;

[0114] A second determination module 540 is used to determine two intersection points of the straight line segment and the convex hull contour;

[0115] The first acquisition module 550 is used to obtain the height of the tray hole according to the point cloud coordinates corresponding to the two intersection points.

[0116] In some embodiments, the second determining module 540 is specifically configured to:

[0117] Traversing two adjacent points in the convex hull contour;

[0118] When the two adjacent points are located on both sides of the straight line segment or on the straight line segment, an intersection point between the contour edge formed by the two adjacent points and the straight line segment is determined.

[0119] In some embodiments, the apparatus further comprises:

[0120] A second acquisition module is used to collect images of the pallet based on a depth camera to obtain a depth image;

[0121] A third acquisition module is used to input the depth image into a trained semantic segmentation network to obtain a segmentation result;

[0122] The fourth acquisition module is used to perform morphological dilation processing on the segmentation result to obtain the tray outline.

[0123] In some embodiments, the apparatus further comprises:

[0124] A fifth acquisition module is used to manually annotate the acquired pallet depth image to obtain a pallet segmentation data set;

[0125] An enhancement module, used for performing enhancement processing on the pallet segmentation data set;

[0126] The training module is used to input the enhanced pallet segmentation data set into the semantic segmentation network for training to obtain the trained semantic segmentation network.

[0127] In some embodiments, the apparatus further comprises:

[0128] The processing module is used to filter the depth image and map it to a point cloud image to obtain a point cloud image.

[0129] In some embodiments, the apparatus further comprises:

[0130] A reporting module, used for reporting the pallet hole height to a forklift dispatching system;

[0131] A first receiving module, configured to receive a forklift instruction issued by the forklift dispatching system when the height of the pallet hole is greater than a preset threshold;

[0132] The second receiving module is used to receive a forklift abandonment instruction issued by the forklift dispatching system when the height of the pallet hole is less than or equal to a preset threshold.

[0133] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the pallet hole height measurement method, which includes: according to the pallet contour, calculating the minimum circumscribed rectangular envelope of the pallet contour, and determining the rotation angle of the pallet center point and the rectangular envelope; constructing a straight line segment based on the pallet center point and the rectangular envelope; the straight line segment passes through the pallet center point, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are respectively the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope; determining the convex hull contour based on the pallet contour; determining the two intersection points of the straight line segment and the convex hull contour; and obtaining the pallet hole height according to the point cloud coordinates corresponding to the two intersection points.

[0134] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0135] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pallet hole height measurement method provided by the above methods, the method including: according to the pallet contour, calculating the minimum circumscribed rectangular envelope of the pallet contour, and determining the center point of the pallet and the rotation angle of the rectangular envelope; constructing a straight line segment based on the center point of the pallet and the rectangular envelope; the straight line segment passes through the center point of the pallet, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are respectively the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope; based on the pallet contour, determining the convex hull contour; determining the two intersection points of the straight line segment and the convex hull contour; and obtaining the pallet hole height according to the point cloud coordinates corresponding to the two intersection points.

[0136] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the pallet hole height measurement method provided by the above-mentioned methods, the method comprising: calculating the minimum circumscribed rectangular envelope of the pallet contour according to the pallet contour, and determining the center point of the pallet and the rotation angle of the rectangular envelope; constructing a straight line segment based on the center point of the pallet and the rectangular envelope; the straight line segment passes through the center point of the pallet, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are respectively the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope; determining a convex hull contour based on the pallet contour; determining two intersection points of the straight line segment and the convex hull contour; and obtaining the pallet hole height according to the point cloud coordinates corresponding to the two intersection points.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring the height of a pallet hole, characterized in that: include: According to the pallet outline, the minimum circumscribed rectangular envelope of the pallet outline is calculated, and the rotation angle of the pallet center point and the rectangular envelope is determined; Based on the center point of the tray and the rectangular envelope, a straight line segment is constructed; the straight line segment passes through the center point of the tray, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope. The slope of the straight line segment is ,in is the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope respectively; Based on the tray contour, determining a convex hull contour; Determine two intersection points of the straight line segment and the convex hull contour; The height of the tray hole is obtained according to the point cloud coordinates corresponding to the two intersection points.

2. The pallet hole height measurement method according to claim 1, characterized in that: Determining two intersection points of the straight line segment and the convex hull contour includes: Traversing two adjacent points in the convex hull contour; When the two adjacent points are located on both sides of the straight line segment or on the straight line segment, an intersection point between the contour edge formed by the two adjacent points and the straight line segment is determined.

3. The pallet hole height measurement method according to claim 1, characterized in that: The method further comprises: Capture images of the pallet based on the depth camera to obtain a depth image; Inputting the depth image into a trained semantic segmentation network to obtain a segmentation result; The segmentation result is subjected to morphological dilation processing to obtain the tray outline.

4. The pallet hole height measurement method according to claim 3, characterized in that: Before inputting the depth image into the trained semantic segmentation network to obtain the segmentation result, the method further includes: Manually annotate the collected pallet depth images to obtain the pallet segmentation dataset; performing enhancement processing on the pallet segmentation dataset; The enhanced pallet segmentation data set is input into the semantic segmentation network for training to obtain the trained semantic segmentation network.

5. The pallet hole height measurement method according to claim 3, characterized in that: After the depth camera is used to capture an image of the pallet and obtain a depth image, the method further includes: The depth image is filtered and mapped to a point cloud image to obtain a point cloud image.

6. The pallet hole height measurement method according to claim 1, characterized in that: The method further comprises: Reporting the pallet hole height to a forklift dispatching system; When the height of the pallet hole is greater than a preset threshold, receiving a forklift instruction issued by the forklift dispatching system; When the height of the pallet hole is less than or equal to a preset threshold, an abandonment forking instruction issued by the forklift dispatching system is received.

7. A pallet hole height measuring device, characterized in that: include: A calculation module, used to calculate the minimum circumscribed rectangular envelope of the pallet outline according to the pallet outline, and determine the rotation angle of the pallet center point and the rectangular envelope; A construction module is used to construct a straight line segment based on the center point of the tray and the rectangular envelope; the straight line segment passes through the center point of the tray, and the slope of the straight line segment is determined according to the rotation angle of the rectangular envelope. ,in is the rotation angle of the rectangular envelope; the row coordinates of the two end points of the straight line segment are the maximum and minimum values ​​of the row coordinates of the vertices of the rectangular envelope respectively; A first determination module, configured to determine a convex hull contour based on the tray contour; A second determination module is used to determine two intersection points of the straight line segment and the convex hull contour; The first acquisition module is used to obtain the height of the tray hole according to the point cloud coordinates corresponding to the two intersection points.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for measuring the height of a pallet hole according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for measuring the height of a pallet hole according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for measuring the height of a pallet hole according to any one of claims 1 to 6 is implemented.

Images