AGV visual detection method and device

Abstract

The application discloses an AGV visual detection method and device, and relates to the field of visual detection. In the moving process of an automatic guided vehicle, the point cloud model of each object in a preset area in front of the moving direction of the automatic guided vehicle is determined, the average distance between the point cloud in each point cloud model and the automatic guided vehicle is determined, the distance between each object and the automatic guided vehicle is obtained, and then it is judged whether the average distance is less than a preset distance. If the average distance is less than the preset distance, a preset obstacle avoidance task is executed. The point cloud model is obtained by continuously measuring the distance of the preset area, various objects in the area are identified according to the point cloud model, and the distance between the objects and the automatic guided vehicle is detected in the form of the point cloud distance. The influence of environmental illumination, image resolution and distortion can be avoided, the accuracy of object identification can be improved, and thus the automatic guided vehicle can execute the obstacle avoidance task in time, and the collision accident can be avoided.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and in particular to a visual inspection method and apparatus for AGVs. Background Technology

[0002] In the field of automated warehousing and logistics, Automated Guided Vehicles (AGVs) are widely used for cargo handling and warehouse operations. To accurately identify obstacles and objects such as goods in the working environment, cameras are typically installed on AGVs to capture images of various objects, and then image recognition is used to identify the objects within the camera's field of view.

[0003] When automated guided vehicles (AGVs) operate indoors, the camera is susceptible to the effects of ambient lighting, resulting in poor image quality. Furthermore, camera distortion and resolution also negatively impact image quality, which directly affects the accuracy of image recognition. If the AGV cannot promptly identify objects in front of it during its movement, a collision may occur. Summary of the Invention

[0004] The purpose of this invention is to provide an AGV visual inspection method and device that can avoid the influence of ambient light, image resolution and distortion, improve the accuracy of object recognition, and enable the automated guided vehicle to perform obstacle avoidance tasks in a timely manner to avoid collision accidents.

[0005] To solve the above-mentioned technical problems, the present invention provides an AGV visual inspection method, comprising:

[0006] During the movement of the automated guided vehicle, the following steps are performed:

[0007] A first point cloud model of each object in a preset area is determined; wherein, the preset area is the area in front of the moving direction of the automated guided vehicle;

[0008] For any of the first point cloud models, determine the average distance between all point clouds in the first point cloud model and the automated guided vehicle;

[0009] Determine whether the average distance is less than a preset distance;

[0010] If the average distance is less than the preset distance, then the preset obstacle avoidance task is executed;

[0011] If the average distance is not less than the preset distance, then the automated guided vehicle is controlled to move along the preset route.

[0012] On the one hand, when the automated guided vehicle uses a single-line radar to determine the first point cloud model, it determines the first point cloud model of each object in the preset area, including:

[0013] The ranging points obtained by the single-line radar scanning the preset area are acquired.

[0014] Determine the radar scanning angle and radar scanning distance corresponding to each of the ranging points;

[0015] Among all ranging points, those ranging points where the difference in radar scanning angle is less than a preset angle difference and the difference in radar scanning distance is less than a preset distance difference are clustered to obtain multiple clustering models.

[0016] Each of the clustering models is used as the first point cloud model for each of the objects.

[0017] On the one hand, when the automated guided vehicle uses multi-line radar to determine the first point cloud model, it determines the first point cloud model of each object in the preset area, including:

[0018] The ranging points obtained by the multi-line radar scanning the preset area are acquired.

[0019] Determine the first distance between the multi-line radar and the ground;

[0020] Based on the first distance and the radar scanning angle corresponding to each ranging point, the theoretical distance to the ground measured by each ranging point is determined respectively.

[0021] Obtain the actual distances of each of the ranging points actually measured by the multi-line radar;

[0022] For any of the distance measurement points, determine whether the difference between the theoretical distance and the actual distance is within a preset difference range;

[0023] If so, then the ranging point is determined to be a ranging point on the ground;

[0024] If not, then it is determined that the ranging point is not a ranging point on the ground;

[0025] Delete all the distance measurement points represented as the ground.

[0026] The first point cloud model of each object in the preset area is determined based on the remaining ranging points.

[0027] On the one hand, after determining the first distance between the multi-line radar and the ground, it also includes:

[0028] Delete all ranging points whose radar scanning angle is greater than a preset angle;

[0029] After deleting all the ranging points represented as the ground, the process also includes:

[0030] All ranging points whose radar scanning angle is greater than the preset angle are restored, and the process proceeds to the step of determining the first point cloud model of each object in the preset area based on the remaining ranging points.

[0031] On the one hand, determining the average distance between all point clouds in the first point cloud model and the automated guided vehicle includes:

[0032] For any of the objects, subtract the first point cloud model obtained in the previous determination from the first point cloud model obtained in this determination to obtain the difference frame point cloud model;

[0033] Determine the average distance between all point clouds in the difference frame point cloud model and the automated guided vehicle.

[0034] On the one hand, it also includes:

[0035] Before the automated guided vehicle performs the task of transporting pallets, the following steps are performed:

[0036] Determine the second point cloud model of the tray;

[0037] The second point cloud model is divided into multiple sub-point cloud models;

[0038] Determine the number of points in each of the sub-point cloud models;

[0039] The position of the pallet's forklift entry hole is determined based on the sub-point cloud model whose point cloud quantity is less than a preset quantity, so as to perform the preset pallet handling task.

[0040] On the one hand, when the automated guided vehicle (AGV) uses a single-line radar to determine a second point cloud model, and the single-line radar is installed on the forks of the AGV, determining the second point cloud model of the pallet includes:

[0041] S21: Control the forks of the automated guided vehicle to descend to the lowest height;

[0042] S22: Obtain the ranging point obtained by the single-line radar scanning the tray;

[0043] S23: Determine whether the forks of the automated guided vehicle have risen to the maximum height; if not, proceed to S24; if yes, proceed to S25.

[0044] S24: Control the forks of the automated guided vehicle to rise to a preset height, then return to S22;

[0045] S25: Determine the second point cloud model of the pallet based on all the ranging points.

[0046] On the one hand, determining the location of the pallet's forklift entry hole based on the sub-point cloud model whose point cloud quantity is less than a preset number includes:

[0047] The model boundary between a sub-point cloud model with fewer point clouds than the preset number and an adjacent sub-point cloud model with more than the preset number of point clouds is used as the boundary contour of the fork hole.

[0048] The center point of the forklift hole is determined based on the boundary profile.

[0049] The position of the fork entrance hole is determined based on the coordinates of the center point in a preset coordinate system, wherein the horizontal axis of the preset coordinate system is the x-axis and the vertical axis is the y-axis.

[0050] This application also provides an AGV visual inspection device, including:

[0051] Memory, used to store computer programs;

[0052] A processor is used to execute the computer program to implement the steps of the AGV visual inspection method described above.

[0053] The beneficial effects of this application are that it provides an AGV visual inspection method and device. During the movement of the automated guided vehicle (AGV), by determining the point cloud models of various objects in a preset area ahead of the AGV's movement direction, and determining the average distance between each point cloud model and the AGV, the distance between each object and the AGV is obtained. Then, it is determined whether any average distance is less than a preset distance; if so, a preset obstacle avoidance task is executed. By continuously measuring distances within the preset area to obtain the point cloud model, and identifying various objects within the area based on the point cloud model, the method of detecting whether any object is too close to the AGV through point cloud distance detection avoids the influence of ambient lighting, image resolution, and distortion, thus improving the accuracy of object recognition. This allows the AGV to perform obstacle avoidance tasks in a timely manner, preventing collisions. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the prior art and embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart of an AGV visual inspection method provided in this application;

[0056] Figure 2A frontal schematic diagram of a radar installation location provided in this application;

[0057] Figure 3 A side view of a radar installation location provided in this application;

[0058] Figure 4 A top view diagram of a radar installation location provided in this application;

[0059] Figure 5 This is a schematic diagram of the structure of an AGV vision inspection system provided in this application;

[0060] Figure 6 A flowchart of a method for determining a second point cloud model based on single-line radar provided in this application;

[0061] Figure 7 A flowchart illustrating a method for locating a forklift entry hole provided in this application;

[0062] Figure 8 A flowchart of another AGV visual inspection method provided in this application;

[0063] Figure 9 This is a schematic diagram of the structure of an AGV visual inspection device provided in this application. Detailed Implementation

[0064] The core of this invention is to provide an AGV visual inspection method and device that can avoid the influence of ambient light, image resolution and distortion, improve the accuracy of object recognition, and enable the automated guided vehicle to perform obstacle avoidance tasks in a timely manner to avoid collision accidents.

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Please refer to Figure 1 , Figure 1 A flowchart of an AGV visual inspection method provided in this application includes:

[0067] During the movement of the automated guided vehicle, the following steps are performed:

[0068] S1: Determine the first point cloud model of each object in the preset area; wherein, the preset area is the area in front of the moving direction of the automated guided vehicle;

[0069] S2: For any first point cloud model, determine the average distance between all point clouds in the first point cloud model and the automated guided vehicle;

[0070] S3: Determine if there is an average distance that is less than a preset distance;

[0071] S4: If the average distance is less than the preset distance, execute the preset obstacle avoidance task;

[0072] S5: If there is no average distance less than the preset distance, control the automatic guided vehicle to move along the preset route.

[0073] Given that existing technologies relying solely on cameras for image recognition are susceptible to ambient lighting conditions, making it difficult to identify objects in the environment surrounding the automated guided vehicle (AGV), and considering that the AGV does not need to perform loading / unloading tasks but only needs to move along a preset route, it is not necessary to actually identify the specific types of objects near the AGV. It is only necessary to know the distance between these objects and the AGV, and whether these objects will obstruct the AGV's path, facilitating timely obstacle avoidance maneuvers.

[0074] Please refer to Figure 5 , Figure 5 This application provides a schematic diagram of the structure of an AGV vision inspection system. The calculation module is mainly used to perform steps such as determining the point cloud model and calculating distances; the communication module is used to communicate with other modules and devices in the automated guided vehicle; the control module is used to control the movement, obstacle avoidance, and loading and unloading operations of the automated guided vehicle; the sensor module includes a lidar for determining the point cloud model, a height encoder for controlling the lifting of the forks, and an infrared sensor for determining the distance to the pallet.

[0075] First, based on the size and speed of the automated guided vehicle (AGV), a pre-defined area is defined as the region in front of the AGV's direction of movement. This pre-defined area represents the region where objects may obstruct the AGV's movement. For example, the AGV's direction of movement can be used as the centerline, and a reasonable distance can be used as the radius to determine a fan-shaped area centered on the AGV and located in front of the AGV's direction of movement as the pre-defined area. Alternatively, a semi-circular area centered on the AGV can be directly used as the pre-defined area.

[0076] During the movement of the automated guided vehicle (AGV), radar continuously scans the pre-defined area. Based on the reflections from the ranging points of various objects within the area, the first point cloud model of each object is determined. The distance between each point cloud in the object's first point cloud model and the AGV is calculated, and then the average distance of all point clouds belonging to the same object is calculated as the distance between the object itself and the AGV. If any object is too close to the AGV (average distance less than the pre-defined distance), it indicates that at least one object in the pre-defined area poses a risk of obstructing the AGV's movement. In this case, the vehicle control system in the AGV needs to be notified immediately so that it can take timely obstacle avoidance actions. For example, if the pre-defined area is a semi-circular area with a radius of 3 meters, and an object is detected 2 meters away at a 30-degree angle to the right of the AGV, while no object is detected to the left of the AGV, the processor notifies the vehicle control system. Upon receiving this information, the vehicle control system can promptly control the AGV to turn left to avoid the obstacle.

[0077] It should be noted that in this application, the terms "range point" and "point cloud" have the same meaning. When a radar measures the distance to a point, the point being measured is called the range point. When the range point is used to determine the point cloud model, the points in the point cloud model are called the point cloud.

[0078] In summary, during the movement of the automated guided vehicle (AGV), by determining the first point cloud model of each object in a preset area ahead of the AGV's direction of movement, and calculating the average distance between all points in each first point cloud model and the AGV, the distance between each object and the AGV is obtained. Then, it is determined whether any average distance is less than a preset distance. If so, a preset obstacle avoidance task is executed. By continuously measuring distances within the preset area to obtain the first point cloud model, and using this model to identify various objects within the area, and by detecting whether any object is too close to the AGV using point cloud distances, the influence of ambient lighting, image resolution, and distortion can be avoided, thus improving the accuracy of object recognition. This allows the AGV to execute obstacle avoidance tasks in a timely manner, preventing collisions.

[0079] Based on the above embodiments:

[0080] In some embodiments, when an automated guided vehicle uses a single-line radar to determine a first point cloud model, determining the first point cloud model of each object in a preset area includes:

[0081] Acquire ranging points obtained by a single-line radar scanning a preset area;

[0082] Determine the radar scanning angle and radar scanning distance corresponding to each ranging point;

[0083] Among all ranging points, those ranging points whose radar scanning angle difference is less than a preset angle difference and whose radar scanning difference is less than a preset difference are clustered to obtain multiple clustering models.

[0084] Each clustering model is used as the first point cloud model for each object.

[0085] To accurately determine the first point cloud model of an object, this application uses a single-line radar. Since the scanning area of ​​the single-line radar is a straight line, it can only obtain ranging points in one dimension of three-dimensional space. Therefore, a straight line perpendicular to the direction of movement of the automated guided vehicle and parallel to the ground is needed as the scanning area of ​​the single-line radar. During scanning, the single-line radar emits ranging lasers at different scanning angles. When the ranging laser contacts an object in the preset area, it returns to the single-line radar, which then calculates the distance between itself and the ranging point.

[0086] Considering that multiple objects in close proximity may exist in real-world applications (such as stacked goods), preset angle differences and preset distance differences are predefined to avoid misclassifying multiple identical objects as a single object. If the radar scan distance difference between two ranging points is less than the preset distance difference, it indicates that the distance between these two ranging points and the radar is relatively close. If the radar scan angle difference between two ranging points is less than the preset angle difference, it indicates that these two ranging points are relatively close to each other and indeed belong to the same object. Combining these two factors, if the radar scan distance difference between two ranging points is less than the preset distance difference and the radar scan angle difference is less than the preset angle difference, it indicates that these two ranging points are ranging points on the same object.

[0087] If the difference in radar scanning distance between two ranging points is greater than the preset distance difference but the difference in radar scanning angle is less than the preset angle difference, it means that these two ranging points do not belong to the same object, but rather to two objects that are approximately angularly close to the automated guided vehicle but at different distances (e.g., two objects stacked one in front of the other). If the difference in radar scanning distance between two ranging points is less than the preset distance difference but the difference in radar scanning angle is greater than the preset angle difference, it means that these two ranging points do not belong to the same object, but rather to two objects that are approximately angularly close to the automated guided vehicle but at different positions (e.g., two objects stacked one on the left and one on the right).

[0088] In this way, a distance measuring point only calculates its distance relative to other nearby distance measuring points, avoiding comparisons with distant points. This prevents multiple identical objects placed at different locations and distances from being mistakenly identified as a single object.

[0089] In some embodiments, when an automated guided vehicle uses multi-line radar to determine a first point cloud model, determining the first point cloud model of each object in a preset area includes:

[0090] Acquire ranging points obtained by multi-line radar scanning a preset area;

[0091] Determine the initial distance between the multi-line radar and the ground;

[0092] Based on the first distance and the radar scanning angle of each ranging point, the theoretical distance to the ground measured by each ranging point is determined respectively;

[0093] Obtain the actual distances of each ranging point measured by the multi-line radar;

[0094] For any distance measurement point, determine whether the difference between the theoretical distance and the actual distance is within the preset difference range;

[0095] If so, then the distance measuring point is determined to be the distance measuring point on the ground;

[0096] If not, then the distance measuring point is determined to be a distance measuring point on the ground;

[0097] Delete all distance measurement points represented as the ground;

[0098] The first point cloud model of each object in the preset area is determined based on the remaining ranging points.

[0099] To reduce computational load, this application employs a multi-line radar to determine the point cloud model. Since the scanning area of ​​a multi-line radar is a plane, it's equivalent to multiple single-line radars scanning simultaneously, resulting in a large number of ranging points. Using all these ranging points for subsequent calculations would consume significant computational resources. Considering that scanning a preset area will yield ranging points for the ground portion of that area, and since the ground portion has more point clouds than the object point clouds within the preset area, and these ground point clouds offer no assistance to the automated guided vehicle's obstacle avoidance, it is necessary to filter out the ground portion of the point cloud.

[0100] After the multi-line radar scans a preset area and obtains a large number of ranging points, the height (first distance) between the multi-line radar itself and the ground is first determined. A triangle is defined by the radar itself, the ground coordinates perpendicular to the radar, and the ranging points. The hypotenuse of the triangle is the ranging laser path corresponding to that ranging point. Since the scanning angle of each ranging point is different, the length of the ranging laser path can be determined according to the sine formula. When calculating the length, the ranging point is assumed to be on the ground for calculation, and the length of the ranging laser path when the ranging point is assumed to be on the ground is obtained, which is the theoretical distance measured by the ranging point to the ground. Then, the actual distance of the ranging point calculated by the radar is obtained. If the difference between the actual distance and the theoretical distance is not significant, it means that the ranging point has indeed reached the ground and returned to the radar. Otherwise, it means that the ranging point has prematurely contacted an object and returned to the radar prematurely.

[0101] After confirming that a distance measurement point truly reaches the ground, that point is deleted. In this way, all distance measurement points representing the ground are removed, leaving only the distance measurement points of the objects. Finally, the first point cloud model of each object is determined based on these remaining distance measurement points. This method significantly reduces the number of distance measurement points and the amount of computation.

[0102] In some embodiments, after determining the first distance between the multi-line radar and the ground, the method further includes:

[0103] Delete all ranging points whose radar scanning angle is greater than the preset angle;

[0104] After deleting the distance measurement points represented as the ground, the following is also included:

[0105] All ranging points with radar scanning angles greater than a preset angle are restored, and the process proceeds to the step of determining the first point cloud model of each object in the preset area based on the remaining ranging points.

[0106] To further reduce computational load, in this application, when determining which ranging points are ground-based ranging points, ranging points that obviously will not touch the ground are first masked. Specifically, based on the radar scanning angle, with the perpendicular line between the radar and the ground as 0 degrees, the angle at which the radar measures the farthest point of the preset area is taken as the preset angle. Then, all ranging points with radar scanning angles greater than this preset angle are deleted. The remaining ranging points are represented as ranging points that may touch the ground within the preset area. These ranging points are then used to determine which ranging points are truly represented as ground-based ranging points.

[0107] After deleting all ranging points represented as the ground, the ranging points that were deleted due to excessive radar scanning angle are restored before proceeding to the next step of determining the first point cloud model. This method effectively reduces computational load while maintaining accuracy.

[0108] In some embodiments, determining the average distance between all point clouds in the first point cloud model and the automated guided vehicle includes:

[0109] For any object, subtract the first point cloud model obtained in the previous determination from the first point cloud model obtained in this determination to obtain the difference frame point cloud model;

[0110] Determine the average distance between all point clouds in the difference frame point cloud model and the automated guided vehicle.

[0111] To reduce computational load, please refer to the following in this application: Figure 8 , Figure 8 This is a flowchart of another AGV visual inspection method provided in this application. During the movement of the automated guided vehicle (AGV), the radar continuously detects the first point cloud model of each object within a preset area. When the AGV's speed is slow, the positional difference between two consecutive first point cloud models of the same object is not significant. If the complete first point cloud model is used in subsequent calculation steps each time, it would consume considerable computational resources. Therefore, after determining the first point cloud model of an object, it is subtracted from the previously obtained first point cloud model. The remaining point cloud represents the positional change of the object's first point cloud model during the AGV's movement, and the subtracted point cloud portion is equivalent to the overlapping part with the previously obtained first point cloud model.

[0112] In practical applications, the remaining point cloud is used as the difference frame point cloud model to replace the original first point cloud model in subsequent calculation processes.

[0113] In some embodiments, it also includes:

[0114] Before the automated guided vehicle performs the task of transporting pallets, the following steps are performed:

[0115] Determine the second point cloud model of the pallet;

[0116] The second point cloud model is divided into multiple sub-point cloud models;

[0117] Determine the number of points in each sub-point cloud model;

[0118] The location of the pallet's forklift entry hole is determined based on the number of sub-point cloud models with fewer point cloud elements than a preset number, so as to perform the preset pallet handling task.

[0119] Considering that existing technologies also use a camera to perform image recognition to locate the fork entry holes on a pallet, they are also easily affected by the shooting angle and ambient lighting, making it difficult to determine the actual location of the fork entry holes.

[0120] In this application, firstly, after the automated guided vehicle arrives in front of the pallet to be transported, before starting to transport the pallet, the pallet is also measured by ranging to obtain multiple ranging points of the pallet. Based on these ranging points, a second point cloud model of the pallet is determined. Because the forklift holes are recessed areas on the pallet, the ranging points of the forklift holes on the pallet surface are farther away than the ranging points of other parts of the pallet surface. Moreover, due to the radar height and radar scanning angle, there are usually areas inside the forklift holes that cannot be scanned, so the number of point clouds in the forklift hole area is usually less than the number of point clouds in other parts.

[0121] Based on this characteristic, the second point cloud model of the pallet is divided into multiple sub-point cloud models. Each sub-point cloud model represents a small area on the pallet. The number of points in each sub-point cloud model without holes is pre-calculated. A reasonable preset number is determined based on the number of points in these sub-point cloud models, representing the threshold number of points in a sub-point cloud model when there are no forklift entry holes. In practical applications, the number of points in each sub-point cloud model is calculated separately. If the number of points in any sub-point cloud model is less than the preset number, it is determined that the sub-point cloud model contains a forklift entry hole. In this way, the actual location of the forklift entry hole can be accurately determined.

[0122] In some embodiments, when the automated guided vehicle (AGV) uses a single-line radar to determine a second point cloud model, and the single-line radar is mounted on the AGV's forks, determining the second point cloud model of the pallet includes:

[0123] S21: Control the forks of the automated guided vehicle to descend to the lowest height;

[0124] S22: Acquire the ranging point obtained by scanning the tray with a single-line radar;

[0125] S23: Determine whether the forks of the automated guided vehicle have risen to the maximum height; if not, proceed to S24; if yes, proceed to S25.

[0126] S24: Control the forks of the automated guided vehicle to rise to a preset height, then return to S22;

[0127] S25: Determine the second point cloud model of the pallet based on all ranging points.

[0128] To accurately determine the second point cloud model, please refer to the following in this application. Figure 2 , Figure 3 , Figure 4 and Figure 6 , Figure 2 This is a frontal schematic diagram of a radar installation location provided in this application. Figure 3This is a side view of a radar installation location provided in this application. Figure 4 This is a top view diagram of a radar installation location provided in this application. Figure 6 This application provides a flowchart of a method for determining a second point cloud model based on a single-line radar. As can be seen from the above embodiments, a single-line radar can only scan a straight line. To obtain the ranging points across the entire plane of the pallet, the single-line radar needs to be moved. This application mounts the single-line radar on the forks of an automated guided vehicle (AGV), specifically below the base of the forks, to avoid obstruction by the pallet or goods after the AGV picks up the pallet. The position of the single-line radar is changed by controlling the movement of the forks. Additionally, for ease of movement, a lifting structure specifically for moving the single-line radar up and down can also be installed on the AGV, such as... Figures 2-4 This is an example of using a lifting structure.

[0129] according to Figures 2-4 and Figure 6 It can be seen that when using a single-line radar for scanning, the height of the automated guided vehicle's lifting structure / forks is first lowered to its lowest point, allowing the single-line radar to scan the bottom of the pallet. After scanning, the lifting structure / forks are raised to a certain height, causing the single-line radar to scan the pallet again. This process is repeated, with the single-line radar scanning the pallet once each time the lifting structure / forks are raised a certain height. If the height value in the height encoder remains unchanged, it indicates that the lifting structure / forks have been raised to their highest point. All the ranging points obtained from the scans are then merged to obtain the second point cloud model of the pallet. In this way, the second point cloud model can be determined simply and accurately.

[0130] In addition to scanning from low to high, it is also possible to scan from high to low.

[0131] In some embodiments, determining the location of the pallet forklift opening based on a sub-point cloud model with a point cloud quantity less than a preset number includes:

[0132] The boundary between a sub-point cloud model with fewer than a preset number of point clouds and an adjacent sub-point cloud model with more than a preset number of point clouds is used as the boundary contour of the fork hole.

[0133] Determine the center point of the fork entrance hole based on the boundary profile;

[0134] The position of the fork entrance hole is determined based on the coordinates of the center point in a preset coordinate system, where the horizontal axis of the preset coordinate system is the x-axis and the vertical axis is the y-axis.

[0135] To easily and accurately determine the location of the fork entry hole, please refer to the following in this application. Figure 7 , Figure 7 A flowchart illustrating a method for locating a forklift entry hole provided in this application. First, it can be determined using a method similar to single-line radar scanning. Specifically, Figure 7 The background filtering mentioned refers to filtering out the point cloud of the non-pallet area obtained by radar scanning. It mainly uses a pass-through filtering method to select the region of interest, and then uses outlier filtering to remove noise to obtain a second point cloud model.

[0136] Then, according to the distribution of each point cloud in the second point cloud model, these point clouds are sorted in order from left to right or right to left, and from top to bottom or bottom to top. Point clouds located in the same row or column are then treated as a whole, and the number of point clouds in one row / column is subtracted from the number of point clouds in the next row / column. In sub-point cloud models without forkholes, the difference obtained by subtraction is usually close to 0. As the sequence proceeds, if the absolute value of the difference obtained by subtraction is greater than the first preset difference / second preset difference, it means that the point cloud in the next row / column belongs to the sub-point cloud model of the forkhole, and this row / column needs to be used as the boundary of the forkhole. To reduce the amount of computation, the sub-point cloud model can be voxelized. Specifically, a position mean point is calculated for the point clouds in each sub-point cloud model, and this mean point is used to replace the entire sub-point cloud model, which can significantly reduce the number of point clouds. In this way, the boundary contours of the four sides of the forklift hole can be determined. By determining the center coordinates of the forklift hole through the boundaries of the four sides, the location of the forklift hole can be accurately obtained.

[0137] Please refer to Figure 9 , Figure 9 A schematic diagram of the structure of an AGV visual inspection device provided in this application includes:

[0138] Memory 21 is used to store computer programs;

[0139] The processor 22 is used to execute computer programs to implement the steps of the AGV visual inspection method described above.

[0140] For a detailed description of the AGV visual inspection device provided in this application, please refer to the embodiments of the AGV visual inspection method described above; further details will not be repeated here.

[0141] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An AGV visual inspection method, characterized in that, include: During the movement of the automated guided vehicle, the following steps are performed: A first point cloud model of each object in a preset area is determined; wherein, the preset area is the area in front of the moving direction of the automated guided vehicle; For any of the first point cloud models, determine the average distance between all point clouds in the first point cloud model and the automated guided vehicle; Determine whether the average distance is less than a preset distance; If the average distance is less than the preset distance, then the preset obstacle avoidance task is executed; If the average distance is not less than the preset distance, then control the automated guided vehicle to move along the preset route; When the automated guided vehicle uses a single-line radar to determine the first point cloud model, it determines the first point cloud model of each object in the preset area, including: The ranging points obtained by the single-line radar scanning the preset area are acquired. Determine the radar scanning angle and radar scanning distance corresponding to each of the ranging points; Among all ranging points, those ranging points where the difference in radar scanning angle is less than a preset angle difference and the difference in radar scanning distance is less than a preset distance difference are clustered to obtain multiple clustering models. Each of the clustering models is used as the first point cloud model for each of the objects. When the automated guided vehicle uses multi-line radar to determine the first point cloud model, it determines the first point cloud model of each object in the preset area, including: The ranging points obtained by the multi-line radar scanning the preset area are acquired. Determine the first distance between the multi-line radar and the ground; Based on the first distance and the radar scanning angle corresponding to each ranging point, the theoretical distance to the ground measured by each ranging point is determined respectively. Obtain the actual distances of each of the ranging points actually measured by the multi-line radar; For any of the distance measurement points, determine whether the difference between the theoretical distance and the actual distance is within a preset difference range; If so, then the ranging point is determined to be a ranging point on the ground; If not, then it is determined that the ranging point is not a ranging point on the ground; Delete all the distance measurement points represented as the ground. The first point cloud model of each object in the preset area is determined based on the remaining ranging points.

2. The AGV visual inspection method as described in claim 1, characterized in that, After determining the first distance between the multi-line radar and the ground, the process also includes: Delete all ranging points whose radar scanning angle is greater than a preset angle; After deleting all the ranging points represented as the ground, the process also includes: All ranging points whose radar scanning angle is greater than the preset angle are restored, and the process proceeds to the step of determining the first point cloud model of each object in the preset area based on the remaining ranging points.

3. The AGV visual inspection method as described in claim 1, characterized in that, Determining the average distance between all point clouds in the first point cloud model and the automated guided vehicle includes: For any of the objects, subtract the first point cloud model obtained in the previous determination from the first point cloud model obtained in this determination to obtain the difference frame point cloud model; Determine the average distance between all point clouds in the difference frame point cloud model and the automated guided vehicle.

4. The AGV visual inspection method according to any one of claims 1 to 3, characterized in that, Also includes: Before the automated guided vehicle performs the task of transporting pallets, the following steps are performed: Determine the second point cloud model of the tray; The second point cloud model is divided into multiple sub-point cloud models; Determine the number of points in each of the sub-point cloud models; The position of the pallet's forklift entry hole is determined based on the sub-point cloud model whose point cloud quantity is less than a preset quantity, so as to perform the preset pallet handling task.

5. The AGV visual inspection method as described in claim 4, characterized in that, When the automated guided vehicle (AGV) uses a single-line radar to determine a second point cloud model, and the single-line radar is mounted on the forks of the AGV, determining the second point cloud model of the pallet includes: S21: Control the forks of the automated guided vehicle to descend to the lowest height; S22: Obtain the ranging point obtained by the single-line radar scanning the tray; S23: Determine whether the forks of the automated guided vehicle have risen to the maximum height; if not, proceed to S24; if yes, proceed to S25. S24: Control the forks of the automated guided vehicle to rise to a preset height, then return to S22; S25: Determine the second point cloud model of the pallet based on all the ranging points.

6. The AGV visual inspection method as described in claim 4, characterized in that, Determining the location of the pallet forklift hole based on the sub-point cloud model whose point cloud quantity is less than a preset number includes: The model boundary between a sub-point cloud model with fewer point clouds than the preset number and an adjacent sub-point cloud model with more than the preset number of point clouds is used as the boundary contour of the fork hole. The center point of the forklift hole is determined based on the boundary profile. The position of the fork entrance hole is determined based on the coordinates of the center point in a preset coordinate system, wherein the horizontal axis of the preset coordinate system is the x-axis and the vertical axis is the y-axis.

7. An AGV visual inspection device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the AGV visual inspection method as described in any one of claims 1 to 6 when executing the computer program.

Images