Ground filtration and obstacle detection methods, devices, electronic equipment and storage media

Abstract

This application relates to a ground filtering and obstacle detection method, apparatus, electronic device, and storage medium. The method includes: extracting a second point cloud data representing a region of interest from a first point cloud data representing the environment in which the transport equipment is located; detecting a slope within the environment using a laser scanning device to obtain a slope detection result; filtering out ground laser points from the second point cloud data based on the slope detection result to obtain non-ground laser points in the second point cloud data; performing clustering processing on the non-ground laser points to obtain a clustering result; if the clustering result indicates that the non-ground laser points cluster into different obstacle clusters, then it is determined that an obstacle exists in the environment in which the transport equipment is located. This method can improve the accuracy of obstacle detection.

Description

Technical Field

[0001] This application relates to the field of forklift technology, and in particular to a ground filtering and obstacle detection method, apparatus, electronic device and storage medium. Background Technology

[0002] Automated Guided Vehicles (AGVs), also known as unmanned forklifts, are intelligent industrial robot vehicles. As key equipment for modern logistics automation and flexible manufacturing, AGVs are a hot application area in intelligent logistics. Integrating forklift technology and AGV technology, AGVs utilize various advanced guidance technologies, mapping algorithms, embedded vehicle software, and safety avoidance technologies to achieve automated guidance, handling, and stacking functions, thus realizing unmanned operation of the forklift. The application of AGVs can solve problems such as large material flow and high labor intensity in manual handling during industrial production.

[0003] Currently, unmanned forklifts mainly rely on 2D LiDAR for obstacle detection. However, 2D LiDAR can only detect a single plane and cannot adapt to complex industrial environments (such as platforms and slopes), leading to inaccurate obstacle detection and consequently reducing the handling efficiency of unmanned forklifts. Therefore, improving the accuracy of obstacle detection has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] Therefore, it is necessary to provide a ground filtering and obstacle detection method, apparatus, electronic device, and storage medium that can improve the accuracy of obstacle detection, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a ground filtering and obstacle detection method. The method includes:

[0006] Extract the second point cloud data of the region of interest from the first point cloud data of the environment where the handling equipment is located;

[0007] The slope in the environment where the transport equipment is located is detected by laser scanning equipment, and the slope detection results are obtained.

[0008] Based on the slope detection results, the ground laser points in the second point cloud data are filtered out to obtain the non-ground laser points in the second point cloud data;

[0009] Clustering is performed on the non-ground laser points to obtain the clustering results;

[0010] If the clustering result indicates that the non-ground laser points are clustered into different obstacle clusters, then it is determined that there are obstacles in the environment where the handling equipment is located.

[0011] In one embodiment, prior to extracting second point cloud data of the region of interest from the first point cloud data of the environment in which the handling equipment is located, the method further includes:

[0012] The laser scanning device scans the environment in which the conveying equipment is located to obtain raw point cloud data.

[0013] The original point cloud data is flipped vertically to obtain the first point cloud data.

[0014] In one embodiment, the step of extracting the second point cloud data of the region of interest from the first point cloud data of the environment in which the handling equipment is located includes:

[0015] Obtain the laser point that is farthest from the center of the first point cloud data;

[0016] The coverage area of ​​the first point cloud data is determined based on the farthest laser point;

[0017] The region of interest is determined based on the coverage area and the preset area; wherein, the preset area is at least one of the scanning blind zone of the laser scanning device that performs laser scanning to obtain the first point cloud data and the area where the point cloud is densely distributed in the first point cloud data.

[0018] Based on the region of interest and the first point cloud data, the second point cloud data is determined.

[0019] In one embodiment, the slope detection result includes a first detection result indicating the presence of a slope or a second detection result indicating the absence of a slope;

[0020] The process of obtaining slope detection results through laser scanning equipment includes:

[0021] The slope is detected by the laser beam emitted by the laser scanning device, and the slope detection result is obtained.

[0022] If the laser beam shows a break, the slope detection result is the first detection result; if the laser beam does not show a break, the slope detection result is the second detection result.

[0023] In one embodiment, the step of filtering out ground laser points in the second point cloud data based on the slope detection results to obtain non-ground laser points in the second point cloud data includes:

[0024] A fabric particle model is established based on the second point cloud data;

[0025] A corresponding motion model is established for each cloth particle in the cloth particle model, and the falling time of each cloth particle is determined according to the slope detection results; wherein, each cloth particle is a laser point in the second point cloud data;

[0026] The initial cloth plane is determined based on the motion model, and the initial cloth plane is placed at the highest point of the second point cloud data;

[0027] The second point cloud data and each of the fabric particles are projected onto the initial fabric plane;

[0028] Based on the falling time and the motion model, the height of each cloth particle after displacement is determined, and the non-ground laser points in each cloth particle are determined according to the height after displacement.

[0029] In one embodiment, determining the non-ground laser points in each of the fabric particles based on the height after displacement includes:

[0030] Obtain the height of the nearest radar point for each cloth particle; where the nearest radar point is the nearest neighbor particle of each cloth particle.

[0031] The cloth particles whose height after displacement is less than or equal to the height of the nearest radar point are marked as immovable points, and the cloth particles whose height after displacement is greater than the height of the nearest radar point are marked as movable points.

[0032] Calculate the first displacement between each movable point and the first preset point, calculate the second displacement between each movable point, calculate the third displacement between each movable point and the second preset point, and calculate the fourth displacement between each movable point and the immovable point; wherein, the first preset point is a neighboring particle point of the movable point in the fabric particle, and the second preset point is a neighboring particle point of the immovable point in the fabric particle.

[0033] The above steps are repeated until a preset condition is met; wherein, the preset condition is reaching a preset number of cycles, or the target displacement is less than a preset displacement threshold, and the target displacement is the first displacement, the second displacement, the third displacement, or the fourth displacement;

[0034] Based on the height difference between the displaced fabric particles and the third preset point, the non-ground laser points in each fabric particle are determined; wherein, the third preset point is the closest fabric particle to the fabric particle.

[0035] In one embodiment, determining the non-ground laser points in each of the fabric particles based on the height difference between the displaced fabric particles and the third preset point includes:

[0036] Calculate the height difference between each fabric particle and the corresponding third preset point;

[0037] If the height difference is greater than a preset height threshold, then the fabric particles corresponding to the height difference greater than the preset height threshold are regarded as non-ground laser points.

[0038] Secondly, this application also provides a ground filtering and obstacle detection device. The device includes:

[0039] The extraction module is used to extract the second point cloud data of the region of interest based on the first point cloud data of the environment in which the handling equipment is located;

[0040] A slope detection module is used to perform slope detection using a laser scanning device to obtain slope detection results; the slope is located in the environment where the transport equipment is situated.

[0041] The point cloud ground point filtering module is used to perform point cloud ground point filtering on the second point cloud data based on the slope detection results to obtain non-ground points in the second point cloud data.

[0042] The clustering module is used to perform clustering processing on the non-ground points to obtain clustering results;

[0043] The judgment module is used to determine whether there are obstacles in the environment where the handling equipment is located if the clustering result indicates that the non-ground points are clustered into different obstacle clusters.

[0044] Thirdly, this application also provides an electronic device. The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the aforementioned ground filtering and obstacle detection method.

[0045] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the aforementioned ground filtering and obstacle detection method.

[0046] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the aforementioned ground filtering and obstacle detection method.

[0047] The aforementioned ground filtering and obstacle detection method, apparatus, electronic device, and storage medium extract second point cloud data of the region of interest from the first point cloud data of the environment where the transport equipment is located. Then, the slope in the environment where the transport equipment is located is detected by a laser scanning device to obtain the slope detection result. Based on the slope detection result, the ground laser points in the second point cloud data are filtered out to obtain the non-ground laser points in the second point cloud data. Then, the non-ground laser points are clustered to obtain the clustering result. If the clustering result indicates that the non-ground laser points are clustered into different obstacle clusters, then it is determined that there are obstacles in the environment where the transport equipment is located. The technical solution of this application extracts second point cloud data of the region of interest from the first point cloud of the environment where the handling equipment is located, avoiding the situation in the prior art where only one plane can be detected. This improves the adaptability to the environment where the handling equipment is located, thereby improving the accuracy of obstacle detection. Furthermore, by detecting the slope in the environment where the handling equipment is located, the slope detection results are obtained to determine whether a slope exists in the environment where the handling equipment is located. This facilitates the identification of non-ground laser points in the second point cloud data, realizing ground filtering of the point cloud data. This further facilitates the determination of whether there are obstacles in the environment where the handling equipment is located based on non-ground laser points, further improving the accuracy of obstacle detection and thus improving the handling efficiency of the handling equipment. Attached Figure Description

[0048] Figure 1 This is an application environment diagram of the ground filtering and obstacle detection method in one embodiment;

[0049] Figure 2 This is a flowchart illustrating a ground filtering and obstacle detection method in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the ground filtering and obstacle detection method in another embodiment;

[0051] Figure 4 This is a schematic diagram of the fabric particle distribution in one embodiment;

[0052] Figure 5 This is a flowchart illustrating the steps for determining non-ground laser points in each fabric particle in one embodiment.

[0053] Figure 6 This is a schematic diagram of the fan-shaped distribution of fabric particles in one embodiment;

[0054] Figure 7 This is a flowchart illustrating the steps for determining non-ground laser points in each fabric particle in another embodiment.

[0055] Figure 8 This is a flowchart illustrating the ground filtering and obstacle detection method in another embodiment;

[0056] Figure 9 This is a structural block diagram of a ground filtering and obstacle detection device in one embodiment;

[0057] Figure 10 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The ground filtering and obstacle detection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on another network server. Terminal 102 extracts second point cloud data of the region of interest from the first point cloud data of the environment where the handling equipment is located. Then, it uses a laser scanning device to detect the slope within the environment where the handling equipment is located, obtaining slope detection results. Based on the slope detection results, it filters out ground laser points in the second point cloud data, obtaining non-ground laser points. These non-ground laser points are then clustered to obtain clustering results. If the clustering results indicate that the non-ground laser points are clustered into different clusters, it is determined that there are obstacles in the environment where the handling equipment is located. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, handling equipment, etc. The handling equipment can be unmanned forklifts, robots for handling goods, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0060] In one embodiment, such as Figure 2 As shown, a ground filtering and obstacle detection method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:

[0061] Step 202: Extract the second point cloud data of the region of interest from the first point cloud data of the environment where the handling equipment is located.

[0062] The term "handling equipment" can refer to equipment used for moving goods or equipment used for transferring personnel. This handling equipment can include, but is not limited to, unmanned forklifts, robots used for transferring goods and personnel, and cargo handling robots.

[0063] Point cloud data can refer to a set of vectors in a three-dimensional coordinate system. These vectors are usually represented in the form of three-dimensional coordinates (X, Y, Z). Point cloud data may also contain color information and reflectance information.

[0064] The first point cloud data can refer to 3D point cloud data used to characterize the environment in which the handling equipment is located.

[0065] The second point cloud data can refer to 3D point cloud data used to characterize the region of interest.

[0066] The region of interest can refer to the area of ​​point cloud data that requires further processing.

[0067] For example, firstly, the area in the environment where the handling equipment is located is determined to be the area where the point cloud data needs to be further processed. Then, the point cloud data corresponding to the area where the point cloud data needs to be further processed is extracted from the first point cloud data to obtain the second point cloud data.

[0068] In one embodiment, prior to step 202, the ground filtering and obstacle detection method further includes, but is not limited to, the following steps: scanning the environment where the transport equipment is located using a laser scanning device to obtain raw point cloud data; flipping the raw point cloud data up and down to obtain first point cloud data.

[0069] Laser scanning equipment can refer to devices capable of scanning the environment in which the handling equipment is located and providing 3D point cloud data of that environment. This laser scanning equipment can be, but is not limited to, lidar, 3D laser scanners, rotating radar, etc.

[0070] In some embodiments, the laser scanning device is installed on the bottom of the handling equipment. This application does not impose specific restrictions on the installation location of the laser scanning device, as long as it is installed on the bottom of the handling equipment. For example, when the handling equipment is an AGV, the laser scanning device can be installed on the bottom of the AGV body, with the front of the laser scanning device facing upwards, and the installation height can be 10-20cm from the ground.

[0071] Raw point cloud data can refer to 3D point cloud data used to characterize the environment surrounding the handling equipment. This raw point cloud data can be represented by (x, y, z).

[0072] For example, a 3D laser scanner installed on the bottom of the handling equipment can be used to scan the surrounding environment of the handling equipment to obtain raw point cloud data of the environment in which the handling equipment is located. Then, the raw point cloud data is flipped up and down to obtain the first point cloud data.

[0073] For example, if the original point cloud data is represented by (x, y, z), then the first point cloud data obtained after flipping the original point cloud data vertically can be represented by (x, y, -z).

[0074] By flipping the original point cloud data, the first point cloud data is obtained, which facilitates the acquisition of the terrain of the current transport equipment's environment. This makes it easier to filter out ground laser points in the second point cloud data, thus improving the accuracy of subsequent obstacle detection.

[0075] Step 204: Use a laser scanning device to detect the slope in the environment where the transport equipment is located, and obtain the slope detection results.

[0076] For example, a slope in the environment where the handling equipment is located can be scanned and detected by a laser scanning device to obtain a slope point cloud. Then, it can be determined whether there are slope breaks in the slope point cloud to determine whether there is a slope in the environment where the handling equipment is located, and the slope detection result can be obtained.

[0077] For example, when the laser scanning device is a rotating radar, the imaging principle of rotating radar can be used to scan and detect the slope in the environment where the transport equipment is located, and obtain a slope point cloud. Specifically, the laser beam emitted by the rotating radar is used to scan and detect the slope in the environment where the transport equipment is located. Then, it is determined whether there are any breaks in the laser beams (i.e., slope breaks), thereby determining whether there are slope sections and obtaining the slope detection results.

[0078] Step 206: Based on the slope detection results, filter out the ground laser points in the second point cloud data to obtain the non-ground laser points in the second point cloud data.

[0079] Among them, the ground laser point can refer to the laser point located on the ground in the second point cloud data, which can be represented by the point cloud in the second point cloud data.

[0080] Non-ground laser points can refer to laser points in the second point cloud data that are not located on the ground. These non-ground laser points can be represented by the point cloud in the second point cloud data.

[0081] For example, a point cloud ground point filtering algorithm (Cloth Simulation Filter, CSF) can be used to filter out ground laser points in the second point cloud data to obtain non-ground laser points in the second point cloud data.

[0082] Step 208: Cluster the non-ground laser points to obtain the clustering results.

[0083] Clustering refers to dividing a dataset (all non-ground laser points) into different classes or clusters according to a specific criterion (such as distance), so that the similarity of data objects within the same cluster is as high as possible, while the difference between data objects in different clusters is also as high as possible. That is, in this embodiment, by clustering the non-ground laser points, the non-ground laser points of the same class are grouped together as much as possible, while the non-ground laser points of different classes are separated as much as possible.

[0084] For example, after obtaining the non-ground laser points, a clustering algorithm is used to cluster the non-ground laser points to obtain clustering results. The clustering results may include non-ground laser points being clustered into different obstacle clusters, or non-ground laser points not being clustered into different obstacle clusters.

[0085] Clustering algorithms can include, but are not limited to, K-MEANS, K-MEDOIDS, CLARANS, BIRCH, CURE, CHAMELEON, etc.

[0086] Step 210: If the clustering result indicates that the non-ground laser points are clustered into different obstacle clusters, then it is determined that there are obstacles in the environment where the handling equipment is located.

[0087] When the clustering results show that non-ground laser points are clustered into different obstacle clusters, it indicates that there are obstacles in the environment where the handling equipment is located.

[0088] The ground filtering and obstacle detection method of this application extracts second point cloud data of the region of interest from the first point cloud of the environment where the transport equipment is located. This avoids the situation in the prior art where only one plane can be detected, thereby improving the adaptability to the environment where the transport equipment is located and thus improving the accuracy of obstacle detection. Furthermore, by detecting the slope in the environment where the transport equipment is located, the slope detection results are obtained to determine whether a slope exists in the environment where the transport equipment is located. This facilitates the identification of non-ground laser points in the second point cloud data, realizing ground filtering of the point cloud data. This further facilitates the determination of whether there are obstacles in the environment where the transport equipment is located based on non-ground laser points, further improving the accuracy of obstacle detection and thus improving the transport efficiency of the transport equipment.

[0089] like Figure 3 As shown, in one embodiment, prior to step 202, the ground filtering and obstacle detection method further includes, but is not limited to, the following steps:

[0090] Step 302: Scan the environment where the transport equipment is located using a laser scanning device to obtain raw point cloud data.

[0091] Step 304: Flip the original point cloud data vertically to obtain the first point cloud data.

[0092] Specifically, for specific embodiments of steps 302 to 304, please refer to the specific description in step 202.

[0093] The technical solution of this application embodiment flips the original point cloud data to simulate a piece of cloth falling under gravity, thereby facilitating the determination of the terrain of the environment in which the handling equipment is located.

[0094] like Figure 4 As shown, in one embodiment, step 202 includes, but is not limited to, the following steps: obtaining the laser point farthest from the center of the first point cloud data; determining the coverage area of ​​the first point cloud data based on the farthest laser point; determining the region of interest based on the coverage area and a preset area; wherein the preset area is at least one of the scanning blind zone of the laser scanning device that performs laser scanning to obtain the first point cloud data and the area where the point cloud is densely distributed in the first point cloud data; and determining the second point cloud data based on the region of interest and the first point cloud data.

[0095] The coverage area can refer to the region covered by the first point cloud data in the xy two-dimensional plane. A large circle can be drawn with the distance from the laser point furthest from the center of the first point cloud data to the center of the first point cloud data as the radius, and the center of the first point cloud data as the center. The area covered by this large circle is the coverage area.

[0096] The preset area can refer to a pre-defined area, which can be input by the user into the terminal or set by the terminal itself. For example, the preset area can be a circle with a radius of 0.5 meters centered on the center of the first point cloud data. This preset area is at least one of the following: the scanning blind zone of the laser scanning device used to obtain the first point cloud data, or a densely distributed area of ​​the point cloud in the first point cloud data. For example, the preset area can refer to both the scanning blind zone and a densely distributed area of ​​the point cloud in the first point cloud data.

[0097] For example, firstly, the center of the first point cloud data is determined. Then, the laser point farthest from the center is determined. Using the distance from this farthest point to the center as the radius and the center as the center, a circle is drawn on the xy two-dimensional plane to obtain the coverage area of ​​the first point cloud data on the xy two-dimensional plane. This coverage area is then used as the cloth plane. After determining the coverage area, a preset area is subtracted from the coverage area to obtain the region of interest. Then, the point cloud data of the region of interest is extracted from the first point cloud data to obtain the second point cloud data.

[0098] For example, such as Figure 4 As shown, Figure 4 The shaded area in the image is the preset area. Figure 4 The area occupied by the largest circle in the diagram is the covered area. Subtracting the shaded area from the covered area gives the region of interest. Figure 4The points in the diagram represent fabric particles.

[0099] By setting a preset area to filter out the region of interest from the coverage area, the computational load is reduced, the interference of densely distributed cloth particles on obstacle detection is avoided, and the accuracy of obstacle detection is improved.

[0100] In one embodiment, the slope detection result includes a first detection result indicating the presence of a slope or a second detection result indicating the absence of a slope. Step 204 includes, but is not limited to, the following steps: performing slope detection using a laser beam emitted by a laser scanning device to obtain a slope detection result; wherein, if a break occurs in the laser beam, the slope detection result is the first detection result; if no break occurs in the laser beam, the slope detection result is the second detection result.

[0101] Specifically, the laser scanning equipment emits multiple laser beams and rotates to scan and detect breaks in the slope. When a break is found in a laser beam, it indicates the presence of a slope section, and the corresponding slope detection result is the first detection result. When no breaks are found in any of the laser beams, it indicates the presence of a slope section, and the corresponding slope detection result is the second detection result.

[0102] The technical solution of this application embodiment detects the slope surface by using a laser beam emitted by a laser scanning device, and determines the slope detection result based on whether the laser beam shows a break, thereby improving the accuracy of slope detection.

[0103] like Figure 5 As shown, in one embodiment, step 206 includes, but is not limited to, the following steps:

[0104] Step 502: Establish a cloth particle model based on the second point cloud data.

[0105] The cloth particle model can refer to the ground filtering cloth model (Filtering Method Based on Cloth Simulation) in deep learning. The distribution density of the cloth particles in the cloth particle model is adapted to the distribution density of the point cloud (laser points) in the second point cloud data. The cloth particles are the point cloud (laser points) in the second point cloud data used for the cloth particle model.

[0106] The distribution density of fabric particles can be expressed by the following formula (1), specifically:

[0107]

[0108] Where d represents the distance of each cloth particle to the center of the second point cloud data. γ represents the angular resolution of the laser scanning device. This angular resolution can refer to the pointing accuracy of the laser scanning device.

[0109] Reference Figure 4 , Figure 4 The included angle of each sector is equal to an integer multiple of the angular resolution of the laser scanning device; as the chord length increases, the arc length also increases, the spacing between the fabric particles also becomes larger, and as the distance increases, the distribution of the fabric particles becomes sparser.

[0110] In some embodiments, it can be Figure 4 The circular distribution of fabric particles in the material is decomposed into... Figure 6 A schematic diagram of the fan-shaped distribution of fabric particles. Figure 6 In the diagram, the coordinates of each fabric particle can be represented as (α... n ,R n ,h), where α=n*γ, and n is an integer; n Let R1 represent the radius. During the initialization phase, R1 = r, where r is the radius of the preset area. This preset area radius is pre-set and its value range can be 0.5 meters to 1 meter, or other values. It can be set according to the scanning blind zone of the laser scanning device. This application does not impose specific restrictions on this. Let the chord length x = arc length l, then R2 = R1 + x, and so on, l n =α*R n x n =l n R n =R n-1 +x n .

[0111] For example, using laser points in the second point cloud data as cloth particles, a cloth particle model of the region of interest is established, and the coordinates of each cloth particle are determined.

[0112] By deconstructing and converting the circular distribution of fabric particles into Figure 6 The fan-shaped distribution of cloth particles in the cloud allows them to adapt to the distribution characteristics of point cloud density, which is denser near the edge and sparser further away. This makes it easier to save computational resources in the subsequent cloth particle model based on the fan-shaped distribution. Furthermore, converting the circular distribution of cloth particles into a fan-shaped distribution makes the distribution of the ground more detailed, so that the cloth particles can represent the groove features of the ground.

[0113] Step 504: Establish a corresponding motion model for each cloth particle in the cloth particle model, and determine the falling time of each cloth particle based on the slope detection results; wherein, each cloth particle is a laser point in the second point cloud data.

[0114] In this embodiment, each fabric particle is subject to gravity and the interaction force (spring force) between the fabric particles. Under the influence of gravity and spring force, the fabric particles can only move vertically up and down. The corresponding motion model can be represented by the following formula (2), which is as follows:

[0115]

[0116] In formula (2), x represents the position of the fabric particle at time t, g(x,t) represents the falling motion of the fabric particle under gravity, g represents the gravitational acceleration, f(x,t) represents the motion of the fabric particle caused by the interaction force between fabric particles, k represents the spring constant, m represents the mass of the object under the spring, and t is the falling time of the fabric particle, which is a time constant. The value of t is related to the slope detection result. When the slope detection result is the first detection result indicating the presence of a slope, the value of t is larger; when the slope detection result is the second detection result indicating the absence of a slope, the value of t is smaller. That is, when the fabric particle is at the slope breakpoint, the corresponding value of t is larger; when the fabric particle is not at the slope breakpoint, the corresponding value of t is smaller.

[0117] For example, after establishing the fabric sub-model, a corresponding motion model is established for each fabric particle in the fabric sub-model, and the falling time of each fabric particle is determined based on the slope detection results.

[0118] Step 506: Determine the initial cloth plane based on the motion model and place the initial cloth plane at the highest point of the second point cloud data.

[0119] The initial cloth plane can refer to the plane of the cloth particle at the initial moment. The position of this initial cloth plane can be the position of the cloth particle at t=0, that is, the position of the initial cloth plane can refer to the height value of the cloth particle after initialization, which is equal to the height of the point cloud with the largest height in the first point cloud data.

[0120] For example, by setting t=0, the position of the initial cloth plane can be determined. Then, the initial cloth plane is set at the highest point of the second point cloud data so that the height of all cloth particles is equal to the height of the point cloud with the largest height in the first point cloud data.

[0121] Step 508: Project the second point cloud data and each cloth particle onto the initial cloth plane.

[0122] For example, the second point cloud data and the cloth particles are projected onto the initial cloth plane to obtain the corresponding two-dimensional data and two-dimensional cloth particles.

[0123] Step 510: Based on the fall time and motion model, determine the height of each cloth particle after displacement, and determine the non-ground laser points in each cloth particle according to the height after displacement.

[0124] The height after displacement refers to the height of the cloth particle after displacement according to the corresponding t-value and motion model. That is, the height of the cloth particle after displacement caused by gravity.

[0125] For example, based on the falling time and motion model of each cloth particle, the height of each cloth particle after displacement is determined, and the non-ground laser point in each cloth particle is determined according to the height after displacement.

[0126] The technical solution of this application embodiment establishes a cloth particle model and a motion model, and determines the falling time of the cloth particles based on the slope detection results, thereby determining the height of each cloth particle after displacement. Then, it can determine the non-ground laser points in each cloth particle based on the height after displacement, realizing the filtering out of ground laser points in the second point cloud data, which facilitates the clustering of non-ground laser points in the subsequent process, thereby facilitating the detection of obstacles.

[0127] like Figure 7 As shown, in one embodiment, step 510 includes, but is not limited to, the following steps:

[0128] Step 702: Obtain the height of the nearest radar point for each cloth particle; where the nearest radar point is the nearest neighbor particle point of each cloth particle.

[0129] Step 704: Mark cloth particles whose height after displacement is less than or equal to the height of the nearest radar point as immovable points, and mark cloth particles whose height after displacement is greater than the height of the nearest radar point as movable points.

[0130] Specifically, if the height after displacement is less than or equal to the elevation, the cloth particle is considered to have collided with the point cloud in the second point cloud data, and in this case, the cloth particle is an immovable point. If the height after displacement is greater than the elevation, the cloth particle is considered not to have collided with the point cloud in the second point cloud data, and in this case, the cloth particle is marked as a movable point.

[0131] Step 706: Calculate the first displacement between each movable point and the first preset point, calculate the second displacement between each movable point, calculate the third displacement between each movable point and the second preset point, and calculate the fourth displacement between each movable point and the immovable point; wherein, the first preset point is a neighboring particle point of the movable point in the fabric particle, and the second preset point is a neighboring particle point of the immovable point in the fabric particle.

[0132] The first displacement can refer to the displacement of a movable point caused by the force between fabric particles. When the first preset point is a movable point, the movable point and the first preset point move towards each other, and the displacement of this movement is the first displacement. When the first preset point is a non-movable point, the movable point moves towards the first preset point under the force of the first preset point, and the displacement of this movement is the first displacement.

[0133] The second displacement can refer to the displacement caused by the interaction forces between movable points.

[0134] The third displacement can refer to the displacement of the field particle point (second preset point) used to represent the immovable point, caused by the force between cloth particles. That is, the displacement caused by the second preset point moving towards the immovable point due to the force exerted by the immovable point.

[0135] The fourth displacement can refer to the displacement produced when a movable point moves towards the immovable point due to the force exerted by the immovable point.

[0136] For example, for all movable points, the neighborhood particle points of each movable point are traversed to obtain a first preset point for each movable point, and the displacement of each movable point relative to the first preset point is determined to obtain a first displacement. The displacement of each movable point relative to immovable points is also determined to obtain a fourth displacement. Finally, the displacement between each movable point is determined to obtain a second displacement. For all immovable points, the neighborhood particle points of each immovable point are traversed to obtain a second preset point for each immovable point. When the second preset point is a movable point, the displacement of the second preset point relative to the immovable point is calculated to obtain a third displacement.

[0137] Step 708: Repeat the above steps until a preset condition is met; wherein, the preset condition is reaching a preset number of cycles, or the target displacement is less than a preset displacement threshold, and the target displacement is a first displacement, a second displacement, a third displacement, or a fourth displacement.

[0138] The preset condition can refer to a pre-set condition. This preset condition can be reaching a preset number of cycles, the first displacement being less than a preset displacement threshold, or the target displacement being less than a preset displacement threshold. The target displacement can be the first displacement, the second displacement, the third displacement, or the fourth displacement.

[0139] The preset number of loops can refer to a pre-set number of loops. The number of loops is incremented by 1 each time steps 602 to 604 are executed in each loop.

[0140] The preset displacement threshold can refer to a pre-set displacement threshold.

[0141] For example, when the preset conditions are not met, steps 602 and 604 are executed repeatedly to update the first displacement, the second displacement, the third displacement and the fourth displacement. When the preset conditions are met, the loop is exited and subsequent steps are executed.

[0142] Step 710: Determine the non-ground laser points in each cloth particle based on the height difference between the displaced cloth particle and the third preset point; wherein, the third preset point is the cloth particle closest to the cloth particle.

[0143] For example, after all the fabric particles are displaced by the forces between them, all the fabric particles form a fitted ground and the fabric particles are combined into a network. The height difference between each fabric particle in the network and a third preset point is compared, and the non-ground laser points in the fabric particles are determined based on the height difference.

[0144] The technical solution of this application embodiment realizes the classification between ground laser points and non-ground laser points, which facilitates the subsequent clustering of non-ground laser points, thereby facilitating the detection of obstacles and improving the handling efficiency of handling equipment.

[0145] In one embodiment, step 608 includes, but is not limited to, the following steps: calculating the height difference between each fabric particle and the corresponding third preset point; if the height difference is greater than a preset height threshold, then the fabric particle corresponding to the height difference greater than the preset height threshold is taken as a non-ground laser point.

[0146] The preset height threshold is a pre-defined height difference threshold.

[0147] For example, firstly, the height difference between each fabric particle and its corresponding third preset point is calculated. When the height difference is greater than a preset height threshold, the fabric particle corresponding to the height difference greater than the preset height threshold is designated as a non-ground laser point. When the height difference is less than the preset height threshold, the fabric particle corresponding to the height difference less than the preset height threshold is designated as a ground laser point.

[0148] The technical solution of this application embodiment calculates the height difference between each fabric particle and its corresponding third preset point, and classifies ground laser points and non-ground laser points in the fabric particles according to the height difference and preset height threshold, thereby improving the accuracy of classification. This facilitates subsequent clustering of non-ground laser points, which in turn facilitates obstacle detection, improves the accuracy of obstacle detection, and ultimately improves the handling efficiency of the handling equipment.

[0149] like Figure 8 As shown, in one embodiment, the ground filtering and obstacle detection method includes, but is not limited to, the following steps:

[0150] Step 802: Scan the environment where the transport equipment is located using a laser scanning device to obtain raw point cloud data.

[0151] Step 804: Flip the original point cloud data vertically to obtain the first point cloud data.

[0152] Step 806: Obtain the laser point that is farthest from the center of the first point cloud data.

[0153] Step 808: Determine the coverage area of ​​the first point cloud data based on the farthest laser point.

[0154] Step 810: Determine the region of interest based on the coverage area and the preset area; wherein, the preset area is the scanning blind zone of the laser scanning device that performs laser scanning to obtain the first point cloud data, or the area in the first point cloud data where the point cloud is densely distributed.

[0155] Step 812: Determine the second point cloud data based on the region of interest and the first point cloud data.

[0156] Step 814: Slope detection is performed using a laser beam emitted by a laser scanning device to obtain slope detection results; if a break occurs in the laser beam, the slope detection result is the first detection result used to characterize the presence of a slope; if no break occurs in the laser beam, the slope detection result is the second detection result used to characterize the absence of a slope.

[0157] Step 816: Establish a cloth particle model based on the second point cloud data.

[0158] Step 818: Establish a corresponding motion model for each cloth particle in the cloth particle model, and determine the falling time of each cloth particle based on the slope detection results; wherein, each cloth particle is a laser point in the second point cloud data.

[0159] Step 820: Determine the initial cloth plane based on the motion model and place the initial cloth plane at the highest point of the second point cloud data.

[0160] Step 822: Project the second point cloud data and each cloth particle onto the initial cloth plane.

[0161] Step 824: Based on the fall time and motion model, determine the height of each cloth particle after displacement; obtain the height of the nearest radar point of each cloth particle, mark cloth particles whose height after displacement is less than or equal to the height of the nearest radar point as immovable points, and mark cloth particles whose height after displacement is greater than the elevation as movable points; wherein, the nearest radar point is the nearest neighbor particle point of the cloth particle.

[0162] Step 826: Calculate the first displacement between each movable point and the first preset point, calculate the second displacement between each movable point, calculate the third displacement between each movable point and the second preset point, and calculate the fourth displacement between each movable point and the immovable point; wherein, the first preset point is the neighboring particle point of the movable point in the cloth particle, and the second preset point is the neighboring particle point of the immovable point in the cloth particle.

[0163] Step 828: Repeat steps 824 to 826 until a preset condition is met; wherein, the preset condition is reaching a preset number of cycles, or the target displacement is less than a preset displacement threshold, and the target displacement is a first displacement, a second displacement, a third displacement, or a fourth displacement.

[0164] Step 830: Calculate the height difference between each fabric particle and the corresponding third preset point; where the third preset point is the closest fabric particle to the fabric particle.

[0165] Step 832: If the height difference is greater than the preset height threshold, then the fabric particles corresponding to the height difference greater than the preset height threshold are taken as non-ground laser points.

[0166] For specific embodiments of steps 802 to 832, please refer to the foregoing. Figures 1 to 7 Examples of implementations.

[0167] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0168] Based on the same inventive concept, this application also provides a ground filtering and obstacle detection device for implementing the above-mentioned ground filtering and obstacle detection method.

[0169] In one embodiment, such as Figure 9 As shown, a ground filtering and obstacle detection device is provided, including: an extraction module 902, a slope detection module 904, a point cloud ground point filtering module 906, a clustering module 908, and a judgment module 910, wherein:

[0170] The extraction module 902 is used to extract the second point cloud data of the region of interest based on the first point cloud data of the environment in which the handling equipment is located.

[0171] The slope detection module 904 is used to perform slope detection using a laser scanning device to obtain slope detection results; the slope is located in the environment where the transport equipment is located.

[0172] The point cloud ground point filtering module 906 is used to filter the second point cloud data based on the slope detection results to obtain the non-ground points in the second point cloud data.

[0173] Clustering module 908 is used to perform clustering processing on non-ground points to obtain clustering results;

[0174] The judgment module 910 is used to determine if the clustering result indicates that non-ground points are clustered into different obstacle clusters, then there are obstacles in the environment where the handling equipment is located.

[0175] In one embodiment, the ground filtering and obstacle detection device further includes:

[0176] The scanning module is used to scan the environment where the transport equipment is located using a laser scanning device to obtain raw point cloud data.

[0177] The flip module is used to flip the original point cloud data vertically to obtain the first point cloud data.

[0178] In one embodiment, the extraction module 902 includes, but is not limited to:

[0179] The farthest laser point acquisition submodule is used to acquire the laser point that is farthest from the center of the first point cloud data.

[0180] The coverage area determination submodule is used to determine the coverage area of ​​the first point cloud data based on the farthest laser point.

[0181] The region of interest determination module is used to determine the region of interest based on the coverage area and the preset area; wherein, the preset area is at least one of the scanning blind zone of the laser scanning device that performs laser scanning to obtain the first point cloud data and the area where the point cloud is densely distributed in the first point cloud data.

[0182] The second point cloud data determination module is used to determine the second point cloud data based on the region of interest and the first point cloud data.

[0183] In one embodiment, the slope detection module 904 includes, but is not limited to:

[0184] The slope detection submodule is used to detect slopes using a laser beam emitted by a laser scanning device and obtain slope detection results. If a break occurs in the laser beam, the slope detection result is the first detection result; if no break occurs in the laser beam, the slope detection result is the second detection result.

[0185] In one embodiment, the point cloud ground point filtering module 906 includes, but is not limited to:

[0186] The Cloth Particle Model Building Submodule is used to build a cloth particle model based on the second point cloud data.

[0187] The motion model establishment submodule is used to establish a corresponding motion model for each cloth particle in the cloth particle model, and to determine the falling time of each cloth particle based on the slope detection results; where each cloth particle is a laser point in the second point cloud data.

[0188] The initial cloth plane determination submodule is used to determine the initial cloth plane based on the motion model and place the initial cloth plane at the highest point of the second point cloud data.

[0189] The projection submodule is used to project the second point cloud data and each cloth particle onto the initial cloth plane.

[0190] The non-ground laser point determination submodule is used to determine the height of each cloth particle after displacement based on the fall time and motion model, and to determine the non-ground laser points in each cloth particle based on the height after displacement.

[0191] In one embodiment, the non-ground laser point determination submodule includes:

[0192] The height acquisition unit is used to acquire the height of the nearest radar point for each cloth particle.

[0193] The marking unit is used to mark cloth particles whose height after displacement is less than or equal to the elevation as immovable points, and cloth particles whose height after displacement is greater than the elevation as movable points.

[0194] The displacement calculation unit is used to calculate the first displacement between each movable point and the first preset point, the second displacement between each movable point, the third displacement between each movable point and the second preset point, and the fourth displacement between each movable point and the immovable point; wherein, the first preset point is the neighboring particle point of the movable point in the cloth particle, and the second preset point is the neighboring particle point of the immovable point in the cloth particle.

[0195] The loop unit is used to repeatedly execute the above steps until a preset condition is met; wherein, the preset condition is to reach a preset number of loops, or the target displacement is less than a preset displacement threshold, and the target displacement is a first displacement, a second displacement, a third displacement, or a fourth displacement.

[0196] The non-ground laser point determination unit is used to determine the non-ground laser points in each cloth particle based on the height difference between the displaced cloth particle and the third preset point; wherein, the third preset point is the cloth particle closest to the cloth particle.

[0197] In one embodiment, the non-ground laser point determination unit includes:

[0198] The height difference calculation subunit is used to calculate the height difference between each fabric particle and the corresponding third preset point.

[0199] The comparison sub-unit is used to treat the fabric particles corresponding to the height difference greater than the preset height threshold as non-ground laser points if the height difference is greater than the preset height threshold.

[0200] Each module in the aforementioned ground filtering and obstacle detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independently of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0201] In one embodiment, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the electronic device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a ground filtering and obstacle detection method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the electronic device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the electronic device, or external keyboards, touchpads, or mice, etc.

[0202] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0203] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method embodiments.

[0204] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0205] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method embodiments.

[0206] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0207] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0208] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of ground filtering and obstacle detection, characterized in that, The method comprises: extracting second point cloud data of a region of interest from first point cloud data of an environment in which the handling equipment is located; detecting a slope surface in the environment in which the handling equipment is located by a laser scanning device to obtain a slope surface detection result; establishing a cloth particle model based on the second point cloud data; establishing a corresponding motion model for each cloth particle in the cloth particle model, and determining a falling time of each cloth particle according to the slope surface detection result; wherein each cloth particle is a laser point in the second point cloud data; determining an initial cloth plane based on the motion model, and placing the initial cloth plane at the highest point of the second point cloud data; projecting the second point cloud data and each cloth particle onto the initial cloth plane; determining the height of each cloth particle after displacement based on the falling time and the motion model, and determining a non-ground laser point in each cloth particle according to the height after displacement; performing clustering processing on the non-ground laser point to obtain a clustering result; if the clustering result indicates that the non-ground laser points are clustered into different obstacle clusters, it is determined that there is an obstacle in the environment in which the handling equipment is located.

2. The method of claim 1, wherein, Before the second point cloud data of the region of interest is extracted from the first point cloud data of the environment in which the handling equipment is located, the method further comprises: scanning the environment in which the handling equipment is located by the laser scanning device to obtain original point cloud data; inverting the original point cloud data upside down to obtain the first point cloud data.

3. The method of claim 1, wherein, The second point cloud data of the region of interest is extracted from the first point cloud data of the environment in which the handling equipment is located, comprising: obtaining the farthest laser point from the center of the first point cloud data in the first point cloud data; determining a coverage area of the first point cloud data based on the farthest laser point; determining the region of interest according to the coverage area and a preset region; wherein the preset region is at least one of a scanning blind area of the laser scanning device for obtaining the first point cloud data by laser scanning, and a region in which point clouds are densely distributed in the first point cloud data; determining the second point cloud data based on the region of interest and the first point cloud data.

4. The method of claim 1, wherein, The slope surface detection result comprises a first detection result indicating the presence of a slope surface or a second detection result indicating the absence of a slope surface; The slope surface in the environment in which the handling equipment is located is detected by the laser scanning device to obtain a slope surface detection result, comprising: detecting the slope surface in the environment in which the handling equipment is located by a laser line bundle emitted by the laser scanning device to obtain a slope surface detection result; wherein if the laser line bundle has a fault, the slope surface detection result is the first detection result; if the laser line bundle does not have a fault, the slope surface detection result is the second detection result.

5. The method of claim 4, wherein, The non-ground laser point in each cloth particle is determined according to the height after displacement, comprising: obtaining the height of the nearest radar point of each cloth particle; wherein the nearest radar point is the nearest neighbor particle point of each cloth particle. marking the cloth particles with the displaced height less than or equal to the height of the nearest radar point as immovable points, and marking the cloth particles with the displaced height greater than the height of the nearest radar point as movable points; calculating a first displacement between each movable point and a first preset point, calculating a second displacement between each movable point, calculating a third displacement between each movable point and a second preset point, and calculating a fourth displacement between each movable point and an immovable point; wherein the first preset point is a neighborhood particle point of the movable point in the cloth particles, and the second preset point is a neighborhood particle point of the immovable point in the cloth particles; recursively performing the above steps until a preset condition is met; wherein the preset condition is that a preset number of cycles is reached, or a target displacement is less than a preset displacement threshold, the target displacement being the first displacement, the second displacement, the third displacement, or the fourth displacement; determining non-ground laser points in each cloth particle according to the height difference between the displaced cloth particles and a third preset point; wherein the third preset point is the nearest cloth particle to the cloth particles.

6. The method of claim 5, wherein, The determination of non-ground laser points in each cloth particle according to the height difference between the displaced cloth particles and a third preset point comprises: calculating the height difference between each cloth particle and the corresponding third preset point; if the height difference is greater than a preset height threshold, the cloth particle corresponding to the height difference greater than the preset height threshold is taken as a non-ground laser point.

7. A ground filtering and obstacle detection apparatus, characterized by, The device comprises: an extraction module configured to extract second point cloud data of a region of interest based on first point cloud data of an environment in which a handling device is located; a slope detection module configured to perform slope detection by a laser scanning device to obtain a slope detection result; the slope is located in the environment in which the handling device is located; a point cloud ground point filtering module configured to establish a cloth particle model based on the second point cloud data, establish a corresponding motion model for each cloth particle in the cloth particle model, and determine a falling time of each cloth particle according to the slope detection result; wherein each cloth particle is a laser point in the second point cloud data, an initial cloth plane is determined based on the motion model, and the initial cloth plane is placed at the highest point of the second point cloud data, the second point cloud data and each cloth particle are projected onto the initial cloth plane, the height of each cloth particle after displacement is determined based on the falling time and the motion model, and non-ground laser points in each cloth particle are determined according to the height after displacement; a clustering module configured to perform clustering processing on the non-ground laser points to obtain a clustering result; a judgment module configured to determine that there is an obstacle in the environment in which the handling device is located if the clustering result indicates that the non-ground laser points are clustered into different obstacle clusters.

8. The apparatus of claim 7, wherein, The device further comprises: a scanning module configured to scan the environment in which the handling device is located by the laser scanning device to obtain original point cloud data; A flipping module is configured to flip the original point cloud data upside down to obtain first point cloud data. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images