Change object detection method and device, computer device and storage medium

Abstract

The application relates to a change object detection method and device, computer equipment and a storage medium. The method comprises the following steps: periodically acquiring point cloud data according to a preset time length during the process that a hook device lifts a hanging object; intercepting a point cloud area where hanging object point clouds exist in each point cloud data to obtain corresponding area point clouds; performing first transformation processing and deduplication processing on each area point cloud to obtain change point clouds of each area point cloud relative to other area point clouds; performing second transformation processing on each change point cloud to obtain corresponding recovery point clouds; respectively performing clustering processing on each recovery point cloud and each area point cloud to obtain first clustering clusters and second clustering clusters; and determining change object information between each point cloud data according to point clouds of second clusters in each second clustering cluster that are successfully matched with first clusters in corresponding first clustering clusters. Compared with manual detection of change objects, the method can improve the timeliness and accuracy of inspection.

Description

Technical Field

[0001] This application relates to the field of changing object detection, and more particularly to a changing object detection method, apparatus, computer device, and storage medium. Background Technology

[0002] In the field of autonomous driving, trajectory planning, collision detection, and obstacle avoidance are three crucial research areas. During the automated lifting process of crane equipment, not only does the object being lifted change, but the surrounding environment may also contain changing objects. To prevent collisions during lifting, it is essential to detect these changing objects. However, current technologies rely on manual visual observation of these changing objects during lifting. Manual observation has blind spots and is not always timely, reducing the accuracy of object detection. Summary of the Invention

[0003] This application provides a method, apparatus, computer device, and storage medium for detecting changing objects, in order to solve the problem of how to automatically detect changing objects during the hoisting process.

[0004] In a first aspect, this application provides a method for detecting changing objects, the method comprising:

[0005] During the lifting process of the hook equipment, point cloud data is acquired periodically according to a preset time interval;

[0006] Extract the point cloud regions containing the suspended object point cloud from each of the point cloud data to obtain the region point cloud corresponding to each of the point cloud data;

[0007] Perform a first transformation and deduplication process on the point clouds of each region to obtain the transformed point clouds of each region relative to the point clouds of other regions.

[0008] A second transformation process is performed on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process;

[0009] Clustering is performed on each of the recovered point clouds and each of the regional point clouds to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the regional point clouds.

[0010] Based on the point clouds of the second clusters in each of the second clusters that successfully match the first clusters in the corresponding first clusters, the changing object information between the point cloud data is determined.

[0011] Secondly, this application provides a changing object detection device, the device comprising:

[0012] The acquisition module is used to periodically acquire point cloud data according to a preset time interval during the lifting process of the hook equipment;

[0013] The interception module is used to intercept the point cloud region containing the suspended object point cloud in each of the point cloud data, and obtain the region point cloud corresponding to each of the point cloud data.

[0014] The first transformation module is used to perform a first transformation process and deduplication process on the point clouds of each region to obtain the transformed point clouds of each region relative to the point clouds of other regions.

[0015] The second transformation module is used to perform a second transformation process on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process;

[0016] The clustering module is used to perform clustering processing on each of the recovered point clouds and each of the region point clouds respectively, to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the region point clouds;

[0017] The matching module is used to determine the changing object information between the point cloud data based on the point cloud of the second cluster that is successfully matched with the first cluster in the corresponding first cluster.

[0018] Thirdly, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for detecting changing objects.

[0019] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the above-described changing object detection method.

[0020] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application acquires point cloud data periodically according to a preset time interval during the lifting process of a hook device; it extracts point cloud regions containing the object's point cloud from each of the point cloud data, obtaining regional point clouds corresponding to each of the point cloud data; it performs a first transformation and deduplication process on each of the regional point clouds to obtain changed point clouds relative to other regional point clouds; it performs a second transformation process on each of the changed point clouds to obtain restored point clouds corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process; it performs clustering processing on each of the restored point clouds and each of the regional point clouds to obtain a first cluster corresponding to each of the restored point clouds and a second cluster corresponding to each of the regional point clouds; based on the point clouds in each of the second clusters that successfully match the first cluster in the corresponding first cluster, it determines the changing object information between the point cloud data.

[0021] Based on the above method, the changing object information between different point cloud data can be automatically determined according to the point cloud data acquired at different times during the lifting process of the hook equipment, thereby detecting the changing object situation. Compared with manual detection of changing objects, it can improve the timeliness and accuracy of inspection. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0025] Figure 1 An application environment diagram for a changing object detection method provided in this application embodiment;

[0026] Figure 2 A schematic flowchart of a changing object detection method provided in an embodiment of this application;

[0027] Figure 3A structural block diagram of a changing object detection device provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0031] Figure 1 This is a diagram illustrating the application environment of a changing object detection method in one embodiment. (Refer to...) Figure 1 This changing object detection method is applied to a changing object detection system. The system includes a hook device 110 and a server 120. The hook device 110 and server 120 are connected via a network. The hook device 110 can be any device containing a hook, such as lifting machinery, tower cranes, gantry cranes, etc., and is equipped with a lidar for collecting point cloud data. The server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0032] In one embodiment, Figure 2 This is a flowchart illustrating a changing object detection method in one embodiment, with reference to... Figure 2 This paper provides a method for detecting changing objects. This embodiment mainly applies this method to the above-mentioned... Figure 1 Taking server 120 as an example, the specific steps of this changing object detection method include the following:

[0033] Step S210: During the process of lifting the load by the hook device 110, point cloud data is acquired periodically according to a preset time interval.

[0034] Specifically, during the lifting process of the hook device 110, point cloud data is collected periodically using a lidar system. This collected data is then sent to the server 120 for processing. The time interval between any two adjacent frames of point cloud data is a preset duration. Since the load is being lifted, there is a height difference between two consecutive frames of point cloud data. This height difference can be obtained by calculating the height difference from the cable length sensor at different times. The preset duration is set based on the preset height difference corresponding to two consecutive frames of point cloud data. This preset height difference is denoted as... The preset height difference can be customized according to actual application needs. For example, The preset duration is set to the corresponding value, so that the height difference between two consecutive frames of point cloud data collected periodically according to the preset duration is 0.5 meters.

[0035] The point cloud data collected by lidar includes point clouds corresponding to different categories of objects, such as background point clouds, hook point clouds, suspended object point clouds, and other object point clouds.

[0036] Step S220: Extract the point cloud regions containing the suspended object point cloud from each of the point cloud data to obtain the region point cloud corresponding to each of the point cloud data.

[0037] Specifically, due to the different acquisition times, the point cloud regions containing the suspended object point cloud are different in the point cloud data of different frames, and thus the point cloud regions corresponding to different point cloud data are also different.

[0038] Step S230: Perform a first transformation process and deduplication process on the point clouds of each region to obtain the changed point clouds of each region relative to the point clouds of other regions.

[0039] Specifically, the point cloud of each region is subjected to a first transformation process, which includes coordinate axis transformation, rotation, and downsampling. The deduplication process involves overlapping the point clouds of each region after the first transformation process to remove points that exist in different region point clouds, thereby obtaining the changed point clouds of each region relative to other region point clouds, that is, there are no identical points between different changed point clouds.

[0040] Step S240: Perform a second transformation process on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process.

[0041] Specifically, the second transformation process is the inverse of the first transformation process. It also includes coordinate axis transformation, rotation, and downsampling, but all processes are reversed compared to the first transformation process. The second transformation process is applied to each transformed point cloud to obtain the recovered point cloud.

[0042] Step S250: Clustering is performed on each of the recovered point clouds and each of the region point clouds to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the region point clouds.

[0043] Specifically, clustering is performed on each recovered point cloud to obtain the first cluster corresponding to each recovered point cloud, and clustering is performed on each regional point cloud to obtain the second cluster corresponding to each regional point cloud. Each recovered point cloud has a corresponding regional point cloud, so each first cluster has a corresponding second cluster.

[0044] Step S260: Based on the point cloud of the second cluster that successfully matches the first cluster in the corresponding first cluster, determine the changing object information between the point cloud data.

[0045] Specifically, the clusters in the first cluster are matched with the clusters in the corresponding second cluster. When there is a second cluster in the second cluster that successfully matches the first cluster in the first cluster, the object information of the corresponding point cloud data compared with other point cloud data is determined based on the point cloud of the second cluster. Since different frames of point cloud data are used to indicate the instantaneous state of the suspended object at different times, the difference between a single frame of point cloud data and other frames of point cloud data can be detected through different frames of point cloud data. This allows for the determination of the object changes in each frame of point cloud data compared with other frames of point cloud data, thereby detecting the object changes. Compared with manual detection of object changes, this can improve the timeliness and accuracy of the inspection.

[0046] In one embodiment, acquiring two point cloud data sets with a preset time interval includes first point cloud data and second point cloud data. The step of extracting point cloud regions containing suspended object point clouds from each of the point cloud data sets to obtain the corresponding region point cloud for each point cloud data set includes:

[0047] For the first point cloud data and the second point cloud data respectively, detect the cable suspension direction vector and the hook point cloud to obtain the first cable suspension direction vector and the first hook point cloud corresponding to the first point cloud data, and the second cable suspension direction vector and the second hook point cloud corresponding to the second point cloud data.

[0048] The highest known point is determined based on the highest projection point of the first hook point cloud on the first steel cable suspension direction vector and the highest projection point of the second hook point cloud on the second steel cable suspension direction vector.

[0049] The top plane of the hook is determined based on the highest known point and the corresponding preset plane normal vector;

[0050] The top plane of the hook is used to extract the area of ​​the point cloud containing the suspended object from each of the point cloud data.

[0051] Specifically, the two consecutively acquired point cloud data frames are denoted as the first point cloud data. Second point cloud data The processing method of the first and second point cloud data is used as an example to illustrate the processing method of continuous multi-frame point cloud data.

[0052] After preprocessing the first and second point cloud data respectively, the cable suspension direction vector of each frame of point cloud data is detected and normalized to obtain the first cable suspension direction vector. Second steel cable suspension direction vector Furthermore, the hook point cloud was segmented from the first point cloud data and the second point cloud data to obtain the first hook point cloud. Second hook cloud Project the first hook suspension point cloud onto the first cable suspension direction vector, and record the highest projection point of the first hook suspension point cloud onto the first cable suspension direction vector as the first projection point. The highest projection point of the second hook point cloud onto the suspension direction vector of the second steel cable is denoted as the second projection point. The highest known point is determined by the projection point with the higher coordinates between the first and second projection points. i takes the value 1 or 2, and is related to... corresponding As the pre-defined plane normal vector, that is, the highest known point is When the first steel cable is suspended in the direction vector As the corresponding preset plane normal vector, the highest known point is... When the second steel cable is suspended in the direction vector As the corresponding preset plane normal vector.

[0053] Based on the highest known point and the corresponding preset plane normal vector, the top plane P of the hook can be constructed. The plane equation of the top plane P of the hook is: Where A, B, C, and D are equation coefficients, specifically calculated using the highest known point and the preset plane normal vector, while x, y, and z are point coordinates. Based on the hook point cloud and the cable suspension direction vector, the top plane of the hook can be accurately determined. Since the suspended object is located below the hook, the point cloud below the top plane of the hook can be considered as the region point cloud containing the suspended object's point cloud. Therefore, based on the principle of linear programming, the region point cloud containing the suspended object's point cloud can be accurately clipped using the top plane of the hook.

[0054] In one embodiment, the step of extracting the area point cloud containing the suspended object from each point cloud data using the top plane of the hook includes:

[0055] Substituting the coordinates of the highest known point into the plane equation of the top plane of the hook, we obtain the first value;

[0056] Substitute the coordinates of a specified point in the specified point cloud data into the plane equation of the top plane of the hook to obtain a second value, wherein the specified point cloud data is either the first point cloud data or the second point cloud data, and the specified point is any point in the specified point cloud data;

[0057] The specified points whose product of the first value and the second value is greater than or equal to zero are used to form the region point cloud corresponding to the specified point cloud data.

[0058] Specifically, the first value is obtained by substituting the coordinates of the highest known point into the plane equation of the top plane of the hook. ,Right now ,in , , The coordinates of the highest known point are given. Then, the coordinates of the specified point in the specified point cloud data are substituted into the plane equation of the top plane of the hook to obtain the second value. ,Right now In this formula, x, y, and z represent the coordinates of a specified point in the specified point cloud data. Since the specified point cloud data is either the first point cloud data or the second point cloud data, the specified point can be any point in the first point cloud data or any point in the second point cloud data. The product of the first value and the second value is... and judge To determine whether the condition is met, select the specified points in the specified point cloud data that satisfy the condition to form the corresponding region point cloud. This process is repeated to crop the corresponding region point clouds from the first and second point cloud data respectively. The region point cloud corresponding to the first point cloud data is denoted as the first region point cloud. The area point cloud corresponding to the second point cloud data is denoted as the second area point cloud. .

[0059] By checking whether the product of the first and second values ​​is less than zero, we can determine whether the specified point corresponding to the second value is the point of the object to be suspended. In this way, we can accurately filter out the area point cloud of the object to be suspended from the point cloud data.

[0060] In one embodiment, performing a first transformation and deduplication on each of the region point clouds to obtain a transformed point cloud of each region point cloud relative to other region point clouds includes:

[0061] The corresponding rotation matrix is ​​determined based on the elevation direction vector opposite to the suspension direction vector of each steel cable and the preset axis unit vector;

[0062] The corresponding region point cloud is rotated using each of the rotation matrices to obtain the rotated point cloud corresponding to each region point cloud.

[0063] Each of the rotating point clouds is downsampled to obtain a downsampled point cloud corresponding to each of the rotating point clouds, and the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel is recorded.

[0064] The different downsampled point clouds are deduplicated to obtain the change point cloud of each downsampled point cloud relative to the other downsampled point clouds.

[0065] Specifically, the cable suspension direction vector refers to the direction vector perpendicularly downwards relative to the ground, while the opposite elevation direction vector is the direction vector perpendicularly upwards relative to the ground. The preset axis unit vector refers to the Z-axis unit vector. The rotation matrix R is determined by the elevation direction vector and the preset axis unit vector. The rotation matrix is ​​then used to rotate the point cloud of each region to obtain the corresponding rotated point cloud. The calculation formula is as follows:

[0066]

[0067] In this formula, x, y, and z are the coordinates of any point in the region point cloud. , , The coordinates of the corresponding point in the rotated point cloud, i.e. The coordinates of points in the region point cloud that have not undergone rotational transformation. These are the coordinates of the points after rotation transformation, corresponding to the point cloud of the first region mentioned above. The first rotated point cloud after rotation is denoted as For the point cloud in the second region The second rotated point cloud is denoted as To downsample the rotated point cloud, the aforementioned rotation process ensures that voxels are divided in one dimension perpendicular to the elevation direction during downsampling. This allows for better amplification of elevation changes, i.e., amplification of the changes in the suspended object during its ascent, thus facilitating better identification of these changes.

[0068] Specifically, voxels are used for rotating point clouds. Downsampling was performed, and the lengths of the voxels on the x, y, and z axes were respectively... , , The point cloud is divided into numerous voxels centered at the origin. If a voxel contains a point, the center coordinates of the voxel are used to replace all points within that voxel, and the correspondence between the downsampled point and all points within its corresponding voxel is saved. The downsampled point cloud obtained after downsampling is denoted as 'i' represents the frame number of the point cloud data, meaning the downsampled point cloud corresponding to the i-th frame is... The downsampling formula is as follows:

[0069]

[0070] Wherein in the formula , , Let represent the coordinates of any point within a voxel after the rotated point cloud has been divided into voxels. , , The indicator shows the points after downsampling. Because the lidar equipment acquiring the point cloud is mounted on the tower crane's boom or tower body, and this lidar equipment has not been calibrated, the point cloud coordinates of the same static ground feature acquired during the lifting and lowering phase exhibit varying degrees of horizontal plane rotation between different frames of point cloud data. Without point cloud registration, to reduce the impact of coordinate differences caused by horizontal plane rotation and amplify the impact of differences in the elevation direction, [further details are needed]. , The value should be greater than The value, and The values ​​are the same. The value is not less than Twice as much, specifically during use and The value is 0.4. The value is 0.2.

[0071] After downsampling, the resulting downsampled point clouds are subjected to overlap and deduplication processing to remove points that appear in both downsampled point clouds. This allows us to obtain the differences between each downsampled point cloud and the others, i.e., the changes in the point cloud relative to other downsampled point clouds. In other words, the changed point cloud is one that exists only in the current downsampled point cloud and not in other downsampled point clouds in adjacent frames. Downsampling reduces the amount of point cloud data that can be processed subsequently, thereby improving the efficiency of point cloud data processing and accelerating the detection of changed objects.

[0072] In one embodiment, determining the corresponding rotation matrix based on the elevation direction vector opposite to the suspension direction vectors of each cable and a preset axis unit vector includes:

[0073] The rotation axis vector and the included angle are determined based on the calculation result between the elevation direction vector opposite to the cable suspension direction vector and the preset axis unit vector.

[0074] The rotation vector is determined based on the dot product of the unit vector of the rotation axis and the angle between the vectors;

[0075] Convert the rotation vector into a rotation matrix.

[0076] Specifically, since the elevation direction vector is opposite to the cable suspension direction vector, the elevation direction vectors are as follows: That is, the elevation direction vector corresponding to the first cable suspension direction vector is The elevation direction vector corresponding to the second cable suspension direction vector is: The product of the elevation direction vector and the preset axis unit vector is the rotation axis vector. The formula for calculating the rotation axis magnitude is as follows: The dot product between the elevation direction vector and the preset axis unit vector is used to determine the vector angle. The formula for calculating the vector angle is: Then, the rotation vector is determined based on the dot product between the unit vector of the rotation axis and the angle between the vectors. The formula for calculating the rotation vector is: ,in Rotation axis vector The unit vector. According to Rodriguez's formula, the rotation vector can be... Convert to rotation matrix .

[0077] In one embodiment, performing a second transformation process on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds includes:

[0078] Each of the changed point clouds is filtered to obtain the filtered point cloud corresponding to each of the changed point clouds.

[0079] Based on the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel, each filtered point cloud is transformed into the point cloud before downsampling to obtain the inverse transformed point cloud corresponding to each filtered point cloud.

[0080] The inverse transformation of each inverse transformation point cloud is performed using an inverse rotation matrix to obtain the recovered point cloud corresponding to each inverse transformation point cloud.

[0081] Specifically, each of the changed point clouds is filtered and isolated points are removed to obtain the filtered point cloud corresponding to each of the changed point clouds. Based on the correspondence between the downsampled points and the points before downsampling. You can get Inverse transform point cloud before downsampling Then, the inverse transformation of the inversely transformed point cloud is performed using the inverse rotation matrix to obtain the recovered point cloud. The reverse rotation matrix is The inverse transform formula is:

[0082]

[0083] Where in the formula , , Let x, y, and z be the coordinates of any point in the inversely transformed point cloud, and let z be the points in the recovered point cloud after the inverse transformation.

[0084] In one embodiment, determining the changing object information among the point cloud data based on the point clouds of the second clusters that successfully match the first clusters in the corresponding first clusters includes:

[0085] When the overlap between the first cluster in the first cluster and the second cluster in the corresponding second cluster is greater than a preset threshold, the matching result between the first cluster and the second cluster is determined to be a successful match, and the point cloud corresponding to the first cluster is added to the object point cloud set of the corresponding point cloud data relative to other point cloud data.

[0086] A set of the number of changed objects corresponding to each point cloud data is generated based on the number of clusters in the set of changed object point clouds corresponding to each point cloud data. The changed object information between each point cloud data includes the set of changed object point clouds and the set of the number of changed objects.

[0087] Specifically, for recovering point clouds Distance clustering was performed to obtain the first cluster. , This represents the cluster of the recovered point cloud corresponding to the i-th frame of point cloud data after clustering. This cluster contains k clusters; however, the number of clusters k in the first cluster corresponding to point cloud data from different frames may be the same or different. For region point clouds... Distance clustering was performed to obtain the second cluster. , This represents the cluster of the region point cloud corresponding to the i-th frame point cloud data after clustering. This cluster contains m clusters. The number of clusters m contained in the second cluster corresponding to point cloud data of different frames may be the same or different.

[0088] The clusters in the first cluster and the clusters in the second cluster corresponding to the same frame of point cloud data are overlap-matched. If the overlap between a first cluster in the first cluster and a corresponding second cluster in the second cluster is greater than a preset threshold, it is considered that the two clusters overlap and are considered a successful match. Then, the first cluster is considered to be a changing object. In this way, the point clouds corresponding to each successfully matched first cluster are added to the changing object point cloud set. middle, Let represent the set of point clouds of changed objects corresponding to the point cloud data of the i-th frame, and n represent the number of change clusters in the set of point clouds of changed objects, i.e., the number of changed objects. The coordinate position and size range of each cluster can be determined by the point cloud of each cluster. A set of the number of changed objects is generated based on the number of clusters in the set of point clouds of changed objects. , This represents the set of changed object counts corresponding to the point cloud data in frame i, which contains n change clusters.

[0089] This method can detect changing objects in continuous different frames of point cloud data, and determine the changing objects in each frame of point cloud data relative to the adjacent frames of point cloud data. Compared with manual detection of changing objects, it can not only improve the detection efficiency of changing objects, but also improve the detection accuracy.

[0090] Figure 2 This is a flowchart illustrating a changing object detection method in one embodiment. It should be understood that, although... Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0091] In one embodiment, such as Figure 3 As shown, a changing object detection device is provided, comprising:

[0092] The acquisition module 310 is used to periodically acquire point cloud data according to a preset time interval during the process of lifting the load by the hook device 110;

[0093] The interception module 320 is used to intercept the point cloud region containing the suspended object point cloud in each of the point cloud data, and obtain the region point cloud corresponding to each of the point cloud data.

[0094] The first transformation module 330 is used to perform a first transformation process and deduplication process on the point clouds of each region to obtain the transformed point clouds of each region relative to the point clouds of other regions.

[0095] The second transformation module 340 is used to perform a second transformation process on each of the changed point clouds to obtain a restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process.

[0096] Clustering module 350 is used to perform clustering processing on each of the recovered point clouds and each of the region point clouds respectively to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the region point clouds.

[0097] The matching module 360 ​​is used to determine the changing object information between the point cloud data based on the point cloud of the second cluster that successfully matches the first cluster in the corresponding first cluster.

[0098] In one embodiment, acquiring two point cloud data sets with a preset time interval includes first point cloud data and second point cloud data, and the interception module 320 is further configured to:

[0099] For the first point cloud data and the second point cloud data respectively, detect the cable suspension direction vector and the hook point cloud to obtain the first cable suspension direction vector and the first hook point cloud corresponding to the first point cloud data, and the second cable suspension direction vector and the second hook point cloud corresponding to the second point cloud data.

[0100] The highest known point is determined based on the highest projection point of the first hook point cloud on the first steel cable suspension direction vector and the highest projection point of the second hook point cloud on the second steel cable suspension direction vector.

[0101] The top plane of the hook is determined based on the highest known point and the corresponding preset plane normal vector;

[0102] The top plane of the hook is used to extract the area of ​​the point cloud containing the suspended object from each of the point cloud data.

[0103] In one embodiment, the interception module 320 is further configured to:

[0104] Substituting the coordinates of the highest known point into the plane equation of the top plane of the hook, we obtain the first value;

[0105] Substitute the coordinates of a specified point in the specified point cloud data into the plane equation of the top plane of the hook to obtain a second value, wherein the specified point cloud data is either the first point cloud data or the second point cloud data, and the specified point is any point in the specified point cloud data;

[0106] The specified points whose product of the first value and the second value is greater than or equal to zero are used to form the region point cloud corresponding to the specified point cloud data.

[0107] In one embodiment, the first transformation module 330 is further configured to:

[0108] The corresponding rotation matrix is ​​determined based on the elevation direction vector opposite to the suspension direction vector of each steel cable and the preset axis unit vector;

[0109] The corresponding region point cloud is rotated using each of the rotation matrices to obtain the rotated point cloud corresponding to each region point cloud.

[0110] Each of the rotating point clouds is downsampled to obtain a downsampled point cloud corresponding to each of the rotating point clouds, and the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel is recorded.

[0111] The different downsampled point clouds are deduplicated to obtain the change point cloud of each downsampled point cloud relative to the other downsampled point clouds.

[0112] In one embodiment, the first transformation module 330 is further configured to:

[0113] The rotation axis vector and the included angle are determined based on the calculation result between the elevation direction vector opposite to the cable suspension direction vector and the preset axis unit vector.

[0114] The rotation vector is determined based on the dot product of the unit vector of the rotation axis and the angle between the vectors;

[0115] Convert the rotation vector into a rotation matrix.

[0116] In one embodiment, the second transformation module 340 is further configured to:

[0117] Each of the changed point clouds is filtered to obtain the filtered point cloud corresponding to each of the changed point clouds.

[0118] Based on the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel, each filtered point cloud is transformed into the point cloud before downsampling to obtain the inverse transformed point cloud corresponding to each filtered point cloud.

[0119] The inverse transformation of each inverse transformation point cloud is performed using an inverse rotation matrix to obtain the recovered point cloud corresponding to each inverse transformation point cloud.

[0120] In one embodiment, the clustering module 350 is further configured to:

[0121] When the overlap between the first cluster in the first cluster and the second cluster in the corresponding second cluster is greater than a preset threshold, the matching result between the first cluster and the second cluster is determined to be a successful match, and the point cloud corresponding to the first cluster is added to the object point cloud set of the corresponding point cloud data relative to other point cloud data.

[0122] A set of the number of changed objects corresponding to each point cloud data is generated based on the number of clusters in the set of changed object point clouds corresponding to each point cloud data. The changed object information between each point cloud data includes the set of changed object point clouds and the set of the number of changed objects.

[0123] like Figure 4 As shown, this application provides a computer device including a processor 711, a communication interface 712, a memory 713, and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714.

[0124] Memory 713 is used to store computer programs;

[0125] When the processor 711 executes the program stored in the memory 713, it implements the changing object detection method provided in any of the foregoing method embodiments.

[0126] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0127] In one embodiment, the changing object detection device provided in this application can be implemented as a computer program, which can be implemented in, for example... Figure 4 The computer device shown is running the program. The computer device's memory can store the various program modules that make up the changing object detection device, for example, Figure 3The diagram shows an acquisition module 310, a capture module 320, a first transformation module 330, a second transformation module 340, a clustering module 350, and a matching module 360. The computer program comprised of these modules causes the processor to execute the changing object detection methods of the various embodiments of this application described in this specification.

[0128] Figure 4 The computer equipment shown can be used as follows Figure 3 The acquisition module 310 in the changing object detection device shown periodically acquires point cloud data according to a preset time interval during the lifting process of the hook device 110. The computer device can use the interception module 320 to intercept the point cloud regions containing the object's point cloud in each of the point cloud data sets, obtaining the corresponding region point clouds. The computer device can use the first transformation module 330 to perform a first transformation and deduplication process on each of the region point clouds, obtaining the changed point clouds of each region point cloud relative to other region point clouds. The computer device can use the second transformation module 340 to perform a second transformation process on each of the changed point clouds, obtaining the corresponding restored point clouds, wherein the second transformation process is the opposite of the first transformation process. The computer device can use the clustering module 350 to perform clustering processes on each restored point cloud and each region point cloud, respectively, obtaining a first cluster corresponding to each restored point cloud and a second cluster corresponding to each region point cloud. The computer device can use the matching module 360 ​​to determine the changing object information between the point cloud data based on the point cloud of the second cluster that is successfully matched with the first cluster in the corresponding first cluster.

[0129] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the changing object detection method provided in any of the foregoing method embodiments.

[0130] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server 120, or network device, etc.) to execute the changing object detection method described in various embodiments or some parts of embodiments.

[0132] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also mean including the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that alternatives or substitutions may be used.

[0133] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for detecting changing objects, characterized in that, The method includes: During the lifting process of the hook equipment, point cloud data is acquired periodically according to a preset time interval; Extract the point cloud regions containing the suspended object point cloud from each of the point cloud data to obtain the region point cloud corresponding to each of the point cloud data; Perform a first transformation and deduplication process on the point clouds of each region to obtain the transformed point clouds of each region relative to the point clouds of other regions. A second transformation process is performed on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process; Clustering is performed on each of the recovered point clouds and each of the regional point clouds to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the regional point clouds. Based on the point cloud of the second cluster that successfully matches the first cluster in the corresponding first cluster, determine the changing object information between the point cloud data; The acquisition of two point cloud data sets with a preset time interval includes a first point cloud data set and a second point cloud data set. The step of extracting point cloud regions containing the suspended object point cloud from each of the point cloud data sets to obtain the corresponding region point cloud for each point cloud data set includes: For the first point cloud data and the second point cloud data respectively, detect the cable suspension direction vector and the hook point cloud to obtain the first cable suspension direction vector and the first hook point cloud corresponding to the first point cloud data, and the second cable suspension direction vector and the second hook point cloud corresponding to the second point cloud data. The highest known point is determined based on the highest projection point of the first hook point cloud on the first steel cable suspension direction vector and the highest projection point of the second hook point cloud on the second steel cable suspension direction vector. The top plane of the hook is determined based on the highest known point and the corresponding preset plane normal vector; The top plane of the hook is used to extract the area of ​​the point cloud containing the suspended object from each of the point cloud data. The step of extracting the area of ​​the point cloud containing the suspended object from each point cloud data using the top plane of the hook includes: Substituting the coordinates of the highest known point into the plane equation of the top plane of the hook, we obtain the first value; Substitute the coordinates of a specified point in the specified point cloud data into the plane equation of the top plane of the hook to obtain a second value, wherein the specified point cloud data is either the first point cloud data or the second point cloud data, and the specified point is any point in the specified point cloud data; The specified points whose product of the first value and the second value is greater than or equal to zero are used to form the region point cloud corresponding to the specified point cloud data.

2. The changing object detection method according to claim 1, characterized in that, The first transformation and deduplication process performed on each of the region point clouds to obtain the transformed point cloud of each region point cloud relative to other region point clouds includes: The corresponding rotation matrix is ​​determined based on the elevation direction vector opposite to the suspension direction vector of each steel cable and the preset axis unit vector; The corresponding region point cloud is rotated using each of the rotation matrices to obtain the rotated point cloud corresponding to each region point cloud. Each of the rotating point clouds is downsampled to obtain a downsampled point cloud corresponding to each of the rotating point clouds, and the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel is recorded. The different downsampled point clouds are deduplicated to obtain the change point cloud of each downsampled point cloud relative to the other downsampled point clouds.

3. The method for detecting changing objects according to claim 2, characterized in that, Based on the elevation direction vector opposite to the suspension direction vector of each steel cable and the preset axis unit vector, determine the corresponding rotation matrix, including: The rotation axis vector and the included angle are determined based on the calculation result between the elevation direction vector opposite to the cable suspension direction vector and the preset axis unit vector. The rotation vector is determined based on the dot product of the unit vector of the rotation axis and the angle between the vectors; Convert the rotation vector into a rotation matrix.

4. The method for detecting changing objects according to claim 2, characterized in that, The step of performing a second transformation on each of the changed point clouds to obtain the recovered point cloud corresponding to each of the changed point clouds includes: Each of the changed point clouds is filtered to obtain the filtered point cloud corresponding to each of the changed point clouds. Based on the correspondence between any point in each downsampled point cloud and the point before downsampling of the corresponding voxel, each filtered point cloud is transformed into the point cloud before downsampling to obtain the inverse transformed point cloud corresponding to each filtered point cloud. The inverse transformation of each inverse transformation point cloud is performed using an inverse rotation matrix to obtain the recovered point cloud corresponding to each inverse transformation point cloud.

5. The changing object detection method according to claim 4, characterized in that, The step of determining the changing object information among the point cloud data based on the point clouds of the second clusters that successfully match the first clusters in the corresponding first clusters includes: When the overlap between the first cluster in the first cluster and the second cluster in the corresponding second cluster is greater than a preset threshold, the matching result between the first cluster and the second cluster is determined to be a successful match, and the point cloud corresponding to the first cluster is added to the object point cloud set of the corresponding point cloud data relative to other point cloud data. A set of the number of changed objects corresponding to each point cloud data is generated based on the number of clusters in the set of changed object point clouds corresponding to each point cloud data. The changed object information between each point cloud data includes the set of changed object point clouds and the set of the number of changed objects.

6. A device for detecting changing objects, characterized in that, The device includes: The acquisition module is used to periodically acquire point cloud data according to a preset time interval during the lifting process of the hook equipment; The interception module is used to intercept the point cloud region containing the suspended object point cloud in each of the point cloud data, and obtain the region point cloud corresponding to each of the point cloud data. The first transformation module is used to perform a first transformation process and deduplication process on the point clouds of each region to obtain the transformed point clouds of each region relative to the point clouds of other regions. The second transformation module is used to perform a second transformation process on each of the changed point clouds to obtain the restored point cloud corresponding to each of the changed point clouds, wherein the second transformation process is the opposite of the first transformation process; The clustering module is used to perform clustering processing on each of the recovered point clouds and each of the region point clouds respectively, to obtain a first cluster corresponding to each of the recovered point clouds and a second cluster corresponding to each of the region point clouds; The matching module is used to determine the changing object information between the point cloud data based on the point cloud of the second cluster that is successfully matched with the first cluster in the corresponding first cluster. The acquisition of two point cloud data sets with a preset time interval includes a first point cloud data set and a second point cloud data set. The step of extracting point cloud regions containing the suspended object point cloud from each of the point cloud data sets to obtain the corresponding region point cloud for each point cloud data set includes: For the first point cloud data and the second point cloud data respectively, detect the cable suspension direction vector and the hook point cloud to obtain the first cable suspension direction vector and the first hook point cloud corresponding to the first point cloud data, and the second cable suspension direction vector and the second hook point cloud corresponding to the second point cloud data. The highest known point is determined based on the highest projection point of the first hook point cloud on the first steel cable suspension direction vector and the highest projection point of the second hook point cloud on the second steel cable suspension direction vector. The top plane of the hook is determined based on the highest known point and the corresponding preset plane normal vector; The top plane of the hook is used to extract the area of ​​the point cloud containing the suspended object from each of the point cloud data. The step of extracting the area of ​​the point cloud containing the suspended object from each point cloud data using the top plane of the hook includes: Substituting the coordinates of the highest known point into the plane equation of the top plane of the hook, we obtain the first value; Substitute the coordinates of a specified point in the specified point cloud data into the plane equation of the top plane of the hook to obtain a second value, wherein the specified point cloud data is either the first point cloud data or the second point cloud data, and the specified point is any point in the specified point cloud data; The specified points whose product of the first value and the second value is greater than or equal to zero are used to form the region point cloud corresponding to the specified point cloud data.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the changing object detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the changing object detection method according to any one of claims 1 to 5.

Images