Cargo handling anomaly detection method, electronic equipment and storage medium
By acquiring and converting the reference point cloud and scene point cloud of goods, and performing rasterization processing and comparison, the problems of low accuracy and low efficiency of goods detection in the prior art are solved, and higher detection accuracy and efficiency are achieved.
Patent Information
- Application Number
- CN202411724983.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-04-22
AI Technical Summary
The prior art has low detection accuracy and low detection efficiency in the inspection of transported goods, making it difficult to achieve general applicability.
By obtaining the reference point cloud of the goods at the initial position and the scene point cloud during the handling process, converting it to the same coordinate system, the first target range point cloud and the second target range point cloud of the effective range are determined, and one of them is rasterized, and the other is compared with the raster image to determine the detection status of the goods.
It effectively improves the detection accuracy during cargo handling, improves the detection efficiency, and avoids dependence on prior data.
Smart Images
Figure CN119863746A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of robot recognition, and in particular to a method for detecting abnormalities in the handling of goods, an electronic device, and a storage medium. Background Art
[0002] With the explosive growth of computer applications, image recognition has been continuously iterating towards the direction of intelligence and humanity, especially robot recognition technology, which is generally used to identify stacked goods to determine whether there are any abnormalities in the stacked goods.
[0003] In current technology, the inspection of transported goods generally targets certain areas of the goods, and then a customized inspection model is developed through model training. This is relatively complex, not universal, and has low accuracy in the inspection results; or it requires prior data for processing, which affects the efficiency of inspection. Summary of the invention
[0004] The main technical problem solved by the present application is to provide a method, electronic device and storage medium for detecting abnormalities in the handling of goods. By obtaining the reference point cloud of the initial position and the scene point cloud during the handling process, converting them into the same coordinate system, and determining the first target range point cloud and the second target range point cloud of the effective range, and rasterizing one of them, and comparing the other with the raster image, the detection status of the goods is determined, which can effectively improve the detection accuracy in the cargo handling process and improve the corresponding detection efficiency.
[0005] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a method for detecting abnormalities in the handling of goods, including: obtaining a reference point cloud of the goods at an initial position, and obtaining a scene point cloud during the handling of the goods, wherein the initial position is the position of the goods on the handling equipment before handling; converting the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud in the vehicle coordinate system, and a second target range point cloud corresponding to the scene point cloud; rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image; comparing the raster image with the other to determine the detection status during the handling of the goods.
[0006] In some embodiments, the rasterizing of one of the first target range point cloud and the second target range point cloud to determine a raster image includes: obtaining the maximum and minimum values of each coordinate axis of the first target range point cloud in the self-vehicle coordinate system; using the maximum and minimum values and preset grid information to perform raster processing on the first target range point cloud, and determining the current grid and the current grid with points within the preset range of the current grid as valid grids; and combining multiple valid grids to determine a first raster image, wherein the first raster image is a raster image corresponding to the first target range point cloud.
[0007] In some embodiments, the use of the grid image to compare with another to determine the detection status of the cargo handling process includes: using the maximum value, the minimum value and the preset grid information to project the second target range point cloud onto the first grid image for comparison to determine a first comparison result; in response to the number of the second target range point cloud located outside the valid grid in the first comparison result being greater than a preset threshold, determining that the detection result of the cargo handling process is abnormal.
[0008] In some embodiments, the effective range of the first target range point cloud is greater than the effective range of the second target range point cloud.
[0009] In some embodiments, rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image includes: obtaining the maximum and minimum values of each coordinate axis of the second target range point cloud in the self-vehicle coordinate system; using the maximum and minimum values and preset grid information to perform raster processing on the second target range point cloud, and determining the current grid and the current grid with points within the preset range of the current grid as valid grids; combining multiple valid grids to determine a second raster image, wherein the second raster image is a raster image corresponding to the second target range point cloud.
[0010] In some embodiments, the use of the grid image to compare with another to determine the detection status of the cargo handling process includes: using the maximum value, the minimum value and the preset grid information to project the first target range point cloud onto the second grid image for comparison to determine a second comparison result; in response to the number of the first target range point cloud located outside the valid grid in the second comparison result being greater than a preset threshold, determining that the detection result of the cargo handling process is abnormal.
[0011] In some embodiments, the effective range of the second target range point cloud is greater than the effective range of the first target range point cloud.
[0012] In some embodiments, at least one fixed camera device and at least one follow-up camera device are provided on the transporting equipment, wherein the fixed camera device is fixedly provided on the transporting equipment, and the follow-up camera device moves following the fork tines of the transporting equipment; the obtaining of the reference point cloud of the goods at the initial position, and the obtaining of the scene point cloud during the transport of the goods, include: using at least one of the fixed camera devices and at least one of the follow-up camera devices to obtain the reference point cloud corresponding to the position of the goods on the transporting equipment before being transported, and obtaining the scene point cloud during the transport of the goods by the transporting equipment.
[0013] In some embodiments, the converting of the reference point cloud and the scene point cloud to the self-vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud in the self-vehicle coordinate system, and a second target range point cloud corresponding to the scene point cloud, includes: using preset camera extrinsics of the fixed camera device and the follow-up camera device to convert the reference point cloud from the camera coordinate system of the initial position to the self-vehicle coordinate system, and converting at least one of the scene point clouds from the camera coordinate system during the transportation process to the self-vehicle coordinate system; obtaining the first point cloud coordinates corresponding to the reference point cloud in the self-vehicle coordinate system, and obtaining the second point cloud coordinates corresponding to the scene point cloud in the self-vehicle coordinate system; using preset cargo information, cropping the reference point cloud to determine the first target range point cloud corresponding to the reference point cloud, and cropping the scene point cloud to determine the second target range point cloud corresponding to the scene point cloud.
[0014] In some embodiments, it also includes: using the fixed camera device and the follow-up camera device to obtain a first height of the fork tine of the transporting equipment corresponding to the reference point cloud in the self-vehicle coordinate system, and obtaining a second height of the fork tine of the transporting equipment corresponding to the scene point cloud in the self-vehicle coordinate system; in response to the camera device for obtaining the scene point cloud being the fixed camera device, using the first height and the second height to correct the second point cloud coordinates of the scene point cloud in the self-vehicle coordinate system.
[0015] In some embodiments, it also includes: clustering the goods using a preset clustering method to determine a point cloud cluster block of the goods; in response to the number of points in the point cloud cluster block being less than a given threshold, removing the point cloud cluster block.
[0016] In order to solve the above-mentioned technical problems, another technical solution adopted in the present application is: to provide an electronic device, the electronic device includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method for detecting abnormalities in handling goods as mentioned above.
[0017] In order to solve the above technical problems, another technical solution adopted in the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium has at least one program, and when the at least one program is loaded and executed by the processor, it is used to implement the above-mentioned method for detecting abnormalities in handling goods.
[0018] Different from the current technology, the method for detecting abnormalities in the handling of goods provided in the present application includes: obtaining a reference point cloud of the goods at the initial position, and obtaining a scene point cloud during the handling of the goods, wherein the initial position is the position of the goods on the handling equipment before handling; converting the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud in the vehicle coordinate system, and a second target range point cloud corresponding to the scene point cloud; rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image; comparing the raster image with the other to determine the detection status during the handling of the goods. That is, in the present application, by obtaining the reference point cloud at the initial position and the scene point cloud during the handling process, converting them to the same coordinate system, and determining the first target range point cloud and the second target range point cloud of the effective range, and rasterizing one of them, and comparing the other with the raster image, and then determining the detection status of the goods, the detection accuracy during the handling of the goods can be effectively improved, and the corresponding detection efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0020] Figure 1 It is a flow chart of an embodiment of a method for detecting abnormality in handling goods in the present application;
[0021] Figure 2 It is a structural schematic diagram of an embodiment of a handling device in the present application;
[0022] Figure 3 It is a schematic diagram of the process of a forward detection embodiment of the present application;
[0023] Figure 4 It is a schematic diagram of the process of a reverse detection embodiment of the present application;
[0024] Figure 5 It is a structural schematic diagram of an embodiment of a system for detecting abnormalities in transported goods in the present application;
[0025] Figure 6 is a structural schematic diagram of an embodiment of an electronic device in the present application;
[0026] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium in the present application. DETAILED DESCRIPTION
[0027] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.
[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0029] Traditional cargo inspection methods usually use distance comparison to determine whether the cargo has fallen during transportation, or require prior data for comparison. However, using distance for judgment is prone to errors, and using prior data makes the solution more complicated, resulting in lower detection efficiency.
[0030] Therefore, a method for detecting abnormalities in transported goods is provided. The reference point cloud at the initial position and the scene point cloud during the transport process are obtained and converted into a self-vehicle coordinate system, and then the corresponding first target range point cloud and second target range point cloud are respectively obtained. One of them is rasterized, and the other is compared with the raster image. The detection status of the goods in the transport process is then obtained, which can effectively improve the accuracy of the detection results, does not require prior data, and correspondingly improves the detection efficiency.
[0031] The present application provides a method for detecting abnormalities in transported goods.
[0032] See also Figure 1 , Figure 1is a flow chart of an embodiment of a method for detecting abnormality in handling goods in the present application; it should be noted that if there are substantial results, the method of the present application is not based on Figure 1 The process sequence shown is limited.
[0033] like Figure 1 As shown, the method for detecting abnormality in the handling of goods of the present application may include the following steps:
[0034] S10, obtaining a reference point cloud of the cargo at an initial position, and obtaining a scene point cloud during cargo transportation, wherein the initial position is a position of the cargo on the transportation equipment before transportation.
[0035] Among them, cargo refers to items that need to be transported, which can be one or multiple items stacked together; the initial position refers to the position after the transport equipment lifts the cargo and before transporting it, which can be used as the starting point of transport. For example, the fork tines of the transport equipment lift the cargo, and the position before the transport equipment moves is used as the starting point of transport.
[0036] Specifically, after the handling equipment lifts the goods, the reference point cloud of the initial position is obtained by using the camera equipment; while the handling equipment is carrying the goods, the scene point cloud of the goods carrying process is obtained by using the camera equipment, wherein the scene point cloud can be the scene point cloud corresponding to any position in the carrying process.
[0037] It can be understood that the reference point cloud refers to the point cloud data corresponding to the entire initial position within the entire viewing angle of the camera device, including at least the point cloud data of the goods, and may also include the point cloud data of the fork tines of the handling equipment.
[0038] S20, converting the reference point cloud and the scene point cloud into the vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud and a second target range point cloud corresponding to the scene point cloud in the vehicle coordinate system.
[0039] Among them, the reference point cloud at the initial position corresponds to the camera coordinate system at the initial position, and the scene point cloud during the transportation process corresponds to the camera coordinate system during the transportation process. Because the positions of the two camera coordinate systems are different, they need to be converted to the same coordinate system, that is, converted to the vehicle coordinate system; the first target range point cloud refers to the valid point cloud data corresponding to the goods in the reference point cloud, and the second target range point cloud refers to the valid point cloud data corresponding to the goods in the scene point cloud.
[0040] Specifically, the camera extrinsic parameter data of the camera device is used to transform the reference point cloud from the camera coordinate system of the initial position to the vehicle coordinate system of the transporting device, and the scene point cloud is transformed from the camera coordinate system during the transport process to the vehicle coordinate system of the transporting device. Then, the valid point cloud data corresponding to the goods in the reference point cloud are obtained as the first target range point cloud, and the valid point cloud corresponding to the goods in the scene point cloud is obtained as the second target range point cloud.
[0041] S30 , rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image.
[0042] Among them, one of them is rasterized while the other is not rasterized, so as to reduce the time consumption caused by rasterizing multiple point cloud data.
[0043] Specifically, after obtaining the first target range point cloud and the second target range point cloud, the first target range point cloud is rasterized to determine the raster image corresponding to the first target range point cloud; or the second target range point cloud is rasterized to determine the raster image corresponding to the second target range point cloud, that is, one of them can be rasterized.
[0044] S40, comparing the grid image with another image to determine the detection status during the cargo handling process.
[0045] Among them, using a raster image to compare with another one refers to comparing the target range point cloud that has not been rasterized with the raster image, that is, projecting the target range point cloud that has not been rasterized into the raster image to determine the number of grid points in the raster image that exist in the target range point cloud that has not been rasterized.
[0046] Specifically, after obtaining the raster image obtained by rasterization processing, the target range point cloud that has not been rasterized is projected into the grid of the raster image, and then whether there is any abnormality with the goods is determined based on the number of grids with points of the target range point cloud projected. For example, the first target range point cloud is rasterized to obtain a raster image, and then the second target range point cloud is projected into the raster image. If the number of points of the second target range point cloud that are not in the valid grids of the raster image exceeds a given threshold, it is determined that there is an abnormality in the transportation of the goods, such as falling of the goods or abnormal interference. Otherwise, it indicates that the goods are normal during the transportation process.
[0047] In this embodiment, by acquiring reference point clouds and scene point clouds at different positions and converting them to the same coordinate system, a first target range point cloud and a second target range point cloud related to the goods are acquired, and then one of them is rasterized to obtain a raster image, and then the other target range point cloud that has not been rasterized is projected onto the raster image for comparison, and the inspection status of the goods during the transportation process is determined by the comparison results, which effectively improves the accuracy of the inspection results during the cargo transportation process and improves the inspection efficiency.
[0048] In some embodiments, at least one fixed camera device and at least one follow-up camera device are provided on the transport device.
[0049] See also Figure 2 , Figure 2 It is a structural schematic diagram of an embodiment of the transport equipment in the present application.
[0050] like Figure 2 As shown, the transport device 100 includes a device body 110 and forks 120 , wherein at least one fixed camera device D1 is fixedly disposed on the device body 110 , and at least one follow-up camera device D2 moves with the forks 120 .
[0051] For example, a first follow-up camera device D2 is arranged at the rear of the equipment body 110 of the transport equipment 100, and the rear is on the same side as the fork tine 120, that is, the first follow-up camera device D2 and the fork tine 120 are arranged on the same side; optionally, a second follow-up camera device D3 can be arranged on the top of the equipment body 110, and a fixed camera device D1 can be arranged on the side of the equipment body 110.
[0052] In some embodiments, step S10 obtains a reference point cloud of the cargo at an initial position, and obtains a scene point cloud during cargo transportation, wherein the initial position is a position of the cargo on the transportation equipment before transportation; and may include the following operations.
[0053] The scene point cloud may be collected at a fixed frequency, for example, once every 1 second.
[0054] After the forks of the handling equipment lift the goods on the pallet, at least one fixed camera device and at least one follow-up camera device are used as the position before handling to respectively obtain corresponding point cloud data, thereby determining the reference point cloud of the position before handling.
[0055] For example, the reference point cloud obtained by the camera device is: camera_i (x i ,y i ,z i ),i=1,2,…,n.
[0056] Among them, P camera_iis the camera coordinate system corresponding to the initial position, x i is the coordinate of the x-axis, y i is the coordinate of the y-axis, z i is the coordinate of the z-axis.
[0057] Next, when the transporting equipment is transporting the goods, at least one fixed camera device and at least one follow-up camera device are used to respectively acquire corresponding point cloud data, thereby determining a scene point cloud at a certain position during the transporting process.
[0058] The scene point cloud of each frame acquired by the camera device is: Q camera_i (x i ,y i ,z i ),i=1,2,…,n.
[0059] Among them, Q camera_i is the camera coordinate system corresponding to the handling process, x i is the coordinate of the x-axis, y i is the coordinate of the y-axis, z i is the coordinate of the z-axis.
[0060] It is understandable that there may be multiple scene point clouds, that is, the scene point clouds corresponding to each position in the transportation process are obtained in real time.
[0061] In some embodiments, step S20 converts the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtains the first target range point cloud corresponding to the reference point cloud in the vehicle coordinate system, and the second target range point cloud corresponding to the scene point cloud; the following operation steps may be included.
[0062] First, the preset camera extrinsics of the camera device are obtained, and then the preset camera extrinsics are used to transform the reference point cloud from the camera coordinate system at the initial position to the vehicle coordinate system of the transporting device, and the scene point cloud is transformed from the camera coordinate system during the transport process to the vehicle coordinate system of the transporting device.
[0063] Among them, the preset camera extrinsics of the camera device are parameters that describe the position and posture of the camera in the world coordinate system, usually including a rotation matrix and a translation vector; the camera coordinate system at the initial position is the camera coordinate system corresponding to the camera device for obtaining the reference point cloud, and the vehicle coordinate system is the vehicle coordinate system based on the transporting equipment; the camera coordinate system during the transport process is the camera coordinate system corresponding to the camera device for obtaining the scene point cloud.
[0064] Specifically, the camera extrinsic parameters of the camera device are obtained in advance, and then the camera coordinate system of the initial position is converted to the vehicle coordinate system of the transporting device using the camera extrinsic parameters, and the camera coordinate system during the transport process is converted to the vehicle coordinate system of the transporting device, that is, converted to the same coordinate system.
[0065] Next, the first point cloud coordinates corresponding to the reference point cloud in the vehicle coordinate system are obtained, and the second point cloud coordinates corresponding to the midfield scenic spot cloud in the vehicle coordinate system are obtained.
[0066] Among them, the first point cloud coordinates are the point cloud coordinates of the reference point cloud in the vehicle coordinate system, and the second point cloud coordinates are the point cloud coordinates of the scene point cloud in the vehicle coordinate system.
[0067] Specifically, after converting to the vehicle coordinate system, the point cloud coordinates of the reference point cloud in the vehicle coordinate system are obtained respectively, and the first height of the fork tine of the handling equipment at the initial position is recorded; and the point cloud coordinates of the scene point cloud in the vehicle coordinate system are obtained, and the second height of the fork tine at the position during the handling process is recorded.
[0068] For example: the camera coordinate system P corresponding to the reference point cloud camera_i Convert to the vehicle coordinate system P carbase_i , and the point cloud coordinates corresponding to the reference point cloud in the vehicle coordinate system are obtained as follows:
[0069]
[0070] And obtain the first height h of the fork tine of the handling equipment corresponding to the reference point cloud origin ;
[0071] And the point cloud coordinates corresponding to the scene point cloud in the vehicle coordinate system are obtained as follows:
[0072] Q carbase_i (x Qcarbase_i ,y Qcarbase_i ,z Qcarbase_i ),i=1,2,…,n.
[0073] And record the second height h of the fork tine of the handling equipment corresponding to the scene point cloud scene .
[0074] Among them, P i is the point cloud coordinate of the reference point cloud in the vehicle coordinate system, is the x-axis coordinate of the reference point cloud in the vehicle coordinate system, is the y-axis coordinate of the reference point cloud in the vehicle coordinate system, is the z-axis coordinate of the reference point cloud in the vehicle coordinate system; Q carbase_i is the point cloud coordinates of the scene point cloud in the vehicle coordinate system, is the x-axis coordinate of the scene point cloud in the vehicle coordinate system, is the y-axis coordinate of the scene point cloud in the vehicle coordinate system, It is the z-axis coordinate of the scene point cloud in the vehicle coordinate system.
[0075] Then, using the preset cargo information, the reference point cloud is cropped to determine the first target range point cloud corresponding to the reference point cloud, and the scene point cloud is cropped to determine the second target range point cloud corresponding to the scene point cloud.
[0076] Among them, the preset cargo information refers to information such as the height and width of the cargo.
[0077] Specifically, the reference point cloud is cropped using cargo information, i.e., actual height, width and other information of the cargo, to obtain a first valid area corresponding to the cargo in the reference point cloud, and then the first valid area is used as the first target range point cloud corresponding to the reference point cloud; and the scene point cloud is cropped using actual height, width and other information of the cargo, to obtain a second valid area corresponding to the cargo in the scene point cloud, and then the second valid area is used as the second target range point cloud corresponding to the scene point cloud.
[0078] For example, we can obtain the height, width and other relevant information of the goods on site, and then limit the ROI range of the point cloud. We can also use straight-through filtering to crop the point cloud above the fork tine. That is, we can obtain the effective area of the goods based on the pallet and handling equipment body used to place the goods on the fork tine:
[0079] x∈(x min ,x max ),y∈(y min ,y max ),z∈(z min ,z max ),x min ,x max ,y min ,y max ,z min ,z max .
[0080] Among them, x min is the minimum value of the x-axis, x max is the maximum value of the x-axis, y min is the minimum value of the y-axis, y max is the maximum value of the y axis, z min is the minimum value of the z axis, z max is the maximum value of the z-axis.
[0081] In some embodiments, the following operations may also be included.
[0082] Using a fixed camera and a follow-up camera, a first height of the fork tine of the transport equipment corresponding to the reference point cloud in the vehicle coordinate system is obtained, and a second height of the fork tine of the transport equipment corresponding to the midfield scenic spot cloud in the vehicle coordinate system is obtained.
[0083] The first height refers to the height of the fork tine of the transport equipment corresponding to the reference point cloud in the vehicle coordinate system, and the second height refers to the height of the fork tine of the transport equipment corresponding to the scene point cloud in the vehicle coordinate system.
[0084] Specifically, the camera extrinsics of the fixed camera device and the follow-up camera device are obtained, and then the corresponding camera extrinsics are used to determine the first height of the fork tine of the transport equipment corresponding to the reference point cloud in the vehicle coordinate system, and to determine the second height of the fork tine of the transport equipment corresponding to the exit point cloud in the vehicle coordinate system.
[0085] In response to the camera device for acquiring the scene point cloud being a fixed camera device, the second point cloud coordinates of the scene point cloud in the vehicle coordinate system are corrected using the first height and the second height.
[0086] If the camera device for acquiring the scene point cloud is a fixed camera device, the coordinates of the scene point cloud need to be adjusted.
[0087] Specifically, when the camera device for acquiring the scene point cloud is a fixed camera device, the first height of the fork tine of the transporting equipment corresponding to the acquired reference point cloud in the vehicle coordinate system and the second height of the fork tine of the transporting equipment corresponding to the scene point cloud in the vehicle coordinate system are used to correct the second point cloud coordinates of the scene point cloud in the vehicle coordinate system.
[0088] For example, the coordinates of the second point cloud of the scene point cloud in the vehicle coordinate system are:
[0089] Among them, Q carbase_i is the point cloud coordinates of the scene point cloud in the vehicle coordinate system, is the x-axis coordinate of the scene point cloud in the vehicle coordinate system, is the y-axis coordinate of the scene point cloud in the vehicle coordinate system, It is the z-axis coordinate of the scene point cloud in the vehicle coordinate system.
[0090] Set the first height of the fork tine of the handling equipment corresponding to the reference point cloud to h origin , the second height of the fork of the handling equipment corresponding to the scene point cloud is h scene .
[0091] The coordinates of the second point cloud after correction are as follows:
[0092]
[0093] in, is the corrected z-axis coordinate.
[0094] If the camera device that obtains the scene point cloud is a follow-up camera device, then Q modify_i =Q carbase_i .
[0095] Among them, Q carbase_i is the z-axis coordinate before correction.
[0096] In some embodiments, the following operations may also be included.
[0097] The goods are clustered using the preset clustering method to determine the point cloud cluster blocks of the goods.
[0098] The preset clustering method may be a region growing clustering method or a Euclidean clustering method, and the point cloud cluster block refers to a point cloud block obtained by clustering using the clustering method.
[0099] Specifically, for the scene point cloud, clustering is performed using a region growing clustering method or a Euclidean clustering method to obtain point cloud clustering blocks of the goods in the scene point cloud.
[0100] In response to the number of points in the point cloud cluster block being less than a given threshold, the point cloud cluster block is removed.
[0101] The given threshold refers to the threshold of the number of points in the point cloud aggregation block, which is used to remove invalid scene point clouds, that is, to remove interfering point clouds.
[0102] Specifically, the number of points in the point cloud in all point cloud aggregation blocks is obtained, and a given threshold is set. The number of points in the point cloud aggregation block is compared with the given threshold. If the number of points in the point cloud aggregation block is less than the given threshold, it is determined to be an invalid scene point cloud. Then, the point cloud aggregation blocks corresponding to the invalid scene point cloud need to be eliminated to obtain the valid scene point cloud corresponding to the scene point cloud.
[0103] It is understandable that the above-mentioned process of removing point cloud cluster blocks may also be performed on the reference point cloud to obtain a valid reference point cloud corresponding to the reference point cloud.
[0104] In some embodiments, the first target range point cloud is rasterized, and step S30 rasterizes one of the first target range point cloud and the second target range point cloud to determine a raster image; the following operations may be included.
[0105] First, the maximum and minimum values of each coordinate axis of the first target range point cloud in the vehicle coordinate system are obtained.
[0106] The vehicle coordinate system includes the X-axis, Y-axis and Z-axis, the maximum and minimum values of each coordinate axis, that is, the maximum and minimum values of the X-axis, the maximum and minimum values of the Y-axis and the maximum and minimum values of the Z-axis.
[0107] Specifically, after the first target range point cloud is rasterized, a raster image, i.e., a first raster image, is obtained, and then the maximum value x of the X axis is obtained in the vehicle coordinate system.p_max and the minimum value x p_min , the maximum value y of the Y axis p_max and the minimum value y p_min and the maximum value of the Z axis z p_max and the minimum value z p_min , and obtain the single grid distance d of the grid.
[0108] Next, the maximum value, the minimum value and the preset grid information are used to perform grid processing on the point cloud of the first target range, and the current grid and the current grid with points within the preset range of the current grid are determined as valid grids.
[0109] The preset grid information refers to grid size, grid cell distance and other grid-related information.
[0110] Specifically, after obtaining the maximum and minimum values of each axis corresponding to the first target range point cloud in the vehicle coordinate system, the first target range point cloud is rasterized according to parameters such as the maximum and minimum values of each axis, the grid size, and the single-grid distance of the grid to obtain a grid image; and the current grid is judged. When there are points of the first target range point cloud in the current grid and there are points of the first target range point cloud in the adjacent grids of the current grid, the current grid is taken as a valid grid.
[0111] For example, when there are points of the first target range point cloud in the current grid, and there are points of the first target range point cloud in the other eight grids adjacent to the current grid, the current grid is regarded as a valid grid; if there is at least one grid adjacent to the current grid that does not have points of the first target range point cloud, the current grid is regarded as an invalid grid, and the invalid grid can be discarded later.
[0112] Then, a first grid image is determined by combining multiple valid grids, wherein the first grid image is a grid image corresponding to the first target range point cloud.
[0113] Specifically, after acquiring a plurality of valid grids corresponding to the first target range point cloud, the plurality of valid grids are combined as a first grid image, that is, the first grid image represents the plurality of valid grids corresponding to the first target range point cloud.
[0114] For example, if the grid distance is set to d, the point cloud coordinates of the reference point cloud in the vehicle coordinate system are The corresponding grid labels are:
[0115]
[0116] Among them, x p_min ,x p_max ,y p_min ,y p_max ,zp_min ,z p_max They are the point cloud coordinates corresponding to the valid reference point clouds. The x-axis minimum value, x-axis maximum value, y-axis minimum value, y-axis maximum value, z-axis minimum value, z-axis maximum value; xindex i is an integer on the x-axis, yindex i An integer for the y-axis, zindex i is an integer for the z-axis.
[0117]
[0118] Among them, floor() is the downward rounding function, xindex min is the smallest integer on the x-axis, xindex max is the maximum integer of the x-axis, yindex min The smallest integer on the y-axis, yindex max The maximum integer of the y-axis, zindex min The minimum integer of the z-axis, zindex max The maximum integer on the z-axis.
[0119] Furthermore, in order to reduce the point cloud jitter caused by normal vehicle jitter, the effective array can be set according to the jitter requirement.
[0120] Specifically, set the index threshold to Δ index , according to the jitter requirements, set the valid array to Valid(i,y,k),
[0121] Among them, xvalid i ,yvalid j ,zvalid k ∈{true,false};
[0122] i=-Δ index ,-Δ index +1,…,xindex max +Δ index -1,xindex max +Δ index ;
[0123] j=-Δ index , -Δ index +1, ..., yindex max +Δ index -1,yindex max +Δ index ;
[0124] k=-Δ index , -Δindex +1, ..., zindex max +Δ index -1,zindex max +Δ index ;
[0125] Among them, Δ index is the jitter threshold, xvalid i is the valid value of the x-axis in the valid array, yvalid j is the valid value of the y-axis in the valid array, zvalid k A valid value for the z-axis in a valid array.
[0126] Used to record the valid grid of the reference point cloud. The default initial value of each value is false. For the jitter threshold Δ index It can be set in the following ways: For example, according to the customer, combined with the forklift structure and sensor accuracy, set the distance d that the cargo is allowed to move under the sensor cargo , then:
[0127]
[0128] For each valid point P of the reference point cloud i , and its corresponding grid number: xindex i , yindex i , zindex i , then judge each value in Valid(i, j, k) when the following judgment conditions are met at the same time:
[0129] i∈[xindex i -Δ index , xindex i +Δ index ];
[0130] j∈[yindex i -Δ index , yindex i +Δ index ];
[0131] k∈[zindex i -Δ index , zindex i +Δ index ];
[0132] Valid(i, j, k) = true.
[0133] Among them, floor() is a rounding function.
[0134] That is, the grid that meets the above conditions is a valid grid.
[0135] In this embodiment, the first target range point cloud is rasterized, and the first grid image corresponding to the first target range point cloud is obtained by determining the effective grid, which further improves the accuracy of the data and provides more accurate results for subsequent detection results.
[0136] Furthermore, step S40 determines a first raster image by combining multiple valid raster images, wherein the first raster image is a raster image corresponding to the first target range point cloud, and may include the following operations.
[0137] First, the second target range point cloud is projected onto the first grid image for comparison using the maximum value, the minimum value and the preset grid information to determine a first comparison result.
[0138] Among them, the second target range point cloud represents the goods in the scene point cloud.
[0139] Specifically, by utilizing the maximum value, minimum value and preset grid information, the second target range point cloud can be made to correspond to the first grid image corresponding to the first target range point cloud, that is, a first comparison result is obtained. Therefore, based on the first comparison result, it can be determined whether there is any abnormality in the transportation of the goods.
[0140] In response to the number of the second target range point cloud located outside the valid grid in the first comparison result being greater than a preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
[0141] Among them, in the first comparison result, if the number of the second target range point cloud located outside the valid grid is greater than the preset threshold, it indicates that the overlap between the second target range point cloud and the first target range point cloud is low, and there is a situation of cargo falling or abnormal interference, and then the detection result of the cargo during the transportation process is determined to be abnormal; if in the first comparison result, the number of the second target range point cloud located outside the valid grid is less than the preset threshold, it indicates that the overlap between the second target range point cloud and the first target range point cloud is high, there is no cargo falling or interference, and then the detection result of the cargo during the transportation process is determined to be normal.
[0142] For example, the point cloud According to the maximum and minimum values x p_min , x p_max ,y p_min ,y p_max , z p_min , z p_max and the single grid distance d in the preset grid information, projected onto the grid of the grid image to obtain the grid number as follows:
[0143]
[0144] in, is the grid number of the x-axis, is the grid number of the y-axis, The grid number for the z-axis.
[0145] A variable value0 is used to count the number of point cloud points with large changes. The initial value is set to 0.
[0146] like:
[0147] or
[0148] or
[0149] Then we have: value0 plus 1.
[0150] Then find the judgment condition and find the value in Valid(i, j, k). The corresponding value is true or false. If it is false, the value0 is increased by 1.
[0151] If the value0 is greater than the given threshold, the cargo is reported to have fallen or abnormal intervention. If the value0 is less than the given threshold, the camera equipment is operating safely.
[0152] In some embodiments, the effective range of the first target range point cloud is larger than the effective range of the second target range point cloud, so that when the second target range point cloud is projected into the first grid image corresponding to the first target range point cloud, it can be more clearly determined which grids contain points of the second target range point cloud, thereby improving the corresponding detection efficiency.
[0153] In some embodiments, a continuous multi-frame alarm may be set to further report the abnormality when the cargo falls or there is abnormal interference, so as to remind the handling equipment to stop moving and inspect the cargo.
[0154] See also Figure 3 , Figure 3 It is a flow chart of an embodiment of forward detection in the present application.
[0155] like Figure 3As shown in the figure, the obtained reference point cloud is converted from the camera coordinate system to the vehicle coordinate system; the reference point cloud is cropped using the preset cargo information to obtain the first target range point cloud; the first target range point cloud is clustered and the cluster blocks with the number of points less than a given threshold are removed; the maximum and minimum values of each axis of the first target range point cloud are obtained, and then the first target range point cloud is rasterized using the maximum and minimum values of each axis and the preset grid information, and the first grid image corresponding to the valid grid is determined; and the obtained scene point cloud is converted from the camera coordinate system to the vehicle coordinate system. If it is a fixed camera device, the scene point cloud is The cloud is height-corrected; the scene point cloud is cropped using the preset cargo information to obtain a second target range point cloud, the effective range of the second target range point cloud is smaller than the effective range of the first target range point cloud; the second target range point cloud is clustered and cluster blocks with a point number less than a given threshold are removed; the second target range point cloud is projected onto the first grid image for comparison using the maximum and minimum values of each axis and the preset grid information to determine a first comparison result; and if the number of the second target range point cloud outside the effective grid in the first comparison result is greater than the preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
[0156] In this embodiment, the first target range point cloud is rasterized as forward detection, that is, real-time detection is performed one by one from the beginning to the transportation process. The detection results of the goods in the transportation process can be accurately determined from the forward direction, which can effectively improve the detection accuracy during the cargo transportation process and improve the corresponding detection efficiency.
[0157] In some embodiments, the reference point cloud and the scene point cloud can be swapped, that is, the steps for the original point cloud and the scene point cloud can be swapped, that is, the operation for the original point cloud is replaced by the operation for the scene point cloud; the operation for the scene point cloud is replaced by the operation for the original point cloud, and a threshold value1 can also be obtained. If value1 is greater than the given threshold, it is reported that the goods have fallen or abnormal intervention, and if value1 is less than the given threshold, the camera device is operating normally.
[0158] Specifically, the second target range point cloud is rasterized, and step S20 is to rasterize one of the first target range point cloud and the second target range point cloud to determine a raster image; the following operations may be included.
[0159] First, the maximum and minimum values of each coordinate axis of the second target range point cloud in the vehicle coordinate system are obtained.
[0160] Specifically, after the second target range point cloud is rasterized, a raster image, i.e., a second raster image, is obtained, and then in the vehicle coordinate system, the maximum and minimum values of the X-axis, the maximum and minimum values of the Y-axis, and the maximum and minimum values of the Z-axis, as well as the single grid distance d of the grid are obtained.
[0161] Next, the maximum value, the minimum value and the preset grid information are used to perform grid processing on the point cloud of the second target range, and the current grid and the current grid with points within the preset range of the current grid are determined as valid grids.
[0162] The preset grid information refers to grid size, grid cell distance and other grid-related information.
[0163] Specifically, after obtaining the maximum and minimum values of each axis corresponding to the second target range point cloud in the vehicle coordinate system, the second target range point cloud is rasterized according to parameters such as the maximum and minimum values of each axis, the grid size, and the single-grid distance of the grid to obtain a grid image; and the current grid is judged. When there are points of the second target range point cloud in the current grid and there are points of the second target range point cloud in the adjacent grids of the current grid, the current grid is taken as a valid grid.
[0164] For example, when there are points of the second target range point cloud in the current grid, and there are also points of the second target range point cloud in the other eight grids adjacent to the current grid, the current grid is regarded as a valid grid; if there is at least one grid adjacent to the current grid that does not have points of the second target range point cloud, the current grid is regarded as an invalid grid, and the invalid grid can be discarded later.
[0165] Then, a second grid image is determined by combining multiple valid grids, wherein the second grid image is a grid image corresponding to the second target range point cloud.
[0166] Specifically, after acquiring a plurality of valid grids corresponding to the second target range point cloud, the plurality of valid grids are combined as a second grid image, that is, the second grid image represents the plurality of valid grids corresponding to the second target range point cloud.
[0167] In this embodiment, the second target range point cloud is rasterized, and the second raster image corresponding to the second target range point cloud is obtained by determining the effective grid, which further improves the accuracy of the data and provides more accurate results for subsequent detection results.
[0168] Furthermore, step S40 determines a second raster image by combining multiple valid raster images, wherein the second raster image is a raster image corresponding to the second target range point cloud, and may include the following operations.
[0169] Firstly, the maximum value, the minimum value and the preset grid information are used to project the first target range point cloud onto the second grid image for comparison, thereby determining a second comparison result.
[0170] Among them, the first target range point cloud represents the cargo of the reference point cloud.
[0171] Specifically, by utilizing the maximum value, minimum value and preset grid information, the first target range point cloud can be made to correspond to the second grid image corresponding to the second target range point cloud, that is, a second comparison result is obtained. Therefore, based on the second comparison result, it can be determined whether there is any abnormality in the transportation of the goods.
[0172] In response to the number of first target range point clouds outside the valid grid in the second comparison result being greater than a preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
[0173] Among them, in the second comparison result, if the number of the first target range point cloud located outside the valid grid is greater than the preset threshold, it indicates that the overlap between the first target range point cloud and the second target range point cloud is low, and there is a situation of cargo falling or abnormal interference, and then it is determined that the detection result of the cargo during the transportation process is abnormal; if in the second comparison result, the number of the first target range point cloud located outside the valid grid is less than the preset threshold, it indicates that the overlap between the first target range point cloud and the second target range point cloud is high, there is no cargo falling or interference, and then it is determined that the detection result of the cargo during the transportation process is normal.
[0174] In some embodiments, the effective range of the second target range point cloud is larger than the effective range of the first target range point cloud, so that when the first target range point cloud is projected into the second grid image corresponding to the second target range point cloud, it can be more clearly determined which grids contain points of the second target range point cloud, thereby improving the corresponding detection efficiency.
[0175] In some embodiments, a continuous multi-frame alarm may be set to further report the abnormality when the cargo falls or there is abnormal interference, so as to remind the handling equipment to stop moving and inspect the cargo.
[0176] See also Figure 4 , Figure 4 It is a flowchart of an embodiment of reverse detection in this application.
[0177] like Figure 4As shown in the figure, the acquired scene point cloud is converted from the camera coordinate system to the vehicle coordinate system; the scene point cloud is cropped using the preset cargo information to obtain the second target range point cloud; the second target range point cloud is clustered and the cluster blocks with the number of points less than a given threshold are removed; the maximum and minimum values of each axis of the second target range point cloud are obtained, and then the second target range point cloud is rasterized using the maximum and minimum values of each axis and the preset grid information, and the second grid image corresponding to the valid grid is determined; and the acquired reference point cloud is converted from the camera coordinate system to the vehicle coordinate system. If it is a fixed camera device, the reference point The cloud is height-corrected; the reference point cloud is cropped using the preset cargo information to obtain a first target range point cloud, the effective range of the first target range point cloud is smaller than the effective range of the second target range point cloud; the first target range point cloud is clustered and cluster blocks with a point number less than a given threshold are removed; the first target range point cloud is projected onto the second grid image for comparison using the maximum and minimum values of each axis and the preset grid information to determine a second comparison result; and if the number of the first target range point cloud outside the effective grid in the second comparison result is greater than the preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
[0178] In this embodiment, the second target range point cloud is rasterized as reverse detection, that is, real-time detection is performed one by one from the transportation process to the starting position. The detection results of the goods in the transportation process can be accurately determined in the reverse direction, which can effectively improve the detection accuracy of the goods in the transportation process and improve the corresponding detection efficiency.
[0179] In some embodiments, the present application also provides a system for detecting abnormalities in transported goods.
[0180] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a system for detecting abnormalities in transported goods in the present application.
[0181] like Figure 5As shown, the cargo handling abnormality detection system 400 includes: a handling device 100, a camera device D, a conversion module 410, a grid module 420 and a determination module 430; wherein the cargo is placed on the fork tine of the handling device 100; the camera device D is arranged on the handling device 100, and is used to obtain a reference point cloud of the cargo at an initial position, and to obtain a scene point cloud during cargo handling, wherein the initial position is a position of the cargo on the handling device before handling; the conversion module 410 is used to convert the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtain a first target range point cloud corresponding to the reference point cloud in the vehicle coordinate system, and a second target range point cloud corresponding to the scene point cloud; the grid module 420 is used to rasterize one of the first target range point cloud and the second target range point cloud to determine a grid image; the determination module 430 compares the grid image with the other to determine the detection situation during cargo handling.
[0182] In some embodiments, an electronic device is also provided.
[0183] See also Figure 6 , Figure 6 1 is a schematic diagram of the structure of an electronic device in the present application. The electronic device can execute the steps of the method for detecting abnormality in the transported goods in the above method.
[0184] The electronic device 500 includes a memory 520, a processor 510 coupled to the memory, and at least one computer program stored in the memory 520 and executable on the processor 510; when the processor 510 loads and executes the at least one computer program, it is used to implement the steps of the method for detecting abnormality in the transport of goods in the above method. For related contents, please refer to the detailed description of the above method, which will not be repeated here.
[0185] In some embodiments, a computer-readable storage medium is also included.
[0186] See also Figure 7 , Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium in the present application.
[0187] The computer readable storage medium 600 stores at least one program 610, which, when loaded and executed by the processor, is used to implement the steps of the method for detecting abnormality of transported goods in the above method. For related contents, please refer to the detailed description of the above method, which will not be repeated here.
[0188] The above scheme obtains the reference point cloud of the initial position and the scene point cloud during the transportation process, converts them into the same coordinate system, and determines the first target range point cloud and the second target range point cloud of the effective range, and rasterizes one of them, and compares the other with the raster image to determine the inspection status of the goods, which can effectively improve the inspection accuracy during the cargo transportation process and improve the corresponding inspection efficiency.
[0189] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0190] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0191] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0192] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0193] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for detecting abnormality in transported goods, characterized in that: include: Acquire a reference point cloud of the cargo at an initial position, and acquire a scene point cloud during the cargo transportation process, wherein the initial position is a position of the cargo on the transportation equipment before transportation; Converting the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud and a second target range point cloud corresponding to the scene point cloud in the vehicle coordinate system; rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image; The grid image is compared with another image to determine the detection status of the cargo handling process.
2. The method according to claim 1, characterized in that The step of rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image includes: Obtaining the maximum and minimum values of each coordinate axis of the first target range point cloud in the vehicle coordinate system; Utilizing the maximum value, the minimum value and the preset grid information, grid processing is performed on the point cloud of the first target range, and a current grid and a current grid with points within the preset range of the current grid are determined as valid grids; A first grid image is determined by combining a plurality of the valid grids, wherein the first grid image is a grid image corresponding to the first target range point cloud.
3. The method according to claim 2, characterized in that The method of comparing the grid image with another image to determine the detection status of the cargo handling process includes: Using the maximum value, the minimum value and the preset grid information, projecting the second target range point cloud onto the first grid image for comparison, and determining a first comparison result; In response to the number of the second target range point cloud located outside the valid grid in the first comparison result being greater than a preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
4. The method according to claim 2, characterized in that: The effective range of the first target range point cloud is greater than the effective range of the second target range point cloud.
5. The method according to claim 1, characterized in that The step of rasterizing one of the first target range point cloud and the second target range point cloud to determine a raster image includes: Obtaining the maximum and minimum values of each coordinate axis of the second target range point cloud in the vehicle coordinate system; Utilizing the maximum value, the minimum value and the preset grid information, grid processing is performed on the point cloud of the second target range, and a current grid and a current grid having points within the preset range of the current grid are determined as valid grids; A second raster image is determined by combining a plurality of the valid raster images, wherein the second raster image is a raster image corresponding to the second target range point cloud.
6. The method according to claim 5, characterized in that The comparing the grid image with another image to determine the detection status during the cargo handling process includes: Using the maximum value, the minimum value and the preset grid information, projecting the first target range point cloud onto the second grid image for comparison, and determining a second comparison result; In response to the number of the first target range point cloud located outside the valid grid in the second comparison result being greater than a preset threshold, it is determined that the detection result in the cargo handling process is abnormal.
7. The method according to claim 6, characterized in that The effective range of the second target range point cloud is greater than the effective range of the first target range point cloud.
8. The method according to claim 1, characterized in that The transport device is provided with at least one fixed camera device and at least one follow-up camera device, wherein the fixed camera device is fixedly arranged on the transport device, and the follow-up camera device moves following the fork tines of the transport device; The step of obtaining a reference point cloud of the cargo at an initial position and obtaining a scene point cloud during the cargo transportation process includes: Using at least one of the fixed camera devices and at least one of the follow-up camera devices, a reference point cloud corresponding to the position of the goods before being transported on the transporting device is obtained, and a scene point cloud during the transporting process of the goods being transported by the transporting device is obtained.
9. The method according to claim 8, characterized in that The converting the reference point cloud and the scene point cloud to the vehicle coordinate system, and obtaining a first target range point cloud corresponding to the reference point cloud and a second target range point cloud corresponding to the scene point cloud in the vehicle coordinate system, includes: Using preset camera extrinsics of the fixed camera device and the follow-up camera device, the reference point cloud is transformed from the camera coordinate system at the initial position to the self-vehicle coordinate system, and the scene point cloud is transformed from the camera coordinate system during the transport process to the self-vehicle coordinate system; Acquire first point cloud coordinates corresponding to the reference point cloud in the vehicle coordinate system, and acquire second point cloud coordinates corresponding to the scene point cloud in the vehicle coordinate system; The reference point cloud is cropped using preset cargo information to determine the first target range point cloud corresponding to the reference point cloud, and the scene point cloud is cropped to determine the second target range point cloud corresponding to the scene point cloud.
10. The method according to claim 9, characterized in that Also includes: Using the fixed camera device and the follow-up camera device, obtain a first height of the fork tine of the transport device corresponding to the reference point cloud in the self-vehicle coordinate system, and obtain a second height of the fork tine of the transport device corresponding to the scene point cloud in the self-vehicle coordinate system; In response to the camera device that acquires the scene point cloud being the fixed camera device, the second point cloud coordinates of the scene point cloud in the vehicle coordinate system are corrected using the first height and the second height.
11. The method according to claim 9, characterized in that Also includes: Clustering the goods using a preset clustering method to determine point cloud cluster blocks of the goods; In response to the number of points in the point cloud aggregation block being less than a given threshold, the point cloud aggregation block is removed.
12. An electronic device, characterized in that: The electronic device includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method for detecting abnormalities in handling goods as described in any one of claims 1-11.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium has at least one program, and when the at least one program is loaded and executed by the processor, it is used to implement the method for detecting abnormalities in transported goods as described in any one of claims 1-11.