Method, device, equipment, medium and program product for determining packing status

Abstract

The present application provides a method, device, equipment, medium and program product for determining the packing status, the method comprising: after obtaining the first point cloud data collected by the first radar installed on the top of the tractor of the container truck and the second point cloud data collected by the second radar installed at the rear of the trailer of the container truck, according to the posture of the first radar, the posture of the second radar, and the structural parameters of the trailer, the target point cloud data is determined from the first point cloud data and the second point cloud data, when the number of point clouds of the target point cloud data is greater than the preset number of point clouds, according to the target point cloud data, the first plane of the container close to the tractor and the second plane of the container close to the rear of the trailer are obtained, so as to determine the position of the container according to the first plane and the second plane. In this technical solution, the packing status is determined by the target point cloud data, which effectively improves the accuracy of detecting the container status.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, equipment, medium and program product for determining a packing status. Background Art

[0002] With the gradual development of port intelligence, more and more terminal operations can be carried out through smart container trucks, which effectively improves the efficiency and safety of terminal operations and reduces labor costs.

[0003] At present, the smart container truck consists of a tractor and a trailer. During the operation, it is necessary to accurately sense the status of the container on the trailer in real time to achieve the purpose of completing the terminal operation process and ensuring the safety of the operation. In the prior art, the real-time and accurate sensing of the status of the container on the trailer is mainly achieved by using the detection equipment installed on the smart container truck (such as a laser radar installed on the tractor, or an ultrasonic or pressure sensor installed on the trailer) to collect data on the area where the container can be placed on the trailer in real time, and obtain the status of the container on the trailer based on the data.

[0004] However, when there are two containers on the trailer, the front box will block the rear box, resulting in the inability of existing technologies to accurately determine whether the rear box is installed or the loading position of the rear box through detection equipment, which limits the operating scenarios supported by smart container trucks. Summary of the invention

[0005] The present application provides a method, device, equipment, medium and program product for determining the packing status to solve the problem in the prior art that it is impossible to accurately determine whether a rear box is installed or the loading position of the rear box through detection equipment, thereby limiting the operating scenarios supported by smart container trucks.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a packing state, comprising:

[0007] Acquire first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck;

[0008] According to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, target point cloud data is determined from the first point cloud data and the second point cloud data, wherein the target point cloud data is point cloud data of an area in the trailer for placing a container;

[0009] When the number of point clouds of the target point cloud data is greater than the preset number of point clouds, obtaining a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer according to the target point cloud data;

[0010] A packing status is determined according to the first plane and the second plane, where the packing status includes at least one of a loading position of a container, a number of containers, and a type of container.

[0011] In a possible design of the first aspect, determining target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer includes:

[0012] The first point cloud data is converted from the coordinate system where the first radar is located to the coordinate system in front of the vehicle, and the second point cloud data is converted from the coordinate system where the second radar is located to the coordinate system in front of the vehicle, to obtain initial point cloud data, and the coordinate system in front of the vehicle is the coordinate system where the container truck is located;

[0013] Determine the ROI of the container according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer;

[0014] The initial point cloud data in the ROI is determined as the target point cloud data.

[0015] Optionally, when the number of point clouds of the target point cloud data is greater than a preset number of point clouds, obtaining a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer according to the target point cloud data includes:

[0016] When the number of point clouds in the target point cloud data is greater than the preset number of point clouds, points in the target point cloud data whose angles between the normal vector and the X-axis of the front vehicle coordinate system are greater than or equal to the preset angle are deleted to obtain third point cloud data;

[0017] The first plane and the second plane are extracted from the third point cloud data using a RANSAC method.

[0018] Optionally, extracting the first plane and the second plane from the third point cloud data using a RANSAC method includes:

[0019] Extracting a first initial plane from the third point cloud data using a RANSAC method;

[0020] Deleting points in the third point cloud data whose distance from the first initial plane is less than a preset distance, to obtain fourth point cloud data;

[0021] Extracting a second initial plane from the fourth point cloud data using a RANSAC method;

[0022] Calculating a first intersection point of the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point of the second initial plane and the X-axis of the vehicle front coordinate system;

[0023] The initial plane corresponding to the intersection point with the smallest absolute value of coordinates is determined as the first plane, and the initial plane corresponding to the intersection point with the largest absolute value of coordinates is determined as the second plane.

[0024] In another possible design of the first aspect, determining the position of the container according to the first plane and the second plane includes:

[0025] According to the structural parameters of the trailer, the first plane and the second plane are converted into a trailer coordinate system to obtain the first plane after the coordinate conversion and the second plane after the coordinate conversion;

[0026] The packing state is determined according to the first plane after the coordinates are transformed and the second plane after the coordinates are transformed.

[0027] In another possible design of the first aspect, after determining the target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, the method further includes:

[0028] When the point cloud quantity of the target point cloud data is less than or equal to the preset point cloud quantity, it is determined that the container is not loaded on the trailer.

[0029] In a second aspect, an embodiment of the present application provides a device for determining a packing state, including:

[0030] An acquisition module, used to acquire first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck;

[0031] a processing module, configured to determine target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, wherein the target point cloud data is point cloud data of an area in the trailer for placing a container;

[0032] The processing module is further configured to obtain, according to the target point cloud data, a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer when the number of point clouds of the target point cloud data is greater than a preset number of point clouds;

[0033] The processing module is further used to determine a packing status according to the first plane and the second plane, wherein the packing status includes at least one of a loading position of a container, a number of containers, and a type of container.

[0034] In a possible design of the second aspect, the processing module is specifically configured to:

[0035] The first point cloud data is converted from the coordinate system where the first radar is located to the coordinate system in front of the vehicle, and the second point cloud data is converted from the coordinate system where the second radar is located to the coordinate system in front of the vehicle, to obtain initial point cloud data, and the coordinate system in front of the vehicle is the coordinate system where the container truck is located;

[0036] Determine the ROI of the container according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer;

[0037] The initial point cloud data in the ROI is determined as the target point cloud data.

[0038] Optionally, the processing module is specifically used to:

[0039] When the number of point clouds in the target point cloud data is greater than the preset number of point clouds, points in the target point cloud data whose angles between the normal vector and the X-axis of the front vehicle coordinate system are greater than or equal to the preset angle are deleted to obtain third point cloud data;

[0040] The first plane and the second plane are extracted from the third point cloud data using a RANSAC method.

[0041] Optionally, the processing module is specifically used to:

[0042] Extracting a first initial plane from the third point cloud data using a RANSAC method;

[0043] Deleting points in the third point cloud data whose distance from the first initial plane is less than a preset distance, to obtain fourth point cloud data;

[0044] Extracting a second initial plane from the fourth point cloud data using a RANSAC method;

[0045] Calculating a first intersection point of the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point of the second initial plane and the X-axis of the vehicle front coordinate system;

[0046] The initial plane corresponding to the intersection point with the smallest absolute value of coordinates is determined as the first plane, and the initial plane corresponding to the intersection point with the largest absolute value of coordinates is determined as the second plane.

[0047] In another possible design of the second aspect, the processing module is specifically configured to:

[0048] According to the structural parameters of the trailer, the first plane and the second plane are converted into a trailer coordinate system to obtain the first plane after the coordinate conversion and the second plane after the coordinate conversion;

[0049] The packing state is determined according to the first plane after the coordinates are transformed and the second plane after the coordinates are transformed.

[0050] In a possible design of the second aspect, after determining the target point cloud data from the first point cloud data and the second point cloud data according to the posture of the first radar, the posture of the second radar, and the structural parameters of the trailer, the processing module is further used to:

[0051] When the point cloud quantity of the target point cloud data is less than or equal to the preset point cloud quantity, it is determined that the container is not loaded on the trailer.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the first aspect and the methods provided by various possible designs when executing the computer program instructions.

[0053] In a fourth aspect, an embodiment of the present application may provide a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect and the methods provided by various possible designs.

[0054] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the first aspect and the methods provided by various possible designs.

[0055] The method, device, equipment, medium and program product for determining the packing state provided by the embodiment of the present application, in which: after obtaining the first point cloud data collected by the first radar installed on the top of the tractor of the container truck and the second point cloud data collected by the second radar installed at the rear of the trailer of the container truck, the target point cloud data is determined from the first point cloud data and the second point cloud data according to the position of the first radar, the position of the second radar, and the structural parameters of the trailer. When the number of point clouds of the target point cloud data is greater than the preset number of point clouds, the first plane of the container close to the tractor and the second plane of the container close to the rear of the trailer are obtained according to the target point cloud data, so as to determine the packing state according to the first plane and the second plane. Wherein, the packing state includes at least one of the loading position of the container, the number of containers and the box type of the container. In this technical solution, by determining the target point cloud data of the area in the trailer for placing the container from the first point cloud data and the second point cloud data, so that the position of the container is determined according to the target point cloud data, the accuracy of detecting the container state is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0057] Figure 1 A schematic diagram of an application scenario of the method for determining the packing status provided in an embodiment of the present application;

[0058] Figure 2 A flowchart of a first embodiment of a method for determining a packing status provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a flow chart of a second embodiment of a method for determining a packing status provided in an embodiment of the present application;

[0060] Figure 4 A flowchart of a third embodiment of a method for determining a packing status provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of a device for determining a packing state provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0063] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] A smart container truck consists of a tractor and a trailer. During operation, it is usually necessary to accurately sense the loading status of the container on the trailer of the vehicle (that is, the container truck) in real time to complete the operation process or ensure operation safety.

[0066] In the prior art, real-time and accurate perception of the status of the container on the trailer can be achieved in the following ways:

[0067] (1) The lidar installed on the top of the tractor collects data on the container-placeable area of ​​the trailer in real time, and obtains the status of the containers on the trailer based on the data. However, this method cannot determine whether there is a rear container when the front container is blocked, and cannot obtain the location of the rear container.

[0068] (2) Based on the above method, the sensor bracket can be raised so that the laser radar is higher than the container, thereby obtaining the status of the container on the trailer. However, this method poses a great challenge to the stability of the bracket, has poor safety, and is costly.

[0069] (3) Track the status of the container on the trailer by installing ultrasonic or pressure sensors on the trailer or based on external information (such as operation instructions). However, this method can only determine whether there is a rear container, but cannot give the location of the rear container well. It is also easily restricted by the operation scene and has a high dependence on the operation scene.

[0070] In summary, the existing technology can handle large-box operation scenarios, but cannot handle small-box operation scenarios well. When there are two containers on the trailer, the front box will block the rear box, making it impossible to accurately obtain the status of the rear box through the detection equipment, and there is a problem of poor accuracy in the detected container status.

[0071] Therefore, the present application provides a method for determining the loading status, in which a first radar is installed on the top of the tractor of the container truck, and a second radar is installed at the rear of the trailer of the container truck. The first radar and the second radar can scan and obtain the target point cloud data of the area in the trailer for placing the container, so that the first plane of the container close to the tractor and the second plane of the container close to the rear of the trailer can be obtained according to the target point cloud data, and the position of the container can be obtained according to the first plane and the second plane. When there are two containers on the trailer, the first plane is the plane of the front box close to the tractor, and the second plane is the plane of the rear box close to the rear of the trailer, so that the loading status can be accurately obtained according to the first plane and the second plane, thereby improving the accuracy of the detected container status.

[0072] For example, the method for determining the packing status provided in the embodiment of the present application can be applied to Figure 1 In a schematic diagram of an application scenario shown. Figure 1 The present invention provides a schematic diagram of an application scenario of the method for determining the packing status provided in the embodiment of the present application, which is used to solve the above technical problems. Figure 1 As shown, the application scenario may include: a tractor 11 , a trailer 12 , a radar 13 , a radar 14 and an electronic device 15 .

[0073] The tractor 11 is used to control the movement of the trailer 12, and the trailer 12 is used to place the container. The tractor 11 and the trailer 12 form a container truck. Figure 1 The container truck in the figure is the front view of the container truck.

[0074] In this embodiment, the radar 13 is installed at the head of the tractor 11, and the radar 14 is installed at the tail of the trailer 12. The electronic device 15 can obtain the first point cloud data collected by the radar 13, and can also obtain the second point cloud data collected by the radar 14, and execute the program code of the method for determining the packing state, so as to determine the packing state of the trailer 12.

[0075] It should be understood that the electronic device can be a controller in the container truck, or a terminal device independent of the container truck, such as a dispatcher's mobile phone, tablet computer, laptop computer, etc., or a server. The embodiment of the present application does not specifically limit the type of electronic device. Whether the electronic device is a controller in the container truck, a terminal device, or a server can be determined according to actual conditions.

[0076] The technical solution of the present application is described in detail below through specific embodiments.

[0077] It should be noted that the following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0078] Figure 2This is a flow chart of a first embodiment of a method for determining a packing status provided in an embodiment of the present application. Figure 2 As shown, the method for determining the packing status may include the following steps:

[0079] S201, obtaining first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck.

[0080] Optionally, the first radar can be installed horizontally or vertically on the top of the tractor, and similarly, the second radar can be installed horizontally or vertically at the rear of the trailer. The embodiment of the present application does not limit the installation method of the first radar and the second radar. The first radar is used to collect first point cloud data and send the first point cloud data to the electronic device. Similarly, the second radar is used to collect second point cloud data and send the second point cloud data to the electronic device. For the electronic device, the first point cloud data sent by the first radar and the second point cloud data sent by the second radar are received.

[0081] Optionally, the radar (that is, the first radar and / or the second radar) can collect point cloud data in real time, and can also collect point cloud data at a preset frequency. For example, the preset frequency can be 1 time per second, 2 times per second, 3 times per second, etc. Point cloud data can also be collected when the truck body is straightened. It can be determined based on actual conditions, and the embodiments of the present application are not limited to this.

[0082] The first radar can scan the plane of the container loaded on the trailer close to the tractor, and the second radar can scan the plane of the container loaded on the trailer close to the rear of the trailer. Optionally, the above containers can be the same or different containers.

[0083] Optionally, the radar may be a multi-line laser radar, such as a 16-line mechanical laser radar, or other radar capable of acquiring point cloud data, without specific limitation.

[0084] Optionally, the radar and the electronic device may be connected via wireless communication or via physical lines.

[0085] S202: Determine target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer.

[0086] The target point cloud data is the point cloud data of the area in the trailer for placing the container.

[0087] It should be understood that the area may be loaded with containers or not loaded with containers. That is, when the trailer is loaded with containers, the target point cloud data is the point cloud data of the container and the point cloud data of the trailer, and when the trailer is not loaded with containers, the target point cloud data is the point cloud data of the trailer for placing the container area.

[0088] In an embodiment of the present application, since the first point cloud data and the second point cloud data include point cloud data of the area in the trailer for placing containers, and also include point cloud data of other areas of the trailer, the first point cloud data and the second point cloud data can be respectively converted to the front coordinate system of the vehicle to obtain initial point cloud data, and the region of interest (ROI) of the container can be determined according to the posture of the first radar, the posture of the second radar and the structural parameters of the trailer. The ROI is the area in the trailer for placing containers, and the initial point cloud data in the ROI is determined as the target point cloud data.

[0089] Among them, ROI refers to the area that needs to be processed, which is outlined in the form of a box, circle, ellipse, irregular polygon, etc. By determining the ROI, the point cloud data in the ROI area can be analyzed and processed in a targeted manner, reducing the subsequent processing time and processing resources and increasing the processing accuracy.

[0090] Optionally, the structural parameters of the trailer may be the size, shape, height, etc. of the trailer.

[0091] For the specific implementation of this step, please refer to the following Figure 3 The description in the illustrated embodiment will not be repeated here.

[0092] S203. When the number of point clouds of the target point cloud data is greater than the preset number of point clouds, a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer are obtained according to the target point cloud data.

[0093] Optionally, the preset number of point clouds can be 50, 60, 70, etc., which can be determined based on actual conditions, and the embodiments of the present application do not impose specific restrictions on this.

[0094] In one feasible method, points in the target point cloud data whose angles between the normal vector and the X-axis of the vehicle front coordinate system are greater than or equal to a preset angle can be deleted to obtain the third point cloud data, and the first plane and the second plane can be extracted from the third point cloud data using the Random Sample Consensus (RANSAC) method, thereby improving the accuracy and efficiency of extracting the first plane and the second plane.

[0095] Exemplarily, the preset angle may be 10 degrees, 12 degrees, 14 degrees, etc., which may be determined according to actual conditions, and the embodiments of the present application do not impose specific limitations on this.

[0096] The specific implementation of extracting the first plane and the second plane from the third point cloud data using the RANSAC method can be found in the following Figure 4 The description in the illustrated embodiment will not be repeated here.

[0097] S204: Determine a packing state according to the first plane and the second plane.

[0098] The packing status includes at least one of the loading position of the container, the number of containers, and the type of the container.

[0099] In a possible implementation, the first plane and the second plane can be transformed into the trailer coordinate system according to the structural parameters of the trailer, and the first plane after the coordinate transformation and the second plane after the coordinate transformation are obtained. Then, the position of the container is determined according to the first plane after the coordinate transformation and the second plane after the coordinate transformation, so that the loading position of the rear box can be accurately obtained even when the front box is blocked.

[0100] The trailer coordinate system refers to the coordinate system where the trailer is located, and its origin may be the position closest to the tractor in the area of ​​the trailer for placing containers, the X-axis is parallel to the ground and the positive direction extends backward along the rear of the trailer, the Y-axis is parallel to the ground and the positive direction extends outward perpendicular to the X-axis, and the Z-axis is perpendicular to the ground and the positive direction extends away from the ground. Optionally, the position closest to the tractor in the area of ​​the trailer for placing containers can be obtained through real-time detection, container truck structure drawings or measurement calculations, and this application does not impose specific restrictions on this.

[0101] The X coordinate of the first plane after the coordinate conversion can be used as the first distance, and the X coordinate of the second plane after the coordinate conversion can be used as the second distance, and the position of the container can be determined according to the first distance and the second distance.

[0102] Optionally, the type of container may be determined according to the first distance and the second distance. The types of containers include large containers, medium containers and small containers. A trailer may be equipped with a large container, a medium container, or at most two small containers, wherein the small container close to the tractor is the front container, and the small container close to the rear of the trailer is the rear container.

[0103] Exemplarily, determining the type and quantity of containers according to the first distance and the second distance may include the following aspects:

[0104] (1) When the first distance is greater than the front box position threshold, and the difference between the first distance and the second distance is less than or equal to the small box length threshold, it is determined that the position of the container has a front box.

[0105] For example, the front box position threshold is related to the working environment of the container truck and the trailer structure, and is generally less than half the length of the small box. For example, the front box position threshold may be 3 meters at a container terminal. It should be understood that the front box position threshold may also be other values, and the embodiments of the present application do not specifically limit this.

[0106] Exemplarily, the small box length threshold may be less than the length of a standard small box, where the length of a standard small box may be 6.058 meters. For example, the small box length threshold may be a length of 3 meters to 5 meters, such as 3 meters, 4 meters, 5 meters, etc., and this application does not impose specific restrictions on this. When the small box length threshold is less than the length of a standard small box, the robustness of the calculation can be effectively improved.

[0107] (2) When the first distance is greater than the front box position threshold, and the difference between the first distance and the second distance is greater than the large box length threshold, the location of the container is that there are front and rear boxes or there is a large box. Since the standard 40-foot large box is 12.192 meters long and the standard 45-foot large box is 13.716 meters long, the large box length threshold can be a value between 9 meters and 11 meters, such as 9 meters, 10 meters, 11 meters, etc., and this application does not impose specific restrictions on this.

[0108] (3) When the first distance is greater than or equal to the rear box position threshold, the position of the container is that there is a rear box. The rear box position threshold and the front box position threshold may have the following relationship: rear box position threshold = front box position threshold + 6.058.

[0109] (4) When the first distance is greater than the front box position threshold and the first distance is less than the rear box position threshold, the position of the container is that there is a middle box.

[0110] In this step, the type and position of each container can be determined through the first plane and the second plane. For example, it can be determined that a medium box is loaded in the trailer, and the medium box is 2 meters away from the origin of the trailer coordinate system, or that two small boxes are loaded in the trailer, the first plane of the front box is 1.5 meters away from the origin of the trailer coordinate system, and the second plane of the rear box is 6 meters away from the origin of the trailer coordinate system, and so on.

[0111] The method for determining the packing state provided in the embodiment of the present application, after obtaining the first point cloud data collected by the first radar installed on the top of the tractor of the container truck and the second point cloud data collected by the second radar installed at the rear of the trailer of the container truck, the target point cloud data is determined from the first point cloud data and the second point cloud data according to the position of the first radar, the position of the second radar, and the structural parameters of the trailer. When the number of point clouds of the target point cloud data is greater than the preset number of point clouds, the first plane of the container close to the tractor and the second plane of the container close to the rear of the trailer are obtained according to the target point cloud data, so as to determine the packing state according to the first plane and the second plane. In this technical solution, by determining the target point cloud data of the area in the trailer for placing the container from the first point cloud data and the second point cloud data, so that the packing state is determined according to the target point cloud data, the accuracy of detecting the packing state is effectively improved. In this method, it is only necessary to install the first radar on the top of the tractor and the second radar at the rear of the trailer, and there is no need to raise the radar, which avoids the problem of the bracket being too high and easy to be hit or the problem of radar shaking caused by instability, and ensures the safety and stability of the detection process.

[0112] Optionally, based on any of the above embodiments, Figure 3 This is a flow chart of Embodiment 2 of the method for determining the packing status provided in the embodiment of the present application. Figure 3 As shown, S202 can be implemented by the following steps:

[0113] S301, converting the first point cloud data from the coordinate system where the first radar is located to the coordinate system in front of the vehicle, and converting the second point cloud data from the coordinate system where the second radar is located to the coordinate system in front of the vehicle, to obtain initial point cloud data.

[0114] Among them, the coordinate system in front of the vehicle is the coordinate system where the container truck is located.

[0115] The coordinate systems of radar and truck are different. The coordinate system of radar is determined by the position of radar. After the radar is installed, the coordinate system of radar is uniquely determined. The front coordinate system of the vehicle is usually a right-hand coordinate system defined according to the position of the truck. Its origin can be the point where the front center of the front of the vehicle is located. The X-axis is parallel to the ground and the positive direction extends forward along the front of the truck. The Y-axis is parallel to the ground and the positive direction extends perpendicular to the X-axis and outward. The Z-axis is perpendicular to the ground and the positive direction extends away from the ground.

[0116] S302: Determine the ROI of the container according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer.

[0117] Since the radar has a wide acquisition range, the collected point cloud includes point cloud data of the area in the trailer used to place the container, as well as point cloud data of other areas of the trailer. Therefore, the ROI of the container can be obtained based on the position of the first radar, the position of the second radar, and the structural parameters of the trailer. The ROI is the area in the trailer used to place the container, and the ROI can better reflect the spatial range of the container.

[0118] It should be understood that the following principles need to be met when determining RoI:

[0119] (1) Ensure that the RoI covers the possible locations of the container;

[0120] (2) Minimize the amount of debris that may be included on non-container objects.

[0121] The coordinate range of the ROI is the maximum and minimum values ​​of the ROI on the X, Y, and Z axes of the vehicle front coordinate system.

[0122] Exemplarily, the maximum value (max_x) of the X-axis can be the position closest to the tractor in the area of ​​the trailer for placing containers, the minimum value (min_x) of the X-axis and max_x can have the following relationship: min_x=max_x-trailer length, the maximum value (max_z) on the Z-axis can be the trailer height plus the preset height, and the minimum value (min_z) on the Z-axis can be the trailer height.

[0123] Among them, the position closest to the tractor in the area of ​​the trailer used to place the container, the trailer length, and the trailer height can all be obtained through real-time detection, container truck structure drawings or measurement calculations.

[0124] Exemplarily, the preset height can be determined according to the height of the container, such as 2 meters. It should be understood that the preset height can also be other heights, and the embodiments of the present application do not impose specific limitations on this.

[0125] S303: Determine the initial point cloud data in the ROI as the target point cloud data.

[0126] In the embodiment of the present application, the initial point cloud data is obtained by converting the first point cloud data from the coordinate system of the first radar to the coordinate system in front of the vehicle, and converting the second point cloud data from the coordinate system of the second radar to the coordinate system in front of the vehicle, and the ROI of the container is determined according to the posture of the first radar, the posture of the second radar and the structural parameters of the trailer, so that the initial point cloud data in the ROI is determined as the target point cloud data. In this technical solution, the redundant point cloud data in the first point cloud data and the second point cloud data are removed to obtain the target point cloud data of the ROI, which effectively improves the efficiency and accuracy of subsequent processing.

[0127] Optionally, based on any of the above embodiments, Figure 4 This is a flow chart of Embodiment 3 of the method for determining the packing status provided in the embodiment of the present application. Figure 4 As shown, in S203, extracting the first plane and the second plane from the third point cloud data using the RANSAC method can be achieved by the following steps:

[0128] S401 . Extract a first initial plane from the third point cloud data using a RANSAC method.

[0129] Among them, the inlier threshold when using the RANSAC method can be 0.2 meters.

[0130] S402: Delete points in the third point cloud data whose distance from the first initial plane is less than a preset distance, and obtain fourth point cloud data.

[0131] Among them, the preset distance can be 0.5 meters, 0.6 meters, 0.7 meters, etc., which can be limited according to actual conditions. The embodiment of the present application does not impose specific restrictions on this.

[0132] Redundant point cloud data in the third point cloud data is deleted by presetting the distance, thereby improving the efficiency and accuracy of subsequent processing.

[0133] S403 : Extract a second initial plane from the fourth point cloud data using the RANSAC method.

[0134] Among them, the inlier threshold when using the RANSAC method can be 0.2 meters.

[0135] S404: Calculate a first intersection point between the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point between the second initial plane and the X-axis of the vehicle front coordinate system.

[0136] S405 , determining the initial plane corresponding to the intersection point with the smallest absolute value of coordinates as the first plane, and determining the initial plane corresponding to the intersection point with the largest absolute value of coordinates as the second plane.

[0137] In the above embodiment, the first plane and the second plane are extracted from the third point cloud data by the RANSAC method, which improves the accuracy of extracting the first plane and the second plane, thereby further improving the accuracy of the packing state determined according to the first plane and the second plane.

[0138] Optionally, in some embodiments, after determining the target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, the method for determining the packing state may further include the following steps:

[0139] When the point cloud quantity of the target point cloud data is less than or equal to the preset point cloud quantity, it is determined that the container is not loaded on the trailer.

[0140] That is to say, when the number of point clouds in the target point cloud data is less than or equal to the preset number of point clouds, it means that the target point cloud data at this time only includes the point cloud data of the trailer part, and does not include the point cloud data of the container.

[0141] In this technical solution, whether a container is loaded in the trailer is judged according to the number of point clouds of the target point cloud data, thereby improving the accuracy of determining the position of the container.

[0142] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0143] Figure 5 This is a schematic diagram of the structure of the device for determining the packing status provided in the embodiment of the present application. Figure 5 As shown, the device for determining the packing state includes:

[0144] An acquisition module 51 is used to acquire first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck;

[0145] The processing module 52 is used to determine target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, where the target point cloud data is the point cloud data of the area in the trailer for placing the container;

[0146] The processing module 52 is further used to obtain a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer according to the target point cloud data when the number of point clouds of the target point cloud data is greater than the preset number of point clouds;

[0147] The processing module 52 is further configured to determine a packing status according to the first plane and the second plane, wherein the packing status includes at least one of a loading position of a container, a number of containers, and a type of container.

[0148] In a possible design of the embodiment of the present application, the processing module 52 is specifically configured to:

[0149] The first point cloud data is converted from the coordinate system of the first radar to the coordinate system in front of the vehicle, and the second point cloud data is converted from the coordinate system of the second radar to the coordinate system in front of the vehicle to obtain initial point cloud data, where the coordinate system in front of the vehicle is the coordinate system of the container truck;

[0150] According to the position and posture of the first radar, the position and posture of the second radar and the structural parameters of the trailer, the region of interest ROI of the container is determined;

[0151] The initial point cloud data in the ROI is determined as the target point cloud data.

[0152] Optionally, the processing module 52 is specifically configured to:

[0153] When the number of point clouds in the target point cloud data is greater than the preset number of point clouds, points in the target point cloud data whose angles between the normal vector and the X-axis of the front coordinate system of the vehicle are greater than or equal to the preset angle are deleted to obtain third point cloud data;

[0154] The first plane and the second plane are extracted from the third point cloud data using the RANSAC method.

[0155] Optionally, the processing module 52 is specifically configured to:

[0156] Extracting a first initial plane from the third point cloud data using the RANSAC method;

[0157] Deleting points in the third point cloud data whose distance from the first initial plane is less than a preset distance, to obtain fourth point cloud data;

[0158] A second initial plane is extracted from the fourth point cloud data using the RANSAC method;

[0159] Calculate a first intersection point between the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point between the second initial plane and the X-axis of the vehicle front coordinate system;

[0160] The initial plane corresponding to the intersection point with the smallest absolute value of coordinates is determined as the first plane, and the initial plane corresponding to the intersection point with the largest absolute value of coordinates is determined as the second plane.

[0161] In another possible design of the embodiment of the present application, the processing module 52 is specifically configured to:

[0162] According to the structural parameters of the trailer, the first plane and the second plane are converted into the trailer coordinate system to obtain the first plane after the coordinate conversion and the second plane after the coordinate conversion;

[0163] The packing state is determined according to the first plane after the coordinates are transformed and the second plane after the coordinates are transformed.

[0164] In another possible design of the embodiment of the present application, after determining the target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, the processing module 52 is further used to:

[0165] When the point cloud quantity of the target point cloud data is less than or equal to the preset point cloud quantity, it is determined that the container is not loaded on the trailer.

[0166] The device for determining the packing status provided in the embodiment of the present application can be used to execute the method for determining the packing status in any of the above embodiments. The implementation principle and technical effect are similar and will not be described in detail here.

[0167] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0168] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 15 may include: a processor 61, a memory 62, and computer program instructions stored in the memory 62 and executable on the processor 61. When the processor 61 executes the computer program instructions, the method for determining the packing status provided in any of the aforementioned embodiments is implemented.

[0169] Optionally, the above-mentioned components of the electronic device 15 may be connected via a system bus.

[0170] The memory 62 may be a separate storage unit or a storage unit integrated in the processor. The number of processors may be one or more.

[0171] Optionally, the electronic device 15 may also include an interface for interacting with other devices.

[0172] It should be understood that the processor 61 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0173] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0174] All or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape, floppy disk, optical disc and any combination thereof.

[0175] The electronic device provided in the embodiment of the present application can be used to execute the method for determining the packing status provided in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0176] An embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the above-mentioned method for determining the packing status.

[0177] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0178] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0179] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the above-mentioned method for determining the packing status can be implemented.

[0180] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining a packing state, characterized in that: include: Acquire first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck; According to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, target point cloud data is determined from the first point cloud data and the second point cloud data, wherein the target point cloud data is point cloud data of an area in the trailer for placing a container; When the number of point clouds of the target point cloud data is greater than the preset number of point clouds, obtaining a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer according to the target point cloud data; Determine a container loading status according to the first plane and the second plane, wherein the container loading status includes at least one of a loading position of a container, a number of containers, and a container type; The front coordinate system is a right-hand coordinate system defined according to the position of the container truck, the origin of which is the point at the center of the front end of the truck head, the X-axis is parallel to the ground and the positive direction extends forward along the front of the truck head, the Y-axis is parallel to the ground and the positive direction extends outward perpendicular to the X-axis, and the Z-axis is perpendicular to the ground and the positive direction extends away from the ground; when the number of point clouds in the target point cloud data is greater than the preset number of point clouds, the first plane of the container close to the tractor and the second plane of the container close to the rear of the trailer are obtained according to the target point cloud data, including: When the number of point clouds in the target point cloud data is greater than the preset number of point clouds, points in the target point cloud data whose angles between the normal vector and the X-axis of the front vehicle coordinate system are greater than or equal to the preset angle are deleted to obtain third point cloud data; Extracting a first initial plane from the third point cloud data using a random sampling consistency RANSAC method; Deleting points in the third point cloud data whose distance from the first initial plane is less than a preset distance, to obtain fourth point cloud data; Extracting a second initial plane from the fourth point cloud data using a RANSAC method; Calculating a first intersection point of the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point of the second initial plane and the X-axis of the vehicle front coordinate system; The initial plane corresponding to the intersection point with the smallest absolute value of coordinates is determined as the first plane, and the initial plane corresponding to the intersection point with the largest absolute value of coordinates is determined as the second plane.

2. The method according to claim 1, characterized in that The determining target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer includes: The first point cloud data is converted from the coordinate system where the first radar is located to the coordinate system in front of the vehicle, and the second point cloud data is converted from the coordinate system where the second radar is located to the coordinate system in front of the vehicle, to obtain initial point cloud data, and the coordinate system in front of the vehicle is the coordinate system where the container truck is located; Determine a region of interest (ROI) of the container according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer; The initial point cloud data in the ROI is determined as the target point cloud data.

3. The method according to claim 1, characterized in that The determining the position of the container according to the first plane and the second plane includes: According to the structural parameters of the trailer, the first plane and the second plane are converted into a trailer coordinate system to obtain the first plane after the coordinate conversion and the second plane after the coordinate conversion; The packing state is determined according to the first plane after the coordinates are transformed and the second plane after the coordinates are transformed.

4. The method according to any one of claims 1 to 3, characterized in that: After determining the target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, the method further includes: When the point cloud quantity of the target point cloud data is less than or equal to the preset point cloud quantity, it is determined that the container is not loaded on the trailer.

5. A device for determining a packing state, characterized in that: include: An acquisition module, used to acquire first point cloud data collected by a first radar installed on the top of the tractor of the container truck, and second point cloud data collected by a second radar installed on the rear of the trailer of the container truck; a processing module, configured to determine target point cloud data from the first point cloud data and the second point cloud data according to the position and posture of the first radar, the position and posture of the second radar, and the structural parameters of the trailer, wherein the target point cloud data is point cloud data of an area in the trailer for placing a container; The processing module is further configured to obtain, according to the target point cloud data, a first plane of the container close to the tractor and a second plane of the container close to the rear of the trailer when the number of point clouds of the target point cloud data is greater than a preset number of point clouds; The processing module is further used to determine a packing status according to the first plane and the second plane, wherein the packing status includes at least one of a loading position of a container, a number of containers, and a type of container; The vehicle front coordinate system is a right-hand coordinate system defined according to the position of the container truck, whose origin is the point where the front end center of the vehicle head is located, the X-axis is parallel to the ground and the positive direction extends forward along the front of the container truck head, the Y-axis is parallel to the ground and the positive direction extends outward perpendicular to the X-axis, and the Z-axis is perpendicular to the ground and the positive direction extends away from the ground; the processing module is specifically used to delete the points in the target point cloud data whose angles between the normal vector and the X-axis of the vehicle front coordinate system are greater than or equal to the preset angle when the number of point clouds in the target point cloud data is greater than the preset number of point clouds, to obtain the third point cloud data; use the random sampling consistency RANSAC method to extract the first initial plane from the third point cloud data; delete the points in the third point cloud data whose distance from the first initial plane is less than the preset distance to obtain the fourth point cloud data; use the RANSAC method to extract the second initial plane from the fourth point cloud data; Calculating a first intersection point of the first initial plane and the X-axis of the vehicle front coordinate system, and a second intersection point of the second initial plane and the X-axis of the vehicle front coordinate system; The initial plane corresponding to the intersection point with the smallest absolute value of coordinates is determined as the first plane, and the initial plane corresponding to the intersection point with the largest absolute value of coordinates is determined as the second plane.

6. An electronic device comprising: A processor, a memory, and computer program instructions stored in the memory and executable on the processor, wherein the processor is used to implement the method for determining the packing status as described in any one of claims 1 to 4 when executing the computer program instructions.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method for determining the packing status according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it is used to implement the method for determining the packing status according to any one of claims 1 to 4.

Images