Vehicle loading and unloading task monitoring method, device and system

Abstract

The present invention provides a vehicle loading and unloading task monitoring method, device, and system, relating to the field of laser detection technology. The method comprises: acquiring point cloud data of the vehicle loading and unloading area, and determining whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data; when the vehicle to be detected has reached the loading and unloading position, acquiring point cloud data of the vehicle to be detected; extracting point cloud data of the vehicle compartment area from the point cloud data of the vehicle to be detected, and calculating the depth of cargo loaded in the vehicle compartment of the vehicle to be detected based on the coordinates of the point cloud data of the vehicle compartment area; and calculating the vehicle compartment loading and unloading rate of the vehicle to be detected based on the vehicle compartment length, cargo depth, and the empty loading and unloading rate of the vehicle to be detected. The present invention can monitor the loading and unloading rate of the vehicle compartment of the vehicle to be detected, reduce the error in the loading and unloading rate detection results, and improve the accuracy of vehicle loading and unloading rate detection.

Description

Technical Field

[0001] The present invention relates to the field of laser detection technology, and in particular to a vehicle loading and unloading task monitoring method, device and system. Background Art

[0002] Currently, transport vehicles (e.g., vans) typically position their rear end close to the loading dock during loading and unloading operations to facilitate cargo loading and unloading. To improve loading and unloading efficiency and avoid congestion within logistics parks, monitoring the loading and unloading tasks of these vehicles is crucial for allocating and deploying loading and unloading locations. Existing vehicle loading and unloading task monitoring technologies primarily rely on camera-captured images and deep learning algorithms to identify vehicle loading and unloading rates. These technologies are susceptible to environmental factors such as lighting, resulting in significant errors in loading and unloading rate detection and reducing accuracy. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a vehicle loading and unloading task monitoring method, device and system, which can monitor the loading and unloading rate of the vehicle to be inspected, reduce the error of the loading and unloading rate detection result, and improve the accuracy of the vehicle loading and unloading rate detection.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring vehicle loading and unloading tasks, comprising: acquiring point cloud data of a vehicle loading and unloading area, and determining whether a vehicle to be detected has arrived at a loading and unloading position based on the point cloud data; acquiring point cloud data of the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position; extracting point cloud data of a carriage area from the point cloud data of the vehicle to be detected, and calculating a depth of cargo loaded in the carriage of the vehicle to be detected based on the coordinates of the point cloud data of the carriage area; and calculating a carriage loading and unloading rate of the vehicle to be detected based on the carriage length of the vehicle to be detected, the cargo depth, and the empty loading and unloading rate of the vehicle to be detected.

[0006] Furthermore, an embodiment of the present invention provides a first possible implementation of the first aspect, wherein the step of calculating the compartment loading and unloading rate of the vehicle to be detected based on the compartment length of the vehicle to be detected, the cargo depth and the empty loading and unloading rate of the vehicle to be detected includes: calculating the ratio of the cargo depth to the compartment length, and calculating the difference between the ratio and the empty loading and unloading rate to obtain the compartment loading and unloading rate of the vehicle to be detected.

[0007] Furthermore, an embodiment of the present invention provides a second possible implementation of the first aspect, wherein the car frame of the vehicle to be detected is provided with an identification tape, the echo intensity of the identification tape is different from the echo intensity of the car frame, and the step of extracting the car frame area from the point cloud data of the vehicle to be detected includes: obtaining the echo intensity of each point cloud data of the vehicle to be detected; filtering out the point cloud data of the car frame based on the echo intensity of each point cloud data of the vehicle to be detected; and determining the car frame area of ​​the vehicle to be detected based on the area enclosed by the point cloud data of the car frame.

[0008] Furthermore, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein the point cloud data of the vehicle to be detected is obtained based on laser radar collection, and the step of calculating the depth of the cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area includes: obtaining the point cloud coordinates of the point cloud data corresponding to the border vertices of the compartment area, and determining the radial distance between the tail end of the compartment of the vehicle to be detected and the laser radar based on the point cloud coordinates of the border vertices of the compartment area; calculating the average distance between each point cloud data in the compartment area and the laser radar; calculating the difference between the average distance and the radial distance to obtain the free length of the compartment of the vehicle to be detected; obtaining the compartment length of the vehicle to be detected, calculating the difference between the compartment length and the free length of the compartment, and obtaining the depth of the cargo loaded in the compartment of the vehicle to be detected.

[0009] Furthermore, an embodiment of the present invention provides a fourth possible implementation method of the first aspect, wherein the vehicle loading and unloading task monitoring method also includes: judging whether there is a vehicle to be detected in the vehicle loading and unloading area based on the point cloud data of the vehicle loading and unloading area or the target image of the vehicle loading and unloading area; when there is a vehicle to be detected in the vehicle loading and unloading area, obtaining the license plate number of the vehicle to be detected, and determining the vehicle model and compartment size of the vehicle to be detected based on the license plate number; wherein the compartment size includes the compartment length.

[0010] Furthermore, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein the step of determining whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data includes: obtaining the target minimum bounding box formed by the vehicle to be detected, and determining the yaw angle of the vehicle to be detected and the target distance between the rear end of the vehicle to be detected and the loading and unloading platform based on the point cloud coordinates of the target minimum bounding box; when the yaw angle is within a preset yaw range and the target distance is within a preset distance range, it is determined that the vehicle to be detected has reached the loading and unloading position.

[0011] Furthermore, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein at least two parallel reference lines are set in the vehicle loading and unloading area, and the method also includes: determining the working status of the vehicle to be detected based on the intersection order of the target minimum bounding box and each of the reference lines; wherein the working status includes an entry status, a loading / unloading status and an exit status; and determining the loading and unloading time of the vehicle to be detected based on the corresponding time of the entry status and the exit status.

[0012] Furthermore, an embodiment of the present invention provides a seventh possible implementation of the first aspect, wherein the laser radar is arranged on a lifting rod, and the method also includes: when it is determined that the vehicle to be detected has arrived at the loading and unloading position, controlling the laser radar to descend to a preset height so that the laser radar can collect the entire image of the door area; when the carriage loading and unloading rate of the vehicle to be detected is calculated, and it is detected that the door in the door area is closed, controlling the laser radar to rise to the preset height to collect point cloud data of the vehicle loading and unloading area.

[0013] In the second aspect, an embodiment of the present invention also provides a vehicle loading and unloading task monitoring device, including: a judgment module, used to obtain point cloud data of the vehicle loading and unloading area, and judge whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data; an acquisition module, used to obtain the point cloud data of the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position; a first calculation module, used to extract point cloud data of the compartment area from the point cloud data of the vehicle to be detected, and calculate the depth of cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area; a second calculation module, used to calculate the compartment loading and unloading rate of the vehicle to be detected based on the compartment length of the vehicle to be detected, the cargo depth and the empty loading and unloading rate of the vehicle to be detected.

[0014] In a third aspect, an embodiment of the present invention provides a vehicle loading and unloading task monitoring system, comprising: a laser radar, an image acquisition device and a controller, the controller comprising a processor and a storage device; a computer program is stored on the storage device, and the computer program, when run by the processor, executes the method described in any one of the first aspects.

[0015] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the above-mentioned first aspects are executed.

[0016] Embodiments of the present invention provide a vehicle loading and unloading task monitoring method, device, and system. The method comprises: acquiring point cloud data of a vehicle loading and unloading area, and determining whether a vehicle to be detected has arrived at a loading and unloading location based on the point cloud data; acquiring point cloud data of the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading location; extracting point cloud data of a vehicle compartment area from the point cloud data of the vehicle to be detected, and calculating the cargo depth loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area; and calculating the compartment loading and unloading rate of the vehicle to be detected based on the compartment length, cargo depth, and empty loading and unloading rate of the vehicle to be detected. The present invention monitors the cargo depth loaded in the compartment by calculating the cargo depth loaded in the compartment based on the point cloud data of the vehicle compartment area when the vehicle to be detected arrives at the loading and unloading location. By calculating the compartment loading and unloading rate based on the compartment length, cargo depth, and empty loading and unloading rate of the vehicle to be detected, the present invention monitors the compartment loading and unloading rate of the vehicle to be detected without being affected by environmental factors such as light, thereby reducing errors in loading and unloading rate detection results and improving the accuracy of vehicle loading and unloading rate detection.

[0017] Other features and advantages of the embodiments of the present invention will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned technologies of the embodiments of the present invention.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flow chart of a vehicle loading and unloading task monitoring method provided by an embodiment of the present invention is shown;

[0021] Figure 2 A schematic diagram of a vehicle loading and unloading area provided by an embodiment of the present invention is shown;

[0022] Figure 3 A schematic diagram of the rear end of a carriage provided by an embodiment of the present invention is shown;

[0023] Figure 4 A schematic diagram of a minimum bounding box of a vehicle to be detected on an XY plane is shown in an embodiment of the present invention;

[0024] Figure 5 A schematic structural diagram of a vehicle loading and unloading task monitoring device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0026] At present, the existing loading and unloading rate detection technologies include handheld sensors to detect the loading and unloading rate of the carriage and sensors installed inside the carriage to detect the loading and unloading rate of the carriage. However, the handheld sensors to detect the loading and unloading rate of the carriage are easily subject to human interference, and it is difficult to estimate the loading and unloading rate when the staff operates improperly. The sensors inside the carriage to detect the loading and unloading rate of the carriage are easily affected by the poor light inside the carriage, resulting in large errors in the loading and unloading rate detection. In addition, both the handheld sensors to detect the loading and unloading rate of the carriage and the sensors installed inside the carriage require one-to-one vehicle loading and unloading rate detection, which consumes manpower and material resources and has high detection costs.

[0027] To address the above issues, embodiments of the present invention provide a method, device, and system for monitoring vehicle loading and unloading tasks. This technology can be used to reduce detection costs and improve the accuracy of vehicle loading and unloading rate detection. The embodiments of the present invention are described in detail below.

[0028] This embodiment provides a method for monitoring vehicle loading and unloading tasks. The method can be applied to electronic devices such as computers, which are connected to a point cloud data acquisition device for communication. Figure 1 The vehicle loading and unloading task monitoring method shown in the flowchart mainly includes the following steps:

[0029] Step S102: acquiring point cloud data of the vehicle loading and unloading area, and determining whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data.

[0030] The vehicle to be inspected may be a van, and the vehicle loading and unloading area includes a loading and unloading platform (also referred to as a dock) and a vehicle parking area, with the vehicle parking area adjacent to the loading and unloading platform. The point cloud data acquisition device may be a laser radar, which collects point cloud data of the vehicle loading and unloading area based on the laser radar, establishes a point cloud coordinate system, identifies the location of the vehicle to be inspected based on the point cloud data of the vehicle loading and unloading area, and determines whether the vehicle to be inspected has reached the loading and unloading location based on the positional relationship between the vehicle to be inspected and the loading and unloading platform.

[0031] Since the height (Z-axis coordinate) values ​​of the ground point cloud data and the vehicle point cloud data are different, the ground point cloud data can be filtered out according to the Z-axis coordinate of the point cloud data, and the point cloud data above the ground can be divided into multiple connected areas. Based on the geometric characteristics of each connected area, the connected area formed by the vehicle is screened out, and then the location of the vehicle is determined according to the point cloud coordinates of the edge of the connected area formed by the vehicle.

[0032] In a specific embodiment, see Figure 2 The vehicle loading and unloading area diagram shown in FIG. 1 shows the location of the laser radar 20. Figure 2 As shown, the laser radar 20 is installed above the loading and unloading platform 21 so that the point cloud data of the vehicle loading and unloading area can be fully collected. According to the installation angle of the laser radar or based on the checkerboard calibration method, the point cloud data of the vehicle loading and unloading area is subjected to posture correction, and the Rodriguez rotation matrix of the ground normal vector to (0,0,1) is calculated to convert the point cloud data from the laser radar coordinate system to the ground point cloud coordinate system XYZO. There are useless areas in the scene collected within the laser radar field of view. Therefore, after the laser radar equipment is installed, the detection range can be manually selected, and the point cloud image of the vehicle loading and unloading area can be extracted by direct filtering based on the point cloud coordinates.

[0033] Step S104: When the vehicle to be detected arrives at the loading and unloading location, the point cloud data of the vehicle to be detected is obtained.

[0034] When the rear end of the vehicle to be detected is aligned with the loading and unloading platform, and the distance between the rear end of the vehicle and the loading and unloading platform is less than the preset loading and unloading distance, it is determined that the vehicle to be detected has arrived at the loading and unloading position, and the point cloud data of the vehicle to be detected is obtained from the point cloud data of the above-mentioned vehicle loading and unloading area.

[0035] In one embodiment, the point cloud data of the vehicle loading and unloading area can be segmented for obstacles to determine the position of the minimum bounding box of the vehicle to be detected, and the point cloud data within the minimum bounding box of the vehicle to be detected is determined as the point cloud data of the vehicle to be detected.

[0036] Step S106 , extracting point cloud data of the vehicle compartment area from the point cloud data of the vehicle to be detected, and calculating the depth of the cargo loaded in the vehicle compartment of the vehicle to be detected based on the coordinates of the point cloud data of the vehicle compartment area.

[0037] The cabin of the vehicle to be detected is a regular rectangular parallelepiped, and a point cloud image is constructed based on the point cloud coordinates of the vehicle to be detected, that is, the point cloud coordinates of the vehicle to be detected are converted into a spherical coordinate system according to the pitch angle and azimuth angle of the laser radar emission line, and then the spherical coordinate system is further converted into an image coordinate system. The vehicle image in the point cloud image is segmented to obtain the front area and the cabin area. The cabin area is extracted from the point cloud image according to the geometric features of the cabin area. Since the laser radar is located in the rear direction of the vehicle when the vehicle arrives at the loading and unloading area to prepare for loading and unloading goods, the above-mentioned extracted cabin area is the rear area of ​​the cabin with the cabin door. The pixel points of the cabin area are converted from the image coordinate system to the point cloud coordinate system, and then the point cloud coordinates of the point cloud data corresponding to the cabin area can be obtained.

[0038] In one embodiment, referring to the schematic diagram of the rear of the vehicle as shown in Figure 3, an identification tape 31 is set around the edges of the door at the rear of the vehicle to be detected. The echo intensity of the identification tape 31 is different from the echo intensity of the area without the identification tape. The point cloud data of the four corner points of the rear of the vehicle can be determined according to the echo intensity of each point cloud data of the vehicle to be detected, and the area surrounded by the four corner points is used as the vehicle area.

[0039] When users can load and unload cargo, the coordinates of the point cloud data collected in the carriage area will change as the amount of cargo in the carriage changes. The volume length of the cargo loaded in the carriage can be determined based on the coordinates of the point cloud data in the carriage area and the coordinates of the point cloud data of the four corner points, which is recorded as the cargo depth. The difference between the carriage length and the cargo depth is calculated to obtain the remaining length of cargo that can be loaded in the carriage.

[0040] Step S108 , calculating the compartment loading and unloading rate of the vehicle to be detected based on the compartment length, cargo depth and empty loading and unloading rate of the vehicle to be detected.

[0041] The image of the vehicle to be detected is collected based on the image sensor, and the license plate number of the vehicle to be detected is identified based on the character recognition algorithm. According to the correspondence table between the license plate number and vehicle information recorded in the database, the vehicle model, empty loading and unloading rate and car body size of the vehicle to be detected are determined, thereby obtaining the car body length and empty loading and unloading rate of the vehicle to be detected.

[0042] Calculate the ratio of cargo depth to car length. The difference between this ratio and the empty loading / unloading ratio yields the car loading / unloading ratio for the vehicle being tested: Car loading / unloading ratio = (cargo depth / car length) - empty loading / unloading ratio. The empty loading / unloading ratio represents the offset from the tare operation for the vehicle being tested. The above-mentioned car loading / unloading ratio represents the percentage of cargo currently loaded in the car.

[0043] The vehicle loading and unloading task monitoring method provided in this embodiment realizes the monitoring of the depth of cargo loaded in the compartment by calculating the cargo depth loaded in the compartment based on the point cloud data of the compartment area of ​​the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position. The compartment loading and unloading rate is calculated based on the compartment length, cargo depth and the empty loading and unloading rate of the vehicle to be detected, thereby realizing the monitoring of the compartment loading and unloading rate of the vehicle to be detected, and will not be affected by environmental factors such as light, thereby reducing the error of the loading and unloading rate detection result and improving the accuracy of the vehicle loading and unloading rate detection.

[0044] In one embodiment, in order to facilitate the extraction of point cloud data of the compartment area from the point cloud data of the vehicle to be detected, the above-mentioned laser radar can be set on a lifting rod. The vehicle loading and unloading task monitoring method provided by this embodiment also includes: when it is determined that the vehicle to be detected has arrived at the loading and unloading position, the laser radar is controlled to descend to a preset height so that the laser radar can collect the entire picture of the door area; when the compartment loading and unloading rate of the vehicle to be detected is calculated, and it is detected that the door in the door area is closed, the laser radar is controlled to rise to a preset height to collect point cloud data of the vehicle loading and unloading area.

[0045] By setting the laser radar on the lifting rod, after the vehicle to be inspected reaches the loading and unloading position, the laser radar can be lowered to a position directly aligned with the door area in the cabin area, so that the collected point cloud data of the vehicle to be inspected only includes the point cloud data of the cabin area but not the point cloud data of the front area. There is no need to segment the front area and the cabin area to obtain the point cloud data of the cabin area, which improves the convenience of obtaining the point cloud data of the cabin area.

[0046] When the vehicle to be inspected completes cargo loading / unloading and calculates the loading and unloading rate of the vehicle to be inspected, if the door in the door area is detected to be closed, it is determined that the loading / unloading state of the vehicle to be inspected is completed and the vehicle to be inspected is about to enter the exit state. The laser radar is controlled to rise to a preset height to restore the original height, so that the point cloud data of the vehicle loading and unloading area can be fully collected. In a specific embodiment, whether the door is closed can be determined based on the coordinates of the point cloud data in the door area. Since the point cloud data in the door area when the door is not closed is the point cloud data of the cargo when the door is not closed, and the point cloud data in the door area after the door is closed is the door point cloud data, the distance between the cargo point cloud data and the door point cloud data and the laser radar is different (that is, the X-axis coordinate values ​​are different), and the surface of the door area is relatively smooth (the X-axis coordinate values ​​of the point cloud data are the same) and has a door handle, therefore, according to the coordinates of the point cloud data in the door area, whether the door is closed can be determined.

[0047] In one embodiment, the preset height of the laser radar can be determined according to the height of the vehicle to be detected and the installation height of the lifting rod, so that the laser radar can fully collect the following information after it is lowered: Figure 3The point cloud data of the door area of ​​the carriage is shown, and the laser radar can fully collect the point cloud data of the vehicle loading and unloading area after rising to the preset height and returning to the original height.

[0048] In one embodiment, the cabin frame of the vehicle to be detected is provided with an identification tape, and the echo intensity of the identification tape is different from the echo intensity of the cabin. This embodiment provides a specific implementation method for extracting the cabin area from the point cloud data of the vehicle to be detected: obtaining the echo intensity of each point cloud data of the vehicle to be detected; filtering out the point cloud data of the cabin frame based on the echo intensity of each point cloud data of the vehicle to be detected; and determining the cabin area of ​​the vehicle to be detected based on the area enclosed by the point cloud data of the cabin frame.

[0049] Taking into account that the echo intensity of the laser radar is similar to the grayscale value of the image, and the echo intensity of the identification band is obviously different from the echo intensity of other areas, the echo intensity of the point cloud data of the carriage area is binarized, and the point cloud data of the identification band is screened out. The point cloud data of the identification band is the point cloud data of the carriage border. Convex hull contour detection or minimum bounding box fitting is performed on the point cloud data of the carriage border to obtain the carriage area surrounded by the four corner points of the rear of the carriage. The carriage area is the rear area of ​​the carriage with a door, so that when the user loads and unloads goods, the coordinates of the point cloud data in the carriage area can be monitored to monitor the loading and unloading rate of the carriage.

[0050] In one embodiment, the point cloud data of the vehicle to be detected is acquired based on laser radar. This embodiment provides an implementation method for calculating the depth of cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area. Specifically, the implementation can refer to the following steps (1) to (4):

[0051] Step (1): Obtain the point cloud coordinates of the point cloud data corresponding to the border vertices of the compartment area, and determine the radial distance between the rear end of the compartment of the vehicle to be detected and the laser radar based on the point cloud coordinates of the border vertices of the compartment area.

[0052] Get the point cloud coordinates of the point cloud data on the four border vertices of the compartment area. Since the point cloud coordinate system where the point cloud data is located is as follows Figure 2 As shown in the figure, the X-axis coordinate value of the point cloud data is the distance between the point cloud and the laser radar. The average value of the X-axis coordinate values ​​of the point cloud data on the four border vertices is calculated as the radial distance x1 from the rear of the car to the laser radar.

[0053] Step (2): Calculate the average distance between each point cloud data and the lidar in the vehicle compartment area.

[0054] Obtain the coordinates of each point cloud data in the carriage area, calculate the average value of the X-axis coordinate value of each point cloud data in the carriage area, record it as the average distance, and use the average value of the X-axis coordinate value of each point cloud data in the carriage area as the distance between the tail of the cargo loaded in the carriage and the lidar.

[0055] Step (3): Calculate the difference between the average distance and the radial distance to obtain the free length of the vehicle compartment to be detected.

[0056] Subtract the radial distance x1 between the rear of the carriage and the laser radar from the above average distance to obtain the distance between the rear of the cargo in the carriage and the rear of the carriage, as follows: Figure 2 As shown, the length of the carriage that has not yet been loaded with goods is recorded as the free length of the carriage x2.

[0057] Step (4): Obtain the length of the vehicle compartment to be tested, calculate the difference between the length of the vehicle compartment and the free length of the vehicle compartment, and obtain the depth of the cargo loaded in the vehicle compartment to be tested.

[0058] Obtain the length w of the vehicle to be detected, calculate the length w - the empty length x2 of the vehicle to be detected, and obtain the depth of cargo loaded in the vehicle to be detected.

[0059] In one embodiment, before step (4) or step S108, the method provided in this embodiment further includes an implementation method for obtaining the length of the vehicle compartment of the vehicle to be detected, which can be specifically performed with reference to the following steps 1) to 2):

[0060] Step 1): Based on the point cloud data of the vehicle loading and unloading area or the target image of the vehicle loading and unloading area, determine whether there is a vehicle to be detected in the vehicle loading and unloading area.

[0061] In one embodiment, the presence of a vehicle to be detected is determined based on point cloud data of a vehicle loading and unloading area: ground detection is performed based on a ground detection algorithm and the Z-axis coordinates of the point cloud data of the vehicle loading and unloading area, the ground point cloud data is filtered out, and obstacle segmentation is performed on the point cloud data of the vehicle loading and unloading area after filtering out the ground point cloud using a clustering algorithm. The three-dimensional minimum bounding box of the obstacle is fitted based on a convex hull contour detection algorithm, and the geometric features of the minimum bounding box of each obstacle, such as geometric dimensions such as length, width, and height, are counted. Based on the minimum bounding box of each obstacle and the size of the truck, it is determined whether the vehicle to be detected exists in the vehicle loading and unloading area. When a minimum bounding box that matches the size of the truck exists, it is determined that the vehicle to be detected exists in the vehicle loading and unloading area. The above-mentioned clustering algorithms include, but are not limited to, clustering algorithms such as Euclidean clustering, K-mean clustering, or DBSCAN clustering.

[0062] In another embodiment, a camera can be provided based on the position of the above-mentioned laser radar, and an image of the vehicle loading and unloading area is synchronously collected based on the camera, recorded as a target image, and the presence of a vehicle to be detected in the vehicle loading and unloading area is determined based on the target image: obstacle target detection is performed on the target image based on a pre-trained machine learning model or deep learning model, and each obstacle category and the minimum external bounding box of each obstacle in the target image are identified, thereby judging whether there is a vehicle to be detected in the vehicle loading and unloading area based on the detected obstacle category. When the detected obstacle category includes a truck vehicle, it is determined that there is a vehicle to be detected in the vehicle loading and unloading area. The above-mentioned deep learning model can be a pulse neural network, a convolutional neural network or a fully connected neural network model. The above-mentioned target detection algorithm includes but is not limited to the YOLO algorithm, the SSD algorithm (Single Shot MultiBox Detectior), the RCNN algorithm (Region-CNN), the MobileNet algorithm, the Squeez-Net algorithm (Squeeze-and-Excitation Networks) and various variant algorithms.

[0063] Step 2): When there is a vehicle to be detected in the vehicle loading and unloading area, the license plate number of the vehicle to be detected is obtained, and the vehicle model and vehicle compartment size of the vehicle to be detected are determined based on the license plate number.

[0064] When a vehicle to be inspected is determined to be present in the loading and unloading area, the camera is automatically triggered to take a photo of the vehicle. Based on the captured image of the vehicle, the vehicle's license plate number is identified. For example, a character recognition algorithm or a pre-trained license plate recognition neural network model can be used to identify the vehicle's license plate number. Trucks can also be equipped with radio frequency identification to identify the vehicle's license plate number. The system can also accept license plate numbers manually entered by the user.

[0065] The license plate number of the vehicle to be detected is matched with the information in the database to obtain the vehicle model and compartment size corresponding to the license plate number of the vehicle to be detected. The compartment size includes information such as the length, width and height of the compartment for loading goods.

[0066] In one embodiment, this embodiment provides a specific implementation method for determining whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data of the vehicle loading and unloading area: obtaining the target minimum bounding box formed by the vehicle to be detected, and determining the yaw angle of the vehicle to be detected and the target distance between the rear end of the vehicle to be detected and the loading and unloading platform based on the point cloud coordinates of the target minimum bounding box; when the yaw angle is within a preset yaw range and the target distance is within a preset distance range, it is determined that the vehicle to be detected has reached the loading and unloading position.

[0067] The minimum bounding box model is constructed based on the detected vehicle to be detected, which is recorded as the target minimum bounding box. The target minimum bounding box of the vehicle to be detected is projected on the XY plane. Figure 4 The diagram shows the minimum bounding box of the vehicle to be detected on the XY plane. Based on the vertex coordinates of target minimum bounding box 40, the angle between target minimum bounding box 41 and the X-axis is calculated, denoted as yaw angle θ. The distance between the rear of the vehicle to be detected and the loading and unloading platform is calculated based on the vertex coordinates of target minimum bounding box 40 and the width of the loading and unloading platform, denoted as the target distance. This target distance is obtained by subtracting the width of the loading and unloading platform from the distance between the rear of the vehicle to be detected and the lidar. The target distance can be the vertical distance between the midpoint of the rear of the vehicle to be detected and the edge of the loading and unloading platform, or it can be the minimum vertical distance between each vertex of the target minimum bounding box and the edge of the loading and unloading platform.

[0068] When the position of the vehicle to be detected satisfies both the yaw angle being within the preset yaw range and the target distance being within the preset distance range, it is determined that the vehicle to be detected has accurately arrived at the loading and unloading position. When the yaw angle is not within the preset yaw range, or when the target distance is not within the preset distance range, an alarm is sounded by a buzzer, and the driver is guided to correct the direction based on the yaw angle and target distance. The above-mentioned preset yaw range and preset distance range can be set according to the position and angle requirements of the vehicle when loading and unloading goods, so that the vehicle can be parked in a standard posture for loading and unloading, which can not only facilitate cargo loading and unloading and improve cargo loading and unloading efficiency, but also avoid the impact of non-standard parking posture on the carriage loading and unloading rate detection results.

[0069] In another embodiment, parking space lines may be drawn on the ground, and whether the vehicle to be detected has reached the loading and unloading position may be determined by detecting the angle between the vehicle to be detected and the parking space lines.

[0070] In one embodiment, at least two parallel reference lines are provided in the vehicle loading and unloading area, such as Figure 4 As shown, two mutually parallel first reference lines 41 and second reference lines 42 are provided in the vehicle loading and unloading area. This embodiment further provides a specific implementation method for determining the working status of the vehicle to be detected: the working status of the vehicle to be detected is determined based on the intersection order of the target minimum bounding box and each reference line; wherein the working status includes an entry status, a loading / unloading status, and an exit status; the time when the vehicle to be detected is in the loading / unloading status can be determined based on the corresponding time of the entry status and the exit status, or based on the time when the vehicle to be detected arrives at the loading and unloading position and the door is closed after loading / unloading.

[0071] The coordinates of the reference line input by the user are obtained, the position of the target minimum bounding box of the vehicle to be detected is continuously obtained, and the running trajectory of the target minimum bounding box of the vehicle to be detected is obtained. When the target minimum bounding box of the vehicle to be detected first intersects with the first reference line 41 and then intersects with the second reference line 42, the working state of the vehicle to be detected is determined to be the inbound state. After determining that the working state of the vehicle to be detected is the inbound state, it is detected in real time whether the vehicle to be detected has accurately arrived at the loading and unloading position. When the target minimum bounding box of the vehicle to be detected first leaves the intersecting second reference line 42 and then leaves the intersecting first reference line 41, the working state of the vehicle to be detected is determined to be the outbound state. According to the time interval between the start time of the inbound state and the end time of the outbound state, the time required for the vehicle to be detected to complete the loading and unloading of goods is determined and recorded as the loading and unloading time, so that the vehicles in the logistics park can be scheduled for loading and unloading positions according to the loading and unloading time.

[0072] The above-mentioned vehicle loading and unloading task monitoring method provided in this embodiment can realize automatic detection of the entry and exit status of all vehicles in the loading and unloading area and automatic detection of the carriage loading and unloading rate by setting a laser radar above the loading and unloading platform, realizing one-to-many service optimization, and does not require manual detection, has low cost, and is not easily affected by environmental factors, thereby improving the accuracy of vehicle loading and unloading task monitoring.

[0073] Corresponding to the vehicle loading and unloading task monitoring method provided in the above embodiment, the embodiment of the present invention provides a vehicle loading and unloading task monitoring device, see Figure 5 The schematic diagram of the structure of a vehicle loading and unloading task monitoring device is shown, and the device includes the following modules:

[0074] The judgment module 51 is used to obtain point cloud data of the vehicle loading and unloading area and judge whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data.

[0075] The acquisition module 52 is used to acquire the point cloud data of the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position.

[0076] The first calculation module 53 is configured to extract point cloud data of the vehicle compartment area from the point cloud data of the vehicle to be detected, and calculate the depth of cargo loaded in the vehicle compartment of the vehicle to be detected based on the coordinates of the point cloud data of the vehicle compartment area.

[0077] The second calculation module 54 is used to calculate the compartment loading and unloading rate of the vehicle to be detected based on the compartment length and cargo depth of the vehicle to be detected and the empty loading and unloading rate of the vehicle to be detected.

[0078] The vehicle loading and unloading task monitoring device provided in this embodiment realizes the monitoring of the depth of cargo loaded in the compartment by calculating the cargo depth loaded in the compartment based on the point cloud data of the compartment area of ​​the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position. It realizes the monitoring of the compartment loading and unloading rate of the vehicle to be detected by calculating the compartment loading and unloading rate based on the compartment length, cargo depth and the empty loading and unloading rate of the vehicle to be detected, and will not be affected by environmental factors such as light, thereby reducing the error of the loading and unloading rate detection result and improving the accuracy of the vehicle loading and unloading rate detection.

[0079] In one embodiment, the second calculation module 54 is configured to calculate a ratio of cargo depth to vehicle compartment length, and calculate the difference between the ratio and the empty loading and unloading rate to obtain the vehicle compartment loading and unloading rate of the vehicle to be inspected.

[0080] In one embodiment, the car body frame of the vehicle to be detected is provided with an identification tape, and the echo intensity of the identification tape is different from the echo intensity of the car body. The first calculation module 53 is used to obtain the echo intensity of each point cloud data of the vehicle to be detected; based on the echo intensity of each point cloud data of the vehicle to be detected, the point cloud data of the car body frame is filtered out; and the car body area of ​​the vehicle to be detected is determined based on the area enclosed by the point cloud data of the car body frame.

[0081] In one embodiment, the point cloud data of the vehicle to be detected is obtained based on laser radar collection, and the first calculation module 53 is used to obtain the point cloud coordinates of the point cloud data corresponding to the border vertices of the car body area, and determine the radial distance between the tail end of the car body of the vehicle to be detected and the laser radar based on the point cloud coordinates of the border vertices of the car body area; calculate the average distance between each point cloud data in the car body area and the laser radar; calculate the difference between the average distance and the radial distance to obtain the free length of the car body of the vehicle to be detected; obtain the car body length of the vehicle to be detected, calculate the difference between the car body length and the free length of the car body, and obtain the depth of cargo loaded in the car body of the vehicle to be detected.

[0082] In one embodiment, the above device further comprises:

[0083] The first detection module is used to determine whether there is a vehicle to be detected in the vehicle loading and unloading area based on the point cloud data of the vehicle loading and unloading area or the target image of the vehicle loading and unloading area; when there is a vehicle to be detected in the vehicle loading and unloading area, the license plate number of the vehicle to be detected is obtained, and the vehicle model and car body size of the vehicle to be detected are determined based on the license plate number; wherein the car body size includes the car body length.

[0084] In one embodiment, the above-mentioned judgment module 51 is used to obtain the target minimum bounding box formed by the vehicle to be detected, and determine the yaw angle of the vehicle to be detected and the target distance between the rear end of the vehicle to be detected and the loading and unloading platform based on the point cloud coordinates of the target minimum bounding box; when the yaw angle is within a preset yaw range and the target distance is within a preset distance range, it is determined that the vehicle to be detected has arrived at the loading and unloading position.

[0085] In one embodiment, at least two mutually parallel reference lines are provided in the vehicle loading and unloading area, the laser radar is provided on a lifting mast, and the device further comprises:

[0086] The second detection module is used to determine the working status of the vehicle to be detected based on the intersection order of the target minimum bounding box and each reference line; wherein the working status includes the entry status, loading / unloading status and exit status; and the loading and unloading time of the vehicle to be detected is determined based on the corresponding time of the entry status and the exit status.

[0087] The height control module is used to control the laser radar to descend to a preset height when it is determined that the vehicle to be detected has reached the loading and unloading position, so that the laser radar can collect the entire image of the door area; when the loading and unloading rate of the vehicle to be detected is calculated and the door in the door area is detected to be closed, the laser radar is controlled to rise to a preset height to collect point cloud data of the vehicle loading and unloading area.

[0088] The above-mentioned vehicle loading and unloading task monitoring device provided in this embodiment can realize automatic detection of the entry and exit status of all vehicles in the loading and unloading area and automatic detection of the carriage loading and unloading rate by setting a laser radar above the loading and unloading platform, realizing one-to-many service optimization, and does not require manual detection, has low cost, and is not easily affected by environmental factors, thereby improving the accuracy of vehicle loading and unloading task monitoring.

[0089] The device provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned embodiments. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0090] Corresponding to the methods and devices provided in the aforementioned embodiments, an embodiment of the present invention also provides a vehicle loading and unloading task monitoring system, which includes: a laser radar, an image acquisition device and a controller, the controller including a processor and a storage device; a computer program is stored on the storage device, and when the computer program is run by the processor, it executes the vehicle loading and unloading task monitoring method provided in the aforementioned embodiment.

[0091] The above-mentioned laser radar and image acquisition device (camera) are arranged above the loading and unloading platform to collect point cloud data and images of the vehicle loading and unloading area.

[0092] An embodiment of the present invention provides a computer-readable medium, wherein the computer-readable medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method described in the above embodiment.

[0093] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned embodiment and will not be repeated here.

[0094] The computer program product of the vehicle loading and unloading task monitoring method, device and system provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments and will not be repeated here.

[0095] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0096] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0097] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0098] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A vehicle loading and unloading task monitoring method, characterized in that: include: Acquire point cloud data of the vehicle loading and unloading area, and determine whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data; When the vehicle to be detected arrives at the loading and unloading position, acquiring point cloud data of the vehicle to be detected; Extracting point cloud data of a compartment area from the point cloud data of the vehicle to be detected, and calculating the depth of cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area; Calculating a carriage loading and unloading rate of the vehicle to be detected based on the carriage length of the vehicle to be detected, the cargo depth, and the empty loading and unloading rate of the vehicle to be detected; The step of calculating the carriage loading and unloading rate of the vehicle to be detected based on the carriage length of the vehicle to be detected, the cargo depth, and the empty loading and unloading rate of the vehicle to be detected includes: Calculating the ratio of the cargo depth to the carriage length, and calculating the difference between the ratio and the empty loading and unloading rate to obtain the carriage loading and unloading rate of the vehicle to be detected; The point cloud data of the vehicle to be detected is acquired based on a laser radar, and the step of calculating the depth of cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area includes: Obtaining point cloud coordinates of the point cloud data corresponding to the vertices of the border of the vehicle compartment area, and determining the radial distance between the rear end of the vehicle compartment to be detected and the laser radar based on the point cloud coordinates of the vertices of the border of the vehicle compartment area; Calculate the average distance between each point cloud data in the vehicle compartment area and the laser radar; Calculating the difference between the average distance and the radial distance to obtain the free length of the vehicle compartment of the vehicle to be detected; The length of the carriage of the vehicle to be detected is obtained, and the difference between the length of the carriage and the free length of the carriage is calculated to obtain the depth of cargo loaded in the carriage of the vehicle to be detected.

2. The method according to claim 1, characterized in that The carriage frame of the vehicle to be detected is provided with an identification band, the echo intensity of the identification band is different from the echo intensity of the carriage, and the step of extracting the carriage area from the point cloud data of the vehicle to be detected includes: Obtaining the echo intensity of each point cloud data of the vehicle to be detected; Filtering the point cloud data of the vehicle compartment frame based on the echo intensity of each point cloud data of the vehicle to be detected; The vehicle compartment area of ​​the vehicle to be detected is determined based on the area enclosed by the point cloud data of the vehicle compartment frame.

3. The method according to claim 1, characterized in that Also includes: Determining whether there is a vehicle to be detected in the vehicle loading and unloading area based on the point cloud data of the vehicle loading and unloading area or the target image of the vehicle loading and unloading area; When there is a vehicle to be detected in the vehicle loading and unloading area, the license plate number of the vehicle to be detected is obtained, and the vehicle model and vehicle compartment size of the vehicle to be detected are determined based on the license plate number; wherein the vehicle compartment size includes the vehicle compartment length.

4. The method according to claim 3, characterized in that The step of determining whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data includes: Obtaining a target minimum bounding box formed by the vehicle to be detected, and determining the yaw angle of the vehicle to be detected and the target distance between the rear end of the vehicle to be detected and the loading and unloading platform based on the point cloud coordinates of the target minimum bounding box; When the yaw angle is within a preset yaw range and the target distance is within a preset distance range, it is determined that the vehicle to be detected has arrived at the loading and unloading position.

5. The method according to claim 4, characterized in that At least two mutually parallel reference lines are provided in the vehicle loading and unloading area, and the method further comprises: Determining the working state of the vehicle to be detected based on the intersection order of the target minimum bounding box and each reference line; wherein the working state includes an entry state, a loading / unloading state, and an exit state; The loading and unloading time of the vehicle to be detected is determined based on the corresponding time of the entry status and the exit status.

6. The method according to any one of claims 3 to 5, characterized in that: The laser radar is arranged on a lifting rod, and the method further comprises: When it is determined that the vehicle to be detected has arrived at the loading and unloading position, the laser radar is controlled to descend to a preset height so that the laser radar can capture the entire image of the vehicle door area; When the compartment loading and unloading rate of the vehicle to be detected is calculated and the door in the door area is detected to be closed, the laser radar is controlled to rise to the preset height to collect point cloud data of the vehicle loading and unloading area.

7. A vehicle loading and unloading task monitoring device, characterized in that: include: a judgment module, configured to obtain point cloud data of a vehicle loading and unloading area and judge whether the vehicle to be detected has reached the loading and unloading position based on the point cloud data; an acquisition module, configured to acquire point cloud data of the vehicle to be detected when the vehicle to be detected arrives at the loading and unloading position; a first calculation module, configured to extract point cloud data of a compartment area from the point cloud data of the vehicle to be detected, and calculate the depth of cargo loaded in the compartment of the vehicle to be detected based on the coordinates of the point cloud data of the compartment area; A second calculation module is used to calculate the carriage loading and unloading rate of the vehicle to be detected based on the carriage length of the vehicle to be detected, the cargo depth and the empty loading and unloading rate of the vehicle to be detected; The second calculation module is used to calculate the ratio of the cargo depth to the carriage length, and calculate the difference between the ratio and the empty loading and unloading rate to obtain the carriage loading and unloading rate of the vehicle to be detected; The point cloud data of the vehicle to be detected is obtained based on laser radar collection. The first calculation module is used to obtain the point cloud coordinates of the point cloud data corresponding to the border vertices of the compartment area, and determine the radial distance between the tail end of the compartment of the vehicle to be detected and the laser radar based on the point cloud coordinates of the border vertices of the compartment area; calculate the average distance between each point cloud data in the compartment area and the laser radar; calculate the difference between the average distance and the radial distance to obtain the free length of the compartment of the vehicle to be detected; obtain the compartment length of the vehicle to be detected, calculate the difference between the compartment length and the free length of the compartment, and obtain the depth of cargo loaded in the compartment of the vehicle to be detected.

8. A vehicle loading and unloading task monitoring system, characterized in that: include: A laser radar, an image acquisition device, and a controller, wherein the laser radar is arranged on the lifting mast, and the controller includes a processor and a storage device; The storage device stores a computer program, which, when executed by the processor, executes the method according to any one of claims 1 to 6.

Images