Open truck empty box detection device and method

Abstract

The invention discloses an empty box detection device for an open truck, comprising: a license plate recognition module, which detects the vehicle and the license plate in real time when the vehicle passes through a detection position, and outputs the vehicle and license plate detection results and the license plate recognition results to a data processing module; the data processing module, which determines whether the vehicle is parked according to the vehicle and license plate detection results, and receives scanned data for real-time processing, determines whether the truck position has been reached, and calculates the measurement position of the sensor; the sensor module, which starts a scanning process when it is determined that the vehicle has stopped, quickly moves forward along a guide rail from an initial position, and transmits the scanned data and the motion data to the data processing module in real time through a network; the motion module, which moves to the measurement position with the sensor module according to the measurement position and a sensor module start signal, performs data scanning and collection, and sends the data to the data processing module, the data processing module processes the collected data, completes volume calculation and foreign matter detection, and outputs the result.

Description

Technical Field

[0001] The invention relates to an empty box detection device and method for an open truck, belonging to the technical field of traffic logistics safety detection. Background Art

[0002] With the growing demand for transportation, truck transportation is an important part of logistics, including a large number of open trucks and containers. Empty container detection is required in certain application scenarios, such as the inspection of vehicles or containers declared empty at customs, and the safety monitoring of whether the mine trucks going to and from get off work at the mine are empty.

[0003] Conventional manual spot checks can no longer meet the growing demand for vehicle traffic. Monitoring and detection technologies have gradually begun to be applied in empty box detection. For example, vehicle cargo security inspection systems using X-ray radiation imaging can obtain transmission images of trucks or containers, and determine whether they are empty boxes through manual or automatic comparison. Such devices are expensive and require certain radiation protection conditions, and their application and maintenance are relatively complex. There are also surveillance cameras that capture video images, which are manually viewed or automatically identified to detect whether they are empty boxes. Such devices or methods are affected by complex light conditions and installation position angles in the application environment, and the consistency of video images is poor, that is, there are large differences in video images taken at different times for the same vehicle, and there are certain differences in the interiors of different trucks. This will lead to a decrease in the accuracy and stability of automatic empty box identification, affecting actual use. Summary of the invention

[0004] The purpose of the present invention is to provide an empty box detection device and method for convertible trucks, so as to solve the defects of the prior art that the accuracy and stability of automatic empty box recognition are reduced due to the influence of complex lighting conditions and installation position angle in the application environment, and the poor consistency of video images, thus affecting actual use.

[0005] First aspect:

[0006] An empty box detection device for an open truck, the device comprising:

[0007] The license plate recognition module is used to detect the vehicle and the license plate in real time when the vehicle passes the detection position, and output the vehicle and license plate detection results and the license plate recognition results to the data processing module;

[0008] The data processing module is used to determine whether the vehicle has stopped based on the vehicle and license plate detection results, and receive the scan data for real-time processing, determine whether the truck has arrived at the location, and calculate the measurement position of the sensor;

[0009] The sensing module is used to start a scanning process when judging that the vehicle is parked, quickly move forward along the guide rail from the initial position, and transmit the scanning data and movement data to the data processing module in real time through the network;

[0010] The motion module is used to move the sensor module to the measurement position according to the measurement position and the sensor module start signal to scan and collect data, and send it to the data processing module. The data processing module processes the collected data, completes volume calculation, foreign body detection, and outputs the results;

[0011] The network module is used to complete the signal connection between the license plate recognition module, the data processing module, the sensor module and the motion module.

[0012] Furthermore, the sensor module is composed of a three-dimensional scanning device and a high-definition camera, which is used to obtain three-dimensional space point cloud data and two-dimensional image data in the truck or container below.

[0013] Furthermore, the three-dimensional scanning device is installed upside down on the top of the box to scan the internal space of the lower car box.

[0014] Furthermore, the high-definition camera adopts a wide-angle lens or a fisheye lens.

[0015] Furthermore, the three-dimensional scanning device adopts laser radar.

[0016] Second aspect:

[0017] A method for detecting an empty box of a convertible truck, the method comprising:

[0018] Detect vehicles and their license plates, determine if a vehicle is approaching and stop the vehicle;

[0019] Start the 3D point cloud data scanning device to drive the sensor module to move along the guide rail from the initial position of the car body through the motion module, and collect the 3D point cloud data and motion data;

[0020] The collected 3D point cloud data is processed in real time to determine whether the vehicle has reached the rear of the truck. If the vehicle has reached the rear of the truck, the measurement position is calculated based on the current movement position.

[0021] 3D point cloud data scanning and 2D high-definition image shooting are performed according to the measurement position, and the collected data is processed to complete volume calculation and foreign body detection.

[0022] Furthermore, the method for determining that a vehicle is approaching and stopping includes:

[0023] Based on the vehicle detection results output by the license plate recognition camera, determine whether the front face of the truck is detected;

[0024] If the front face of the truck is detected, the license plate recognition module is further used to obtain the detected license plate result and obtain the position information of the license plate target frame in the video image. Otherwise, it is further determined whether the front face of the truck is detected.

[0025] Determine whether the front face area and the license plate area of ​​the truck move in the image. If neither the front face area nor the license plate area of ​​the truck moves in the image, it is determined that the vehicle has stopped and the result is output.

[0026] Furthermore, the method for determining whether the vehicle reaches the rear position of the truck compartment includes:

[0027] Get the three-dimensional point cloud below the current position. The point cloud coordinates in the three-dimensional coordinate system based on the ground are P n (x n ,y n ,z n );

[0028] Take the rectangular area below the radar from the point cloud as the rear tailgate detection area, that is, the three-dimensional coordinate value x n ,y n ,z n Point cloud P within the set range b ,P b ∈P n ;

[0029] Calculate the sum of the vertical coordinate values ​​of all point clouds in the tailgate detection area S = ∑z b ;

[0030] If S>T z , that is, if a certain number of point clouds are detected that are higher than the ground, it is determined that the motion module and the sensor module have reached the rear position of the vehicle and output the current position d0; where T z It is the point cloud Z value and threshold, which are set comprehensively according to the number of 3D point clouds and the height of the truck tailgate.

[0031] Furthermore, the method for calculating the measurement position according to the current motion position includes:

[0032] Assume that the initial position of the motion module is the coordinate origin 0, the position of the motion module at the rear of the vehicle detected in the above step 4 is d0, and the length of the car box is l, then the measurement position is d0+l / 2; the length of the car box can be set in advance according to the type or license plate number of the inspected vehicle. When it is impossible to know in advance, the position of the front fender of the car box d1 can be detected according to the method of the above step 4, l=d1-d0.

[0033] Furthermore, the method of processing the collected data to complete volume calculation and foreign body detection includes:

[0034] According to the point cloud Pn Search for the maximum and minimum values ​​in the X and Y directions, select the point cloud data set in the range according to the boundary value, and perform plane fitting to obtain the four faces of the side wall of the car box;

[0035] Test the four sides of the vehicle compartment wall;

[0036] According to the point cloud data set of each side wall, search for point clouds with Z values ​​greater than the set threshold, and fit a straight line to obtain the upper edge of each side wall;

[0037] According to the coordinates of the four surfaces of the bottom surface and the side wall, the internal volume of the car box is calculated;

[0038] Point cloud P n The point clouds of the four side walls detected in the test are removed, and the remaining point clouds are the bottom surface of the car box and the point clouds of foreign objects. After plane fitting, the bottom surface of the car box is obtained;

[0039] Check whether there are protruding foreign objects on each surface, select the point clouds that do not belong to the side wall and whose Z value is greater than the Z value coordinate of the bottom surface for clustering, and if the point cloud density is high enough to belong to an object, it is considered to be a detected foreign object.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The method and device adopt a three-dimensional scanning module to obtain three-dimensional data of the box on the open truck, determine whether it is an empty box, and automatically identify the items in the box with a high-definition camera to obtain the detection result of the items in the box; the method and device are simple in composition, easy to install and maintain, and low in cost;

[0042] The method and device use a license plate recognition system to make vehicle parking judgments, which is low in cost and can obtain license plate information, making it convenient for users to manage detection data, and is also conducive to automatic comparison and detection processing by vehicle, which can improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the device of the present invention;

[0044] Figure 2 It is a schematic diagram of the motion module and the sensor module of the present invention;

[0045] Figure 3 is a flow chart of the empty box detection method of the present invention;

[0046] Figure 4 is a flow chart of a method for judging whether a truck has arrived and stopped according to the present invention;

[0047] Figure 5 This is a flow chart of the method for judging whether the rear position of the truck box has been reached in the present invention.

[0048] Figure 6 is a schematic diagram of a three-dimensional point cloud of the present invention;

[0049] Figure 7 It is a flow chart of three-dimensional point cloud data processing of the present invention. DETAILED DESCRIPTION

[0050] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0051] Embodiment 1:

[0052] like Figure 1-Figure 2 As shown, the present invention discloses an empty box detection device for an open truck, the device comprising a sensor module, a motion module, a guide rail, a license plate recognition module, a data processing module, and a network module;

[0053] The license plate recognition module is used to detect the vehicle and the license plate in real time when the vehicle passes the detection position, and output the vehicle and license plate detection results and the license plate recognition results to the data processing module;

[0054] The data processing module is used to determine whether the vehicle has stopped based on the vehicle and license plate detection results, and receive the scan data for real-time processing, determine whether the truck has arrived at the location, and calculate the measurement position of the sensor;

[0055] The sensing module is used to start a scanning process when judging that the vehicle is parked, quickly move forward along the guide rail from the initial position, and transmit the scanning data and movement data to the data processing module in real time through the network;

[0056] The motion module is used to move the sensor module to the measurement position according to the measurement position and the sensor module start signal to scan and collect data, and send it to the data processing module. The data processing module processes the collected data, completes volume calculation, foreign body detection, and outputs the results;

[0057] The network module is used to complete the signal connection between the license plate recognition module, the data processing module, the sensor module and the motion module;

[0058] When the vehicle passes the detection position, the license plate recognition module detects the vehicle and the license plate in real time, and outputs the vehicle and license plate detection results and the license plate recognition results to the data processing module. The data processing module determines whether the vehicle is parked based on the vehicle and license plate detection results; when it is determined that the vehicle is parked, a scanning process is started, the sensor module turns on various sensors, and the motion module moves the sensor module quickly forward from the initial position along the guide rail, and transmits the scanning data and motion data to the data processing module in real time through the network; the data processing module receives the scanning data for real-time processing, determines whether the truck position has been reached, and calculates the measurement position of the sensor and sends it to the motion module. The motion module moves the sensor module to the measurement position for data scanning and collection, and sends it to the data processing module. The data processing module processes the collected data, completes volume calculation and foreign body detection, and outputs the results.

[0059] The sensor module consists of a 3D scanning device and a high-definition camera to obtain 3D spatial point cloud data and 2D image data inside the truck or container below. Figure 2 As shown. Preferably, the three-dimensional scanning device uses a laser radar, such as the RS-Bpearl laser radar of RoboSense, which has an ultra-wide angle of 360°×90° and is mounted on the top of the box to scan the interior space of the car below. Preferably, the high-definition camera uses a wide-angle lens or a fisheye lens, the field of view of which can cover the entire interior space of the car box, and the resolution is sufficient to meet the requirements of being able to detect the target object. The sensor module is connected to the data processing module, and can transmit the collected three-dimensional point cloud data and two-dimensional image data to the data processing module, and receive the data acquisition control signal output by the data processing module.

[0060] The motion module is used to mount the sensor module and drive the sensor module to move on the guide rail so as to perform 3D data scanning and image capture at different positions. Preferably, the motion module can be implemented using a guide rail robot, which has a faster movement speed and higher positioning accuracy.

[0061] The license plate recognition module is used to detect and identify vehicles and their license plates entering the detection area, and output the detection results of the vehicles appearing in the video image (such as the vehicle front detection frame) and the license plate detection and OCR recognition results in real time, and output them to the data processing module. Preferably, the license plate recognition camera used for entering and exiting the closed parking lot is used for implementation.

[0062] The data processing module is composed of a high-performance industrial computer and data processing software. The industrial computer is connected to the 3D scanning device, high-definition camera, license plate recognition module, and the motion module to control its movement in real time. The data processing software mainly includes:

[0063] Motion control module, 3D data acquisition and point cloud processing module, high-definition camera capture and recognition module, and vehicle license plate information processing module.

[0064] Embodiment 2:

[0065] A method for detecting an empty box of a convertible truck, the method comprising:

[0066] Detect vehicles and their license plates, determine if a vehicle is approaching and stop the vehicle;

[0067] Start the 3D point cloud data scanning device to drive the sensor module to move along the guide rail from the initial position of the car body through the motion module, and collect the 3D point cloud data and motion data;

[0068] The collected 3D point cloud data is processed in real time to determine whether the vehicle has reached the rear of the truck. If the vehicle has reached the rear of the truck, the measurement position is calculated based on the current movement position.

[0069] Three-dimensional point cloud data scanning and two-dimensional high-definition image shooting are carried out according to the measurement position, and the collected data is processed to complete volume calculation and foreign body detection.

[0070] Empty box detection method process Figure 3 As shown, the specific steps are as follows:

[0071] Step 1: Use Figure 1 The license plate recognition module shown in the figure detects the vehicle and its license plate, determines whether a truck is coming and stops. The specific method is described in detail below.

[0072] Step 2: If a truck arrives and stops, start the scanning process and open Figure 1 The three-dimensional point cloud data scanning device of the middle sensor module collects the three-dimensional point cloud data; otherwise, return to step one.

[0073] Step 3: The motion module carries the sensor module from the initial position to the front along the guide rail, and transmits the collected 3D point cloud data and motion data to the data processing module in real time through the network.

[0074] Step 4: The data processing module processes the received three-dimensional point cloud data in real time to determine whether it has reached the rear position of the truck compartment.

[0075] Step 5: If the vehicle reaches the rear of the truck box, the measurement position is calculated based on the current position of the motion module and the measurement position is sent to the motion module; otherwise, return to step 3.

[0076] Step 6: The motion module moves the sensor module to the measurement position to perform 3D point cloud data scanning and 2D high-definition image capture, and sends the collected data to the data processing module.

[0077] Step 7: The data processing module processes the collected data to complete volume calculation and foreign body detection.

[0078] Step 8: Output the detection result. The motion module brings the sensor module back to the initial position, ending the scanning process.

[0079] The flowchart of the method for determining whether a truck has arrived and stopped in step 1 is as follows: Figure 4 As shown, the specific steps are as follows:

[0080] Step 1-1: Determine whether the front face of the truck is detected based on the vehicle detection result output by the license plate recognition camera. Existing mature license plate recognition cameras can obtain vehicle detection results and output the position information of the detected vehicle target frame in the image for each frame of video image.

[0081] Step 1-2: If the front face of the truck is detected, the license plate recognition module is further used to obtain the detected license plate result, and the position information of the license plate target frame in the video image is obtained; otherwise, the process returns to step 1-1.

[0082] Step 1-3, determine whether the front face area and the license plate area of ​​the truck move in the image. Specifically, the vehicle can be determined based on the change in the coordinate position of the target frame in the image. When the change in the coordinate position is less than the set threshold, it is considered that the vehicle has not moved. The optical flow method can also be used to obtain the motion feature information of the targets in the truck front face target frame sub-image and the license plate target frame sub-image respectively. When the speed is 0, it is determined that the vehicle has not moved.

[0083] Step 1-4: If the front face area and the license plate area of ​​the truck do not move in the image, it is determined that the vehicle has stopped and the result is output; otherwise, return to step 1-3.

[0084] The flowchart of the method for determining whether the rear position of the truck box is reached in step 4 is as follows: Figure 5 As shown, the specific steps are as follows:

[0085] Step 4-1, obtain the 3D point cloud below the current position, such as Figure 6 The point cloud in the hemispherical scanning area below the laser radar is shown. The point cloud coordinates in the three-dimensional coordinate system based on the ground are P n (x n ,y n ,z n ).

[0086] Step 4-2: Take the rectangular area below the radar from the point cloud as the rear tailgate detection area, such as Figure 6 The detection area at the rear of the truck shown in FIG. n ,y n ,z n Point cloud P within the set range b ,P b∈P n .

[0087] Step 4-3: Calculate the sum of the vertical coordinate values ​​of all point clouds in the tailgate detection area S = ∑z b .

[0088] Step 4-4: If S>T z , that is, if a certain number of point clouds are detected that are higher than the ground, the motion module and the sensor module are determined to have reached the rear position of the vehicle and the current position d0 is output; otherwise, return to step 4-1. z It is the point cloud Z value and threshold, which can be set comprehensively according to the number of 3D point clouds and the height of the truck tailgate.

[0089] The calculation method of the measured position in step 5 is as follows:

[0090] Assume that the initial position of the motion module is the coordinate origin 0, the position of the motion module at the rear of the vehicle detected in step 4 is d0, and the length of the vehicle box is l, then the measured position is d0+l / 2. The length of the vehicle box can be set in advance according to the type or license plate number of the vehicle being inspected. If it is not known in advance, the position of the front baffle of the vehicle box d1 can be detected according to the method in step 4 above, l=d1-d0.

[0091] The data processing method in step seven includes three-dimensional point cloud data processing and two-dimensional image target detection, as follows:

[0092] The 3D point cloud data processing process is as follows: Figure 7 As shown, the specific steps are as follows:

[0093] Step 7-1: Detect the four faces of the front, back, left, and right sides of the side wall of the vehicle box. n The maximum and minimum values ​​in the X and Y directions are searched, which are the spatial boundary values ​​of the side wall. According to the boundary values, the point cloud data set in the range is selected for plane fitting to obtain the four faces of the side wall.

[0094] Step 7-2: Calculate the upper edge of the side wall of the vehicle box. According to the point cloud data set of each side wall surface, search for point clouds with Z values ​​greater than the set threshold, and fit a straight line to obtain the upper edge of each side wall.

[0095] Step 7-3: Detect the bottom surface of the car box. n The point clouds of the four side wall surfaces detected in step 7-1 are removed, and the remaining point clouds are the bottom surface of the car box and the foreign matter. The bottom surface of the car box is obtained after plane fitting.

[0096] Step 7-4: Calculate the volume based on the side, top and bottom surfaces. Calculate the internal volume of the car based on the coordinates of the bottom and side walls.

[0097] Step 7-5: Check whether there are protruding objects, i.e. foreign objects, on each surface. For example, based on the bottom surface in step 7-3, select the point clouds that do not belong to the side wall surface and whose Z value is greater than the Z value coordinate of the bottom surface for clustering. If the point cloud density is high enough to belong to an object, it is considered to be a detected foreign object.

[0098] Step 7-6: Output volume calculation results and foreign body detection results.

[0099] 2D image foreign body detection: 2D high-definition images are used for foreign body detection. The type of foreign body is specified by the user in advance and is implemented using a deep learning detection network, such as the Yolo series network, SSD, CenterNet, etc.

[0100] Among them, in step six, determine whether it is necessary to scan multiple times at different positions based on the resolution of the 3D point cloud scanning system (i.e., the density of the point cloud) and the size of the detection target. If the resolution of the 3D point cloud scanning system is not high enough, multiple scans can be performed at different positions on the rail above the car body. An upgrade device can also be installed on the motion module at the same position of the rail to achieve multiple scans at different heights to obtain more 3D point cloud data in the detected car body space. Combine or splice the point cloud data of multiple scans. Multiple scans can also be achieved by using multiple 3D point cloud scanning systems to scan at different positions simultaneously.

[0101] During the above multiple scanning processes, the license plate recognition module can be used to determine whether the vehicle has changed from a parked state to a moving state. If the vehicle starts to move, the data processing module controls the sensor module to stop working, and the motion module to retract and return to the initial position to avoid collision with the vehicle. The method for determining vehicle movement is similar to the method in step 1 above, which is to determine whether the vehicle and the license plate are moving.

[0102] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting an empty box of an open truck, characterized in that: The method comprises: Detect vehicles and their license plates, determine if a vehicle is approaching and stop the vehicle; Start the 3D point cloud data scanning device to drive the sensor module to move along the guide rail from the initial position of the car body through the motion module to collect 3D point cloud data and motion data; The collected 3D point cloud data is processed in real time to determine whether the vehicle has reached the rear of the truck. If the vehicle has reached the rear of the truck, the measurement position is calculated based on the current movement position. Perform 3D point cloud data scanning and 2D high-definition image capture based on the measurement position, and process the collected data to complete volume calculation and foreign body detection; The method for determining whether the rear position of the truck box has been reached comprises: Get the three-dimensional point cloud below the current position. The point cloud coordinates in the three-dimensional coordinate system based on the ground are P n (x n ,y n , z n ); Take the rectangular area below the radar from the point cloud as the rear tailgate detection area, that is, the three-dimensional coordinate value x n ,y n , z n Point cloud P within the set range b , P b ∈P n ; Calculate the sum of the vertical coordinate values ​​of all point clouds in the tailgate detection area S = ∑z b ; If S>T z , that is, if a certain number of point clouds are detected that are higher than the ground, it is determined that the motion module and the sensor module have reached the rear position of the vehicle and output the current position d0; where T z It is the point cloud Z value and threshold, which are set comprehensively according to the number of 3D point clouds and the height of the truck tailgate.

2. The method for detecting an empty box of a convertible truck according to claim 1, characterized in that: The method for judging the arrival of a vehicle and stopping the vehicle comprises: Based on the vehicle detection results output by the license plate recognition camera, determine whether the front face of the truck is detected; If the front face of the truck is detected, the license plate recognition module is further used to obtain the detected license plate result and obtain the position information of the license plate target frame in the video image. Otherwise, it is further determined whether the front face of the truck is detected. Determine whether the front face area and the license plate area of ​​the truck move in the image. If neither the front face area nor the license plate area of ​​the truck moves in the image, it is determined that the vehicle has stopped and the result is output.

3. The method for detecting an empty box of a convertible truck according to claim 1, characterized in that: The method for calculating the measurement position according to the current motion position includes: Assume that the initial position of the motion module is the coordinate origin 0, the position of the motion module at the rear of the vehicle detected is d0, the length of the car body is l, and the measured position is d0+1 / 2; the length of the car body is set in advance according to the type or license plate number of the inspected vehicle. When it cannot be known in advance, the position of the front fender of the car body d1 is detected based on whether it has reached the rear position of the truck car body, l=d1-d0.

4. The method for detecting an empty box of a convertible truck according to claim 1, characterized in that: The method for processing the collected data to complete volume calculation and foreign body detection includes: According to the point cloud P n Search for the maximum and minimum values ​​in the X and Y directions, select the point cloud data set in the range according to the boundary value, and perform plane fitting to obtain the four faces of the side wall of the car box; Detect the four sides of the vehicle compartment wall; According to the point cloud data set of each side wall, search for point clouds with Z values ​​greater than the set threshold, and fit a straight line to obtain the upper edge of each side wall; According to the coordinates of the four surfaces of the bottom surface and the side wall, the internal volume of the car box is calculated; Point cloud P n The point clouds of the four side walls detected in the test are removed, and the remaining point clouds are the bottom surface of the car box and the point clouds of foreign objects. After plane fitting, the bottom surface of the car box is obtained; Check whether there are protruding foreign objects on each surface, select the point clouds that do not belong to the side wall and whose Z value is greater than the Z value coordinate of the bottom surface for clustering, and if the point cloud density is high enough to belong to an object, it is considered to be a detected foreign object.

Images