Control method and system for unattended empty vehicle inspection

Abstract

The invention discloses a control method and system for unattended empty vehicle inspection. The method comprises the following steps that a tail gate labeling data set and a carriage state labeling data set are constructed, a YOLO network is constructed, and a carriage tail gate recognition model and an empty vehicle recognition model are obtained through training based on the corresponding tail gate labeling data set and the carriage state labeling data set; measuring linear distances between different positions of the vehicle and the rear portal frame; obtaining the position of the vehicle in the image and the middle position of the identification frame based on a compartment tail gate identification model; obtaining the distance between the vehicle tail and the rear portal frame through coordinate transformation and geometric calculation; calculating the moving distances of the fixed-focus camera along the X axis, the Y axis and the Z axis; and identifying the state in the compartment based on the trained vehicle empty compartment identification model, judging whether the vehicle accords with a release rule or not, and completing an automatic vehicle detection process. According to the invention, the flexibility and effectiveness of video monitoring are obviously improved, and the automation and fairness of the vehicle card passing process are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle detection, and in particular to a control method and system for unattended empty vehicle inspection. Background Art

[0002] With the advancement of science and technology, the checkpoints in the special customs supervision areas are becoming more and more informationized, and the means of supervision are becoming more and more abundant. Through IoT devices such as cameras, floor scales, electronic locks, undercar scanners, and license plate recognition, all-round, multi-angle, and real-time dynamic active perception of vehicles entering and leaving the checkpoints is achieved. The application of these technologies has greatly reduced the work of manual intervention and promoted the development of unmanned checkpoints. However, it is difficult to automatically detect empty trucks and private cars entering and leaving the supervision area through current facilities. It is necessary to rely on security personnel for manual inspection, which not only increases labor costs, but also has risks such as non-inspection and concealment. In addition, the lack of automated inspection records makes it impossible to form a complete chain of evidence, which brings challenges to the tracing of subsequent illegal incidents.

[0003] At ports with large freight volumes, large container inspection systems have been deployed. The system uses high-energy X-rays to penetrate the carriages, generate clear internal images, and combines AI algorithms to achieve intelligent image review, which can effectively analyze whether there are goods, hidden compartments or contraband in the car. However, due to the use of high-energy X-rays, there are radiation hazards to the human body, so the safety requirements for the use of the equipment are relatively high. In addition, the equipment is expensive and requires professional operators, so it is not suitable for promotion and use at unmanned checkpoints in the supervision area. Summary of the invention

[0004] In order to overcome the defects and shortcomings of the prior art, the present invention provides a control method and system for unattended empty vehicle inspection. The present invention realizes spatial ranging positioning based on a monitoring camera and a rangefinder, can efficiently identify the internal state of the vehicle, and automatically determine whether the release conditions are met, which significantly improves the flexibility and effectiveness of video monitoring, and provides richer and more detailed visual information, thereby enhancing the performance and practicality of the checkpoint monitoring system, and enhancing the automation and fairness of the vehicle passing process.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a control method for unattended empty vehicle inspection, comprising the following steps:

[0007] Based on the fixed-focus camera, the vehicle rear image is collected and the different states of the tailgate are annotated, including closed tailgate, half-open tailgate and fully open tailgate, to form the corresponding tailgate annotation dataset;

[0008] Collect carriage images and annotate the different states of the carriage with and without cargo, forming corresponding empty state and non-empty state annotated data sets;

[0009] Construct a YOLO network, train the tailgate recognition model based on the corresponding tailgate annotation dataset, and train the empty compartment recognition model based on the corresponding empty state and non-empty state annotation datasets;

[0010] Taking the rear gantry as the reference, measure the straight-line distance between the vehicle and the rear gantry at different positions, calibrate it in the fixed-focus camera image, and save the corresponding pixel position;

[0011] Based on the tailgate recognition model, the tailgate is identified to obtain the position of the vehicle in the image and the middle position of the recognition frame;

[0012] By calculating the relative angle between the middle position of the rear of the vehicle and the rangefinder, setting the observation direction of the rangefinder, the distance between the rangefinder and the rear of the vehicle is obtained, and the distance between the rear of the vehicle and the rear gantry is obtained through coordinate conversion and geometric calculation;

[0013] The gantry is used as the origin to set the coordinate system, and the initial value is set according to the coordinate system standard. The distance that the fixed-focus camera moves along the X-axis, Y-axis, and Z-axis is calculated based on this position.

[0014] After driving the fixed-focus camera to the corresponding position, the car compartment image is collected, and the status inside the car compartment is identified based on the trained vehicle empty compartment recognition model to determine whether the vehicle meets the release rules, thus completing the automatic vehicle inspection process.

[0015] As a preferred technical solution, the fixed-focus camera and the rangefinder are arranged on a gantry and are located on the same longitudinal axis.

[0016] As a preferred technical solution, by calculating the relative angle between the middle position of the rear of the vehicle and the rangefinder, setting the observation direction of the rangefinder, obtaining the distance between the rangefinder and the rear of the vehicle, and obtaining the distance between the rear of the vehicle and the rear gantry through coordinate conversion and geometric calculation, specifically including:

[0017] Calculate the heading angle of the rangefinder, expressed as:

[0018]

[0019] Where PA is the horizontal field of view of the camera, X is the horizontal axis position of the center point of the rear of the vehicle, and W is the image width;

[0020] The actual distance between the rear of the vehicle and the rear gantry is:

[0021]

[0022] D c3=sin(abs(90-Ay))*Ad

[0023] D c2 =arcsin(β)*Ad

[0024] Where Ad is the actual distance measured by the rangefinder.

[0025] As a preferred technical solution, the gantry is used as the origin to set the coordinate system, the initial value is set according to the coordinate system standard, and the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis is calculated based on the position, specifically including:

[0026] The sliding distance along the X axis is: dx = L-0.5;

[0027] Among them, L represents the actual distance between the rear end of the vehicle and the rear gantry;

[0028] The sliding distance along the Y axis is: dy = D c3 *sgn(90-Ay);

[0029] Where Ay represents the heading angle of the rangefinder;

[0030] The sliding distance along the Z axis is:

[0031] dz=-(L 56 +0.5)

[0032] L 56 =cot(90-A 416 )*L

[0033] Among them, A 416 It is the size of ∠A4A1A6, A1 represents the position of the fixed-focus camera, A6 represents the point above the center axis of the rear of the vehicle, which is solved by identifying the corner vertices of the frame, and A4 represents the projection point of the midpoint of the center axis of the rear of the vehicle on the straight line where the set point A0 and the fixed-focus camera position A1 are located.

[0034] As a preferred technical solution, the step of calculating the pitch angle of the rangefinder is also included, which is expressed as:

[0035]

[0036] Among them, D cal is the straight-line distance between the rear of the vehicle and the gantry calculated according to the calibration value, β is the complementary angle between the fixed-focus camera and the observation point, and R is the distance between the fixed-focus camera and the rangefinder.

[0037] The present invention also provides a control system for unattended empty vehicle inspection, comprising: a fixed-focus camera, a rangefinder, a rear gantry, a tailgate annotation data set construction module, a compartment state annotation data set construction module, a YOLO network construction module, a compartment tailgate recognition model training module, a vehicle empty compartment recognition model training module, a straight-line distance measurement module, a vehicle recognition module, a distance calculation module between the rear of the vehicle and the rear gantry, a fixed-focus camera motion distance calculation module, a compartment state recognition module, and a release judgment module;

[0038] The tailgate annotation dataset construction module is used to collect vehicle tail images based on a fixed-focus camera, and annotate different states of the tailgate of the vehicle compartment, including a closed tailgate, a half-open tailgate, and a fully-open tailgate, to form a corresponding tailgate annotation dataset;

[0039] The carriage state annotation data set construction module is used to collect carriage images, annotate different states of carriages with and without cargo, and form corresponding empty state and non-empty state annotation data sets;

[0040] The YOLO network construction module is used to construct a YOLO network;

[0041] The vehicle tailgate recognition model training module is used to train a vehicle tailgate recognition model based on a corresponding tailgate annotation data set;

[0042] The vehicle empty compartment recognition model training module is used to train a vehicle empty compartment recognition model based on corresponding empty state and non-empty state labeling data sets;

[0043] The straight-line distance measurement module is used to measure the straight-line distance between different positions of the vehicle and the rear gantry based on the rear gantry, and calibrate it in the fixed-focus camera image to save the corresponding pixel position;

[0044] The vehicle recognition module is used to recognize the tailgate of the vehicle based on the tailgate recognition model, and obtain the position of the vehicle in the image and the middle position of the recognition frame;

[0045] The module for calculating the distance between the rear end of the vehicle and the rear gantry is used to calculate the relative angle between the middle position of the rear end of the vehicle and the rangefinder, set the observation direction of the rangefinder, obtain the distance between the rangefinder and the rear end of the vehicle, and obtain the distance between the rear end of the vehicle and the rear gantry through coordinate conversion and geometric calculation;

[0046] The fixed-focus camera movement distance calculation module is used to set the coordinate system with the gantry as the origin, set the initial value according to the coordinate system standard, and calculate the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis based on the position;

[0047] The compartment state recognition module is used to collect the compartment image after the fixed-focus camera reaches the corresponding position, and recognize the state inside the compartment based on the trained vehicle empty compartment recognition model;

[0048] The release judgment module is used to judge whether the vehicle meets the release rules and complete the automatic vehicle inspection process.

[0049] As a preferred technical solution, the fixed-focus camera and the rangefinder are arranged on a gantry and are located on the same longitudinal axis.

[0050] As a preferred technical solution, the module for calculating the distance between the rear of the vehicle and the rear gantry is used to calculate the relative angle between the middle position of the rear of the vehicle and the rangefinder, set the observation direction of the rangefinder, obtain the distance between the rangefinder and the rear of the vehicle, and obtain the distance between the rear of the vehicle and the rear gantry through coordinate conversion and geometric calculation, specifically including:

[0051] Calculate the heading angle of the rangefinder, expressed as:

[0052]

[0053] Where PA is the horizontal field of view of the camera, X is the horizontal axis position of the center point of the rear of the vehicle, and W is the image width;

[0054] The actual distance between the rear of the vehicle and the rear gantry is:

[0055]

[0056] D c3 =sin(abs(90-Ay))*Ad

[0057] D c2 =arcsin(β)*Ad

[0058] Where Ad is the actual distance measured by the rangefinder.

[0059] As a preferred technical solution, the fixed-focus camera movement distance calculation module is used to set the coordinate system with the gantry as the origin, set the initial value according to the coordinate system standard, and calculate the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis based on the position, specifically including:

[0060] The sliding distance along the X axis is: dx = L-0.5;

[0061] Among them, L represents the actual distance between the rear end of the vehicle and the rear gantry;

[0062] The sliding distance along the Y axis is: dy = D c3 *sgn(90-Ay);

[0063] Where Ay represents the heading angle of the rangefinder;

[0064] The sliding distance along the Z axis is:

[0065] dz=-(L 56 +0.5)

[0066] L 56 =cot(90-A 416 )*L

[0067] Among them, A 416 It is the size of ∠A4A1A6, A1 represents the position of the fixed-focus camera, A6 represents the point above the center axis of the rear of the vehicle, which is solved by identifying the corner vertices of the frame, and A4 represents the projection point of the midpoint of the center axis of the rear of the vehicle on the straight line where the set point A0 and the fixed-focus camera position A1 are located.

[0068] As a preferred technical solution, the pitch angle of the rangefinder is also calculated, which is expressed as:

[0069]

[0070] Among them, D cal is the straight-line distance between the rear of the vehicle and the gantry calculated according to the calibration value, β is the complementary angle between the fixed-focus camera and the observation point, and R is the distance between the fixed-focus camera and the rangefinder.

[0071] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0072] (1) In the traditional fixed-position shooting mode, due to the limitation of shooting angle and the influence of the flash light’s working distance, it is often impossible to completely cover the vehicle compartment, making it difficult to obtain clear, high-quality photos. The present invention calculates the pitch angle of the rangefinder and combines coordinate transformation and geometric calculation to accurately obtain the distance between the rear of the vehicle and the rear gantry. The present invention completes flexible shooting based on three-axis motion and can dynamically adjust the shooting position according to the needs of specific scenes. This dynamic adjustment not only ensures full coverage of the shooting range, but also can capture clear and complete image materials. Compared with the traditional mode, the present invention significantly improves the flexibility and effectiveness of video monitoring, provides richer and more detailed visual information, and thus enhances the performance and practicality of the bayonet monitoring system.

[0073] (2) The present invention can efficiently identify the internal state of a vehicle and automatically determine whether the conditions for release are met, effectively solving the bottleneck problem that the current checkpoint system can only identify the external state of the vehicle through sensing equipment and relies on manual judgment of the internal state of the vehicle. It provides a feasible technical solution for unmanned checkpoints. On the one hand, it reduces or eliminates the need for personnel on duty, thereby reducing costs and increasing efficiency for the park; on the other hand, it enhances the automation and fairness of the vehicle passing process, avoiding false detection, missed detection, and non-pickup due to human factors.

[0074] (3) The present invention realizes spatial ranging and positioning based on a surveillance camera and a rangefinder, and is applicable to various application scenarios for accurately positioning spatial objects. The rangefinder can accurately measure the distance of stationary objects and can provide solutions for various scenarios requiring depth measurement data. Compared with the depth camera solution, the present invention has excellent environmental adaptability and can maintain stable working performance under complex lighting and weather conditions. Compared with the millimeter wave radar and ultrasonic radar solutions, the present invention can not only measure the distance of moving targets, but also accurately measure the distance of stationary objects. It can be widely used in the fields of building monitoring, indoor route planning, indoor object measurement and safety protection, and has obvious advantages in cost-effectiveness, versatility, and adaptability.

[0075] (4) The present invention performs azimuth calculation based on the field of view and the camera heading angle and pitch angle. On the premise of determining the height of the object, accurate spatial transformation can be performed based on the monocular camera, that is, the conversion between the pixel position and the spatial physical position, such as the conversion between the geodetic coordinate system position, the custom coordinate axis position and the image pixel position, which is suitable for object labeling and object positioning in AR scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a flow chart of the control method of unattended empty vehicle inspection of the present invention;

[0077] Figure 2 This is a working schematic diagram of the fixed-focus camera and the rangefinder of the present invention;

[0078] Figure 3 is a schematic diagram of the polar lines of the rangefinder and the camera of the present invention;

[0079] Figure 4 This is a schematic diagram of camera calibration of the present invention;

[0080] Figure 5 This is a schematic diagram of calculating the heading angle of the rangefinder of the present invention;

[0081] Figure 6 It is a schematic diagram of the field of view angle of a fixed-focus camera based on ground coordinates of the present invention;

[0082] Figure 7 This is a schematic diagram of image correction based on image coordinates of the present invention;

[0083] Figure 8 It is a schematic diagram of calculating the pitch angle of the rangefinder of the present invention;

[0084] Fig. 9 This is a schematic diagram of calculating the distance between the rear end of the vehicle and the gantry frame of the present invention;

[0085] Fig.10 It is a schematic diagram of the shaft system and mechanical movement of the present invention;

[0086] Fig.11 This is a schematic diagram of the photographing position of the zoom camera of the present invention. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0088] Example 1

[0089] like Figure 1 As shown, this embodiment provides a control method for unattended empty vehicle inspection, comprising the following steps:

[0090] S1: The fixed-focus camera collects images of the rear of the vehicle and annotates the different states of the tailgate, including closed tailgate, half-open tailgate and fully-open tailgate, to form the corresponding tailgate annotation dataset;

[0091] Collect carriage images and annotate the different states of the carriage with and without cargo, forming corresponding empty state and non-empty state annotated data sets;

[0092] S2: Build a YOLO network, train the tailgate recognition model based on the corresponding tailgate annotation dataset, and train the empty compartment recognition model based on the corresponding empty state and non-empty state annotation datasets;

[0093] Specifically, in this embodiment, a fixed-focus camera and a rangefinder with a fixed field of view angle are installed on the gantry, wherein the fixed-focus camera is used to capture an image of the rear of the vehicle, and the rangefinder is preferably a laser rangefinder with a heading angle and a pitch angle adjustment, which is used to measure the specific position of the vehicle;

[0094] In the initial stage of tailgate status recognition, it is necessary to configure a fixed-focus camera to capture the image data of the passing trucks, and use image annotation tools (such as LabelImg) to annotate the different states of the tailgate, including closed tailgate, half-open tailgate and fully open tailgate, so as to form the corresponding annotation data set. After the data set is built, the data is loaded and converted into the format required by the YOLO network using a deep learning framework (such as PyTorch), followed by deep learning training to build a tailgate recognition model. Each time the tailgate status is recognized, the trained tailgate recognition model is used to judge the status of the tailgate in real time and output the specific position of the vehicle in the picture.

[0095] For the recognition of the contents of the carriage images, a large amount of image materials need to be collected in the early stage, and image annotation tools (such as LabelImg) are used to annotate the different states of the carriage with and without cargo, forming the corresponding empty state and non-empty state annotation data sets. Then, the data is loaded and converted into the format required by the YOLO network through a deep learning framework (such as PyTorch), deep training is performed, and a vehicle empty compartment recognition model is constructed to determine whether the vehicle meets the release rules;

[0096] S3: Taking the rear gantry as a reference, measure the straight-line distance between different positions of the vehicle and the rear gantry, calibrate it in the fixed-focus camera image, and save the corresponding pixel position;

[0097] S4: Identify the tailgate based on the tailgate recognition model to obtain the position of the vehicle in the image and the middle position of the recognition frame;

[0098] S5: By calculating the relative angle between the middle position of the rear of the vehicle and the rangefinder, setting the observation direction of the rangefinder, obtaining the distance between the rangefinder and the rear of the vehicle, and obtaining the distance between the rear of the vehicle and the rear gantry through coordinate conversion and geometric calculation;

[0099] In this embodiment, by identifying the rear of the truck, the position of the vehicle in the image (the four vertices of the rectangle) can be determined, and the middle position C (X, Y) can be obtained. Then, by calculating the relative angle between the position and the rangefinder (i.e., the heading angle and pitch angle calculated in the following steps), the rangefinder observation direction is set to ensure that the measured distance is the distance between the rangefinder and the rear of the vehicle (diagonal distance). Based on the measured distance, the precise distance (straight line distance) between the rear of the vehicle and the rear gantry is finally obtained through coordinate conversion and geometric calculation.

[0100] like Figure 2 As shown in the figure, the fixed-focus camera and the rangefinder are installed on the same vertical axis, the pan / tilt angle range of the rangefinder is 0° to 180°, and the distance unit is meter. In addition, the horizontal field of view angle of the fixed-focus camera is PA, the vertical field of view angle is TA, the pitch angle is Ca (determined during installation), and the vertical distance between the fixed-focus camera and the rangefinder is R.

[0101] like Figure 3 As shown, since the image captured by the fixed-focus camera is two-dimensional, for the observation point in the middle of the rear of the vehicle, the observation position of the rangefinder may be on the polar line formed by the fixed-focus camera and point P, and the specific point position cannot be determined, so the pitch angle of the rangefinder cannot be accurately set.

[0102] The present invention uses a camera calibration method to obtain image depth estimation data. For the fixed-focus camera image, a marking line is set according to the straight-line measurement distance between the real scene and the gantry, and the pixel position of the corresponding marking line is recorded. The position of the rear of the vehicle in the image is identified through image analysis, and the pixel position of the rear of the vehicle is compared with the calibration position to calculate the pitch angle of the rangefinder. Figure 4 As shown, according to the relative relationship between the horizontal line below the rear of the vehicle and the calibration line, the approximate distance D between the rear of the vehicle and the longitudinal axis where the camera is located can be calculated. cal , that is, the parking position of the vehicle, thereby guiding the rangefinder to perform accurate distance measurement.

[0103] like Figure 5 As shown in the figure, assuming that the resolution of the captured images in the surveillance video is W*H, the pixel coordinates of the lower right corner of the identified rear rectangle are PR(X r ,Y r ), the upper left pixel coordinate PL(X l ,Y l ), the center pixel position of the rectangle is C(X, Y), and the heading angle of the rangefinder is calculated based on the image width, recognition position and horizontal field of view of the camera:

[0104]

[0105] Where PA is the horizontal field of view of the camera, X is the horizontal axis position of the center point of the rear of the vehicle, and W is the image width;

[0106] The pitch angle calculation of the rangefinder depends on the calibration value of the camera, the pitch angle of the camera, the vertical height difference between the rangefinder and the camera, etc. Since the camera is not shooting parallel to the ground, but shooting along the set pitch angle, this will cause perspective distortion. Therefore, the method of calculating the heading angle (geometric relationship) cannot be used to calculate the pitch angle of the rangefinder, and image correction is required. In order to facilitate the calculation, the coordinates based on the ground need to be converted into coordinates based on the image. By fusing the information of the two coordinate systems, the image can be corrected and positioned, so that the elevation angle of any point in the image can be calculated. First, calculate the lens coverage range based on the ground coordinate system, such as Figure 6 The specific values ​​are as follows:

[0107]

[0108] The distance between A′ and B′ is:

[0109] The distance between O′A′ is Lo′a′=Xa′

[0110] Based on the principle of similar triangles, other data of the image coordinate system can be inferred, such as Figure 7As shown, PD = PB, PF is the bisector of ∠APB, and the specific values ​​are as follows:

[0111] The length of OA is Loa=H*Lo′a′ / La′b′, where H is the height of the image;

[0112] The length of PO is

[0113] Xa=Loa

[0114] Xb=Loa+H

[0115]

[0116] The length of DB is

[0117] The slope of DB is Kdb = (Xb-Xb) / Yd

[0118] Through image recognition, the specific position of the rear of the car in the image is obtained. Assuming that the center coordinates of the rear of the car are C(X, Y), the value converted into the image coordinate system is C(Xc, 0), then Xc = Y + Loa. Assuming that the intersection of the PC extension line and BD is E(Xe, Ye), the following equation can be obtained according to the slope principle:

[0119]

[0120] The specific values ​​of Xe and Ye can be calculated from the above equations:

[0121] Xe=Xc*Lpo / (Lpo+Xc)

[0122]

[0123] The length of DE can be calculated as:

[0124] ∠APC is α=TA*Lde / Lbd

[0125] ∠OPC

[0126] The pitch angle of the rangefinder is calculated by the calibration value of the camera, the pitch angle Ca of the camera, the vertical height difference between the rangefinder and the camera, and the identification position. It is specifically expressed as:

[0127]

[0128] Among them, D cal is the straight-line distance between the rear of the vehicle and the gantry calculated based on the calibration value, β is the complementary angle between the fixed-focus camera and the observation point, and R is the distance between the fixed-focus camera and the rangefinder;

[0129] like Figure 8 As shown, due to D cal As an approximation, the elevation angle of the rangefinder may not necessarily point strictly to the vertical midpoint of the trunk, however, considering the large area of ​​the trunk door, the elevation angle can still be regarded as valid because the actual distance measured from the rangefinder is ultimately used.

[0130] Through the above calculation, the rangefinder can be controlled to accurately measure the distance of the vehicle. As an optimization solution, the rangefinder can be controlled to perform multiple distance measurements to determine whether the rangefinder is measuring the distance to the trunk. Assuming that the distance measured by the rangefinder is Ad, the actual distance between the rear of the vehicle and the rear gantry can be calculated. Since the heading angle of the rangefinder may not be 0°, the plane where the rangefinder ray is located is not perpendicular to the plane where the rear of the vehicle is located. Further spatial conversion is required, such as Fig. 9 As shown, the plane where the surface P1CP2 is located is not perpendicular to the plane where the rear of the vehicle is located. The distance of CP3 can be calculated as: D c3 =sin(abs(90-Ay))*Ad, the distance of CP2 is: D c2 =arcsin(β)*Ad, these two values ​​can be used to calculate the actual distance between the rear end of the vehicle and the rear gantry

[0131] D c3 =sin(abs(90-Ay))*Ad

[0132] D c2 =arcsin(β)*Ad

[0133] When the heading angle of the rangefinder is 90°, then L = sin(180-Cp)*Ad;

[0134] S5: Set the coordinate system with the gantry as the origin, set the initial value of the motor according to the coordinate system standard, and calculate the distance required to move along each axis based on this position;

[0135] In this embodiment, the crossbar needs to slide longitudinally according to the parking position of the truck to guide the filming equipment to a suitable position for taking pictures. The rear gantry adopts a truss structure as a whole, and three-axis (X-axis, Y-axis, Z-axis) movement is achieved by a motor. Gear racks are installed on the gantry track, and servo motors or stepper motors are installed at both ends of the crossbar to drive the crossbar to move forward and backward. A motor (sliding trolley) is also installed on the crossbar for sliding left and right. A push rod (fixed axis) is installed on the crossbar, which can be extended up and down, and the filming equipment can be lowered to a specified position. The motor used in the present invention is equipped with a high-resolution encoder that can accurately control the movement position. Fig.10As shown, the gantry is used as the origin to set the coordinate system, and the motor is set to the initial value according to the coordinate system standard. The device is located at (0, 0, 0) by default, and the distance required to move along each axis is calculated based on this position.

[0136] From the previous calculation, we know that the distance from the rear gantry to the rear of the vehicle is L, and the distance the horizontal axis needs to slide along the X axis is dx = L-0.5, where 0.5 is subtracted to leave enough space for the shooting equipment to fall down and avoid the equipment from hitting the vehicle while still being able to take clear photos;

[0137] The sliding distance of the Y axis needs to be determined according to the value of the heading angle Ay of the rangefinder, dy = D c3 *sgn(90-Ay), where sgn(90-Ay) is -1 when Ay>90 and 1 when Ay≤90;

[0138] According to practical results, the camera shoots from top to bottom, which has better effect and the flash has a longer working distance. Therefore, it is necessary to calculate the position close to the upper part of the rear of the car, such as Fig.11 As shown, the camera should be between A3 and A6. In the image, by identifying the corner vertices of the box, the coordinate value of A6 can be calculated, assuming it is A6 (X6, Y6). By using the same method as the rangefinder elevation angle (ignoring the specific process in this step), the relevant angle can be calculated. According to the relationship between the camera capture image and space, the size of ∠A4A1A6 is calculated to be A 416 , then the distance between A5 and A6 is: L 56 =cot(90-A 416 )*L, so the distance moved along the Z axis is dz=-(L 56 +0.5), where 0.5 meters is subtracted from the height to prevent the camera from being above the car and unable to fully capture the interior of the car.

[0139] The horizontal displacement of the control crossbar is dx meters, the vertical displacement of the motor on the crossbar is dy meters, and the telescopic arm of the crossbar is extended downward by dz meters, so that the camera equipped with an independent flash can take clear pictures inside the car.

[0140] S6: After the motor drives the fixed-focus camera to reach the corresponding position, it collects the image of the vehicle compartment, identifies the status inside the vehicle compartment based on the trained vehicle empty compartment recognition model, determines whether the vehicle meets the release rules, and completes the automatic vehicle inspection process.

[0141] The above method realizes automatic photography, judgment and release of the status inside the carriage, truly realizing unmanned checkpoints.

[0142] In current vehicle inspections, when trucks enter or exit checkpoints, the customs inspects and releases the business carried by the vehicles based on the collected data. If on-site inspection is required, the driver stops the vehicle when passing the checkpoint and opens the door to allow security personnel to check. If there is no problem, the barrier is lifted to release the vehicle. The above business process requires security personnel to be on call 24 hours a day to inspect vehicles entering and leaving the area at any time. The present invention captures clear photos of the interior of the vehicle, performs AI recognition in the background, and intelligently determines whether the vehicle meets the conditions for exiting the area, thereby automatically completing the inspection without the need for security personnel to intervene.

[0143] Example 2

[0144] The present embodiment provides a control system for unattended empty vehicle inspection, which is used to implement the control method for unattended empty vehicle inspection of the above-mentioned embodiment 1, including: a fixed-focus camera, a rangefinder, a rear gantry, a tailgate annotation data set construction module, a car body state annotation data set construction module, a YOLO network construction module, a car body tailgate recognition model training module, a vehicle empty compartment recognition model training module, a straight-line distance measurement module, a vehicle recognition module, a distance calculation module between the rear end of the car and the rear gantry, a fixed-focus camera motion distance calculation module, a car body state recognition module, and a release judgment module;

[0145] In this embodiment, the tailgate annotation dataset construction module is used to collect vehicle rear images based on a fixed-focus camera, and annotate different states of the tailgate of the vehicle compartment, including a closed tailgate, a half-open tailgate, and a fully-open tailgate, to form a corresponding tailgate annotation dataset;

[0146] In this embodiment, the carriage state annotation data set construction module is used to collect carriage images, annotate different states of no cargo and cargo in the carriage, and form corresponding empty state and non-empty state annotation data sets;

[0147] In this embodiment, the YOLO network construction module is used to construct a YOLO network;

[0148] In this embodiment, the vehicle tailgate recognition model training module is used to train the vehicle tailgate recognition model based on the corresponding tailgate annotation data set;

[0149] In this embodiment, the vehicle empty compartment recognition model training module is used to train a vehicle empty compartment recognition model based on corresponding empty state and non-empty state labeling data sets;

[0150] In this embodiment, the straight-line distance measurement module is used to measure the straight-line distance between different positions of the vehicle and the rear gantry based on the rear gantry, and calibrate it in the fixed-focus camera image to save the corresponding pixel position;

[0151] In this embodiment, the vehicle recognition module is used to recognize the tailgate of the vehicle based on the tailgate recognition model, and obtain the position of the vehicle in the image and the middle position of the recognition frame;

[0152] In this embodiment, the module for calculating the distance between the rear of the vehicle and the rear gantry is used to calculate the relative angle between the middle position of the rear of the vehicle and the rangefinder, set the observation direction of the rangefinder, obtain the distance between the rangefinder and the rear of the vehicle, and obtain the distance between the rear of the vehicle and the rear gantry through coordinate conversion and geometric calculation;

[0153] In this embodiment, the fixed-focus camera movement distance calculation module is used to set the coordinate system with the gantry as the origin, set the initial value according to the coordinate system standard, and calculate the movement distance of the fixed-focus camera along the X-axis, Y-axis, and Z-axis based on the position;

[0154] In this embodiment, the compartment state recognition module is used to collect the compartment image after the fixed-focus camera reaches the corresponding position, and recognize the state inside the compartment based on the trained vehicle empty compartment recognition model;

[0155] In this embodiment, the release judgment module is used to judge whether the vehicle meets the release rules and complete the automatic vehicle inspection process.

[0156] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A control method for unattended empty vehicle inspection, characterized in that: The steps include: Based on the fixed-focus camera, the vehicle rear image is collected and the different states of the tailgate are annotated, including closed tailgate, half-open tailgate and fully open tailgate, to form the corresponding tailgate annotation dataset; Collect carriage images and annotate the different states of the carriage with and without cargo, forming corresponding empty state and non-empty state annotated data sets; Construct a YOLO network, train the tailgate recognition model based on the corresponding tailgate annotation dataset, and train the empty compartment recognition model based on the corresponding empty state and non-empty state annotation datasets; Taking the rear gantry as the reference, measure the straight-line distance between the vehicle and the rear gantry at different positions, calibrate it in the fixed-focus camera image, and save the corresponding pixel position; Based on the tailgate recognition model, the tailgate is identified to obtain the position of the vehicle in the image and the middle position of the recognition frame; By calculating the relative angle between the middle position of the rear of the vehicle and the rangefinder, setting the observation direction of the rangefinder, the distance between the rangefinder and the rear of the vehicle is obtained, and the distance between the rear of the vehicle and the rear gantry is obtained through coordinate conversion and geometric calculation; The gantry is used as the origin to set the coordinate system, and the initial value is set according to the coordinate system standard. The distance that the fixed-focus camera moves along the X-axis, Y-axis, and Z-axis is calculated based on this position. After driving the fixed-focus camera to the corresponding position, the car compartment image is collected, and the status inside the car compartment is identified based on the trained vehicle empty compartment recognition model to determine whether the vehicle meets the release rules, thus completing the automatic vehicle inspection process.

2. The control method for unattended empty vehicle inspection according to claim 1, characterized in that: The fixed-focus camera and the rangefinder are arranged on the gantry and are located on the same longitudinal axis.

3. The control method for unattended empty vehicle inspection according to claim 1, characterized in that: By calculating the relative angle between the middle position of the rear of the vehicle and the rangefinder, setting the observation direction of the rangefinder, and obtaining the distance between the rangefinder and the rear of the vehicle, the distance between the rear of the vehicle and the rear gantry is obtained through coordinate conversion and geometric calculation, including: Calculate the heading angle of the rangefinder, expressed as: Where PA is the horizontal field of view of the camera, X is the horizontal axis position of the center point of the rear of the vehicle, and W is the image width; The actual distance between the rear of the vehicle and the rear gantry is: <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> c3 <h2 style=";text-align:left;direction:ltr"> = (sin(abs(90-Ay))*Ad) D c2 =arcsin(β)*Ad Where Ad is the actual distance measured by the rangefinder.

4. The control method for unattended empty vehicle inspection according to claim 3 is characterized in that: From now on, the gantry is used as the origin, the coordinate system is set, the initial value is set according to the coordinate system standard, and the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis is calculated based on this position, including: The sliding distance along the X axis is: dx = L-0.5; Among them, L represents the actual distance between the rear end of the vehicle and the rear gantry; The sliding distance along the Y axis is: dy = D c3 *sgn(90-Ay); Where Ay represents the heading angle of the rangefinder; The sliding distance along the Z axis is: dz=-(L 56 +0.5) IT 56 =elbow(90-A 416 )*IT Among them, A 416 It is the size of ∠A4A1A6, A1 represents the position of the fixed-focus camera, A6 represents the point above the center axis of the rear of the vehicle, which is solved by identifying the corner vertices of the frame, and A4 represents the projection point of the midpoint of the center axis of the rear of the vehicle on the straight line where the set point A0 and the fixed-focus camera position A1 are located.

5. The control method for unattended empty vehicle inspection according to claim 1, characterized in that: It also includes the rangefinder pitch angle calculation step, which is expressed as: Among them, D cal is the straight-line distance between the rear of the vehicle and the gantry calculated according to the calibration value, β is the complementary angle between the fixed-focus camera and the observation point, and R is the distance between the fixed-focus camera and the rangefinder.

6. A control system for unattended empty vehicle inspection, characterized in that: include: Fixed-focus camera, rangefinder, rear gantry, tailgate annotation dataset construction module, car state annotation dataset construction module, YOLO network construction module, car tailgate recognition model training module, vehicle empty compartment recognition model training module, straight-line distance measurement module, vehicle recognition module, distance calculation module between the rear of the car and the rear gantry, fixed-focus camera motion distance calculation module, car state recognition module, release judgment module; The tailgate annotation dataset construction module is used to collect vehicle tail images based on a fixed-focus camera, and annotate different states of the tailgate of the vehicle compartment, including a closed tailgate, a half-open tailgate, and a fully-open tailgate, to form a corresponding tailgate annotation dataset; The carriage state annotation data set construction module is used to collect carriage images, annotate different states of carriages with and without cargo, and form corresponding empty state and non-empty state annotation data sets; The YOLO network construction module is used to construct a YOLO network; The vehicle tailgate recognition model training module is used to train a vehicle tailgate recognition model based on a corresponding tailgate annotation data set; The vehicle empty compartment recognition model training module is used to train a vehicle empty compartment recognition model based on corresponding empty state and non-empty state labeling data sets; The straight-line distance measurement module is used to measure the straight-line distance between different positions of the vehicle and the rear gantry based on the rear gantry, and calibrate it in the fixed-focus camera image to save the corresponding pixel position; The vehicle recognition module is used to recognize the tailgate of the vehicle based on the tailgate recognition model, and obtain the position of the vehicle in the image and the middle position of the recognition frame; The module for calculating the distance between the rear end of the vehicle and the rear gantry is used to calculate the relative angle between the middle position of the rear end of the vehicle and the rangefinder, set the observation direction of the rangefinder, obtain the distance between the rangefinder and the rear end of the vehicle, and obtain the distance between the rear end of the vehicle and the rear gantry through coordinate conversion and geometric calculation; The fixed-focus camera movement distance calculation module is used to set the coordinate system with the gantry as the origin, set the initial value according to the coordinate system standard, and calculate the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis based on the position; The compartment state recognition module is used to collect the compartment image after the fixed-focus camera reaches the corresponding position, and recognize the state inside the compartment based on the trained vehicle empty compartment recognition model; The release judgment module is used to judge whether the vehicle meets the release rules and complete the automatic vehicle inspection process.

7. The control system for unattended empty vehicle inspection according to claim 1, characterized in that: The fixed-focus camera and the rangefinder are arranged on the gantry and are located on the same longitudinal axis.

8. The control system for unattended empty vehicle inspection according to claim 1, characterized in that: The module for calculating the distance between the rear end of the vehicle and the rear gantry is used to calculate the relative angle between the middle position of the rear end of the vehicle and the rangefinder, set the observation direction of the rangefinder, obtain the distance between the rangefinder and the rear end of the vehicle, and obtain the distance between the rear end of the vehicle and the rear gantry through coordinate conversion and geometric calculation, which specifically includes: Calculate the heading angle of the rangefinder, expressed as: Where PA is the horizontal field of view of the camera, X is the horizontal axis position of the center point of the rear of the vehicle, and W is the image width; The actual distance between the rear of the vehicle and the rear gantry is: <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> c3 <h2 style=";text-align:left;direction:ltr"> = (sin(abs(90-Ay))*Ad) D c2 =arcsin(β)*Ad Where Ad is the actual distance measured by the rangefinder.

9. The control system for unattended empty vehicle inspection according to claim 8, characterized in that: The fixed-focus camera movement distance calculation module is used to set the coordinate system with the gantry as the origin, set the initial value according to the coordinate system standard, and calculate the distance of the fixed-focus camera moving along the X-axis, Y-axis, and Z-axis based on the position, specifically including: The sliding distance along the X axis is: dx = L-0.5; Among them, L represents the actual distance between the rear end of the vehicle and the rear gantry; The sliding distance along the Y axis is: dy = D c3 *sgn(90-Ay); Where Ay represents the heading angle of the rangefinder; The sliding distance along the Z axis is: dz=-(L 56 +0.5) IT 56 =elbow(90-A 416 )*IT Among them, A 416 It is the size of ∠A4A1A6, A1 represents the position of the fixed-focus camera, A6 represents the point above the center axis of the rear of the vehicle, which is solved by identifying the corner vertices of the frame, and A4 represents the projection point of the midpoint of the center axis of the rear of the vehicle on the straight line where the set point A0 and the fixed-focus camera position A1 are located.

10. The control system for unattended empty vehicle inspection according to claim 1, characterized in that: It also includes the calculation of the rangefinder pitch angle, expressed as: Among them, D cal is the straight-line distance between the rear of the vehicle and the gantry calculated according to the calibration value, β is the complementary angle between the fixed-focus camera and the observation point, and R is the distance between the fixed-focus camera and the rangefinder.

Images