An assisted driving precise positioning method based on image visual technology

Abstract

The application relates to an image visual technology-based assisted driving precision positioning method, and belongs to the technical field of intelligent assisted driving, which solves the problem of great difficulty in existing vehicle precision positioning. The image visual technology-based assisted driving precision positioning method comprises the following steps: step S1, setting a ground mark in a planned parking area; step S2, calibrating an auxiliary line and an auxiliary point, and making and storing a calibration parameter file; step S3, driving the vehicle to the vicinity of the planned parking area, and running an assisted driving positioning software; step S4, moving the vehicle to find the arrow mark on the ground in a video picture of a display device, so that the arrow mark direction is consistent with the vehicle head direction, and the arrow mark line is on a straight line with the parking auxiliary line; and step S5, continuously moving the vehicle to make the auxiliary point coincide with the fixed-point parking point mark on the ground, and completing fixed-point parking. The method realizes precision positioning without personnel command, and saves manpower cost.

Description

Technical Field

[0001] This invention belongs to the field of intelligent assisted driving technology, specifically relating to a precise positioning method for assisted driving based on image vision technology. Background Technology

[0002] Currently, large vehicles, such as fire trucks, crane trucks, and special transport vehicles, face extremely demanding technical requirements during special operations. These requirements include parking the vehicle at a fixed point or next to a target with an error margin of less than 10cm, or ensuring the vehicle's orientation is within a precise directional error margin of less than 0.5°. These stringent technical requirements pose significant challenges to the drivers.

[0003] To meet these stringent technical requirements, the current method mainly relies on rearview mirrors, positioning rulers, and the assistance of a driving instructor.

[0004] The current solution has the following problems: 1. Using a positioning ruler relies on human eyes and experience to judge whether it is level or vertical, which introduces significant errors. 2. If the positioning point changes, the positioning ruler becomes meaningless and needs to be remade. 3. It requires a high level of experience from both the commanding personnel and the driver, increasing training difficulty. 4. Throughout the operation, the driver is completely under command and cannot have a comprehensive grasp of the vehicle's attitude, posing a safety hazard. 5. For certain special positioning requirements of the vehicle, such as center of gravity or center positioning, the vehicle's center and center of gravity are generally located at the bottom of the vehicle, which cannot be accurately determined by the driver and commanding personnel, increasing the difficulty of positioning. Therefore, the existing vehicle positioning method suffers from low accuracy, difficulty in personnel command, and inflexible positioning aids, failing to meet the requirements of refined operational procedures. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide a precise positioning method for assisted driving based on image vision technology, which assists drivers in completing precise positioning and attitude adjustment of large vehicles, thereby solving the problem of the difficulty of precise positioning of existing vehicles.

[0006] The objective of this invention is mainly achieved through the following technical solutions:

[0007] A precise positioning method for assisted driving based on image vision technology includes the following steps:

[0008] Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings.

[0009] Step S2: Park the vehicle at any location, mark the vehicle width line, the predetermined parking point, and the parking line on the ground, collect real-time video information from the ground, push it to the display device, and then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, mark the auxiliary line of the vehicle width line marked on the ground, the auxiliary point corresponding to the predetermined parking point, and the parking auxiliary line corresponding to the parking line in the video on the display device, form a layer image, and save it as a calibration parameter file;

[0010] Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in step S2, and at the same time collect real-time video images of the ground and push them to the display device. The display device simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points and parking auxiliary lines, as well as the real-time video images of the ground.

[0011] Step S4: Move the vehicle, locate the ground markings set in the planned parking area in the real-time video image, and make the arrow markings align with the direction of the vehicle's front, and ensure that the arrow markings are in a straight line with the parking auxiliary lines in the layer image to determine the vehicle's travel path.

[0012] Step S5: Continue moving the vehicle until the auxiliary points in the layer image displayed on the display device coincide with the fixed-point parking location markers in the real-time ground video image, thus completing the fixed-point parking.

[0013] Furthermore, in step S1, the fixed parking point is the vertical projection of the vehicle's center of gravity onto the ground, or a point around the vehicle at a certain distance from the vehicle.

[0014] Furthermore, in step S1, the fixed parking spot marker is circular with a cross mark in the middle, the center of the circle being the fixed parking spot, and the radius is 15-25cm; the arrow marker extending from the fixed parking spot marker points in the direction of the vehicle's front, the arrow marker line is 70-100cm long, and the marker line is 1-2cm wide.

[0015] Furthermore, the radius of the designated parking spot marker is preferably 20cm; the length of the arrow marker line is preferably 80cm; and the width of the marker line is preferably 2cm.

[0016] Furthermore, step S2 includes the following steps:

[0017] Step S21: Park the vehicle at any location;

[0018] Step S22: First, mark the longitudinal centerline and width line of the vehicle. Then, accurately mark the predetermined parking point and parking line of the vehicle. The predetermined parking point and the fixed parking point in step S1 are completely consistent with the distance and orientation of the vehicle.

[0019] Step S23: Install a camera sensor on the vehicle so that the vehicle width line, the predetermined parking point and the parking line marked on the ground are within the field of view of the camera.

[0020] Step S24: Based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line in the video image captured by the camera sensor, the vehicle width auxiliary line, auxiliary point, and parking auxiliary line are calibrated accordingly, forming a layer image and storing it as a calibration parameter file, thus completing the calibration of the parking auxiliary line, vehicle width auxiliary line, and auxiliary point.

[0021] Furthermore, in step S21, the ground level deviation at the vehicle parking location is controlled within 5°.

[0022] Furthermore, in step S22, when marking the longitudinal centerline of the vehicle, a plumb line is placed at the center of the front and rear of the vehicle, and a mark is made at the point where the plumb line touches the ground. The two mark points are connected to form the longitudinal centerline of the vehicle.

[0023] When marking the width lines of a vehicle, two straight lines parallel to the longitudinal centerline are marked at the two ends where the vehicle is widest laterally.

[0024] The parking line is a straight line that passes through the predetermined parking point and is parallel to or coincides with the longitudinal centerline; when the predetermined parking point is on the longitudinal centerline, the parking line coincides with the longitudinal centerline; when the predetermined parking point is not on the longitudinal centerline, the parking line is parallel to the longitudinal centerline.

[0025] Furthermore, both the parking line and the vehicle width line must be extended in both directions along a straight line, exceeding the length of the vehicle body.

[0026] Furthermore, the parking line and vehicle width line should extend at least 200cm beyond the front and rear of the vehicle, respectively.

[0027] Furthermore, in step S23, camera sensors are installed on the centerline directly in front of and behind the vehicle, such that the field of view of the camera sensor directly in front is forward and downward, and the field of view of the camera sensor directly behind is backward and downward; if the vehicle's intended parking point and parking line cannot fall within the field of view of the above two camera sensors, a camera sensor is installed at an appropriate location on the vehicle so that the vehicle's intended parking point and parking line fall within the field of view of the camera sensor.

[0028] Compared with the prior art, the present invention can achieve at least one of the following technical effects:

[0029] (1) This invention can achieve precise positioning without personnel command, saving manpower costs.

[0030] (2) The present invention improves the positioning accuracy by using auxiliary coordinates. The theoretical maximum positioning error is 5cm and the orientation error is <0.3°. Moreover, the real-time image of the fixed parking point can be observed through video, which improves driving safety.

[0031] (3) The present invention includes a vehicle width auxiliary line and a parking auxiliary line. The vehicle width auxiliary line can clearly show people or obstacles on the vehicle's movement trajectory; the parking auxiliary line can determine the vehicle's travel path.

[0032] (4) The method of the present invention is simple to operate, easy to learn, and greatly improves work efficiency.

[0033] (5) The method of the present invention is suitable for fixed-point positioning at any position in front, rear, left, right and bottom of a vehicle, and has a low implementation cost. Attached Figure Description

[0034] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0035] Figure 1 This is a schematic diagram of the ground markings for the planned parking area;

[0036] Figure 2 This is a schematic diagram of the components of an advanced driver assistance system;

[0037] Figure 3 Draw a side view diagram for the longitudinal centerline;

[0038] Figure 4 Draw a top-view diagram of the vehicle width line and the predetermined parking point 100cm from the center of the rear of the vehicle;

[0039] Figure 5 Side view diagram showing the installation of cameras on the centerline at the front and rear of the vehicle;

[0040] Figure 6 A top-down view of the field of view of the cameras on the centerline directly in front of and behind the vehicle;

[0041] Figure 7 A schematic diagram of the calibration display screen;

[0042] Figure 8 This is a schematic diagram of the rear-view fixed-point parking video footage from Example 1;

[0043] Figure 9 This is a schematic diagram of a video image showing a center-of-gravity parking maneuver in Example 2;

[0044] Figure 10 This is a schematic diagram of the lateral fixed-point parking video footage from Example 3;

[0045] In the diagram, 1-Planned parking area; 2-Designated parking spot marker; 3-Arrow markers extending from the designated parking spots; 4-Vehicle; 5-Front-end camera; 6-Front-end camera field of view; 7-Rear camera; 8-Rear camera field of view; 9-Ground; 10-Plumb line at the front of the vehicle; 11-Plumb line at the front of the vehicle; 12-Plumb line at the rear of the vehicle; 13-Plumb line at the rear of the vehicle; 14-Vehicle width line; 15-Parking line; 16-Reserved parking spot; 17-Arrow markers extending from the reserved parking spots; 18-Video feed; 19-Vehicle width auxiliary line; 20-Parking auxiliary line; 21-Auxiliary points and arrows. Detailed Implementation

[0046] The following describes in further detail a precise positioning method for assisted driving based on image vision technology with reference to specific embodiments. These embodiments are for comparison and explanation purposes only, and the present invention is not limited to these embodiments.

[0047] Currently, large vehicles, such as fire trucks, crane trucks, and special transport vehicles, face extremely demanding technical requirements during special operations. These requirements include parking the vehicle at a fixed point or next to a target with an error margin of less than 10cm, or ensuring the vehicle's orientation is within a precise directional error margin of less than 0.5°. These stringent technical requirements pose significant challenges to the drivers.

[0048] Based on this, the present invention provides a method for precise positioning in assisted driving based on image vision technology, comprising the following steps:

[0049] Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings.

[0050] Step S2: Park the vehicle at any location, mark the vehicle width line, the predetermined parking point, and the parking line on the ground, collect real-time video information from the ground, push it to the display device, and then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, mark the auxiliary lines for the vehicle width line marked on the ground, the auxiliary points for the predetermined parking point, and the parking auxiliary lines for the parking line in the video on the display device, forming a layer image, and then create and store the calibration parameter file;

[0051] Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in step S2, and at the same time collect real-time video information of the ground and push it to the display device. The display device simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points and parking auxiliary lines, as well as the real-time collected ground image.

[0052] Step S4: Move the vehicle, locate the ground markings set in the planned parking area in the real-time video image, and make the arrow markings align with the direction of the vehicle's front, and ensure that the arrow markings are in a straight line with the parking auxiliary lines in the layer image to determine the vehicle's travel path.

[0053] Step S5: Continue moving the vehicle until the auxiliary points in the layer image displayed on the display device coincide with the fixed-point parking location markers on the real-time video ground, thus completing the fixed-point parking.

[0054] It should be noted that in step S1, as Figure 1 As shown. The larger the icon and the thicker the line, the clearer the video image display. However, excessively thick lines reduce accuracy. The line width should be 1-2cm, preferably 2cm. Fixed-point parking location markers are circular with a cross in the center. The center of the circle indicates the fixed-point parking location, and the radius is 15-25cm, preferably 20cm. The arrow points in the direction of the vehicle's direction, and the arrow marker line length is 70-100cm, preferably 80cm.

[0055] Fixed parking spots are points related to the vehicle's physical data, such as the vehicle's center of gravity, center position, or a point around the vehicle at a certain distance, such as 100cm from the center of the rear of the vehicle.

[0056] Center of gravity positioning refers to the requirement that the vehicle's center of gravity be vertically projected onto a specific point on the ground. A vehicle's center of gravity is typically located at the bottom, making parking difficult. This requirement arises when the vehicle is transported by train flatbed or airplane, ensuring that the center of gravity falls precisely in the designated position to prevent tilting or overturning.

[0057] Center positioning refers to projecting the center of the vehicle vertically onto a specific point on the ground. This mainly addresses the stress issues that occur when the vehicle is in operation, and the center is typically located at the bottom of the vehicle.

[0058] Specifically, step S2 includes the following steps:

[0059] Step S21: Park the vehicle at any location.

[0060] Step S22: The calibration personnel first mark the longitudinal centerline and width line of the vehicle, and then accurately mark the vehicle's intended parking point and parking line;

[0061] Step S23: Install a camera sensor on the vehicle so that the vehicle width line, the predetermined parking point and the parking line marked on the ground are within the field of view of the camera.

[0062] Step S24: Start and run the assisted driving software system. Based on the vehicle width line, the predetermined parking point, and the coordinate position of the parking line in the video, mark the vehicle width auxiliary line, auxiliary point, and parking auxiliary line accordingly, form a layer image, and save it as a calibration parameter file to complete the calibration of the parking auxiliary line, vehicle width auxiliary line, and auxiliary point.

[0063] It should be noted that in step S21, the levelness deviation of the ground is required to be controlled within 5°.

[0064] In step S22, to ensure a clean ground, it is recommended to use red electrical tape or chalk to mark the ground. When marking the longitudinal centerline of the vehicle, place a plumb line at the center of the front and rear of the vehicle, and mark the points where the plumb line touches the ground. Connecting the two marked points marks the longitudinal centerline of the vehicle. At the two widest points of the vehicle, mark two straight lines parallel to the longitudinal centerline; these are the vehicle width lines. A tape measure or laser level can be used for marking.

[0065] The predetermined parking point and the fixed parking point in step S1 are completely consistent in distance and orientation relative to the vehicle. Generally, this is the vertical projection of the vehicle's center of gravity onto the ground, or a point around the vehicle at a certain distance, such as 100cm from the center of the rear of the vehicle. The parking line is a straight line passing through the predetermined parking point and parallel to or coinciding with the vehicle's longitudinal centerline. Specifically, when the predetermined parking point is on the vehicle's longitudinal centerline, the longitudinal centerline and the parking line coincide; when the predetermined parking point is not on the vehicle's longitudinal centerline, a straight line parallel to the longitudinal centerline is drawn through the predetermined parking point marker, which serves as the parking line.

[0066] When marking the vehicle, both the parking line and the width line should extend in both directions along a straight line, exceeding the length of the vehicle body, and at least 200cm beyond the front and rear of the vehicle.

[0067] The designated parking points are precisely marked based on the vehicle's longitudinal centerline and width lines. These designated parking point markers are circular with a cross in the center, the center of which is the designated parking point. The radius is 15-25cm, preferably 20cm. An arrow extends from the designated parking point marker, pointing towards the front of the vehicle and coinciding with the parking line. The arrow's length is 70-100cm, preferably 80cm. If the designated parking point is 100cm from the center of the rear of the vehicle, a designated parking point is marked 100cm outward from the point where a plumb bob touches the center of the rear of the vehicle, along the parking line. The arrow in this case points towards the front of the vehicle, and the arrow's line coincides with the parking line.

[0068] To ensure accuracy and visibility, ground marking lines are typically 1-2cm wide, with 2cm being the preferred width.

[0069] Figure 3Draw a side view diagram for the longitudinal central axis. Figure 4 Draw a top-view diagram showing the vehicle width line and the predetermined parking point 100cm from the center of the rear of the vehicle.

[0070] Specifically, in step S23, firstly, camera sensors are installed on the centerline directly in front of and behind the vehicle, so that the field of view of the front camera is downward and forward, and the field of view of the rear camera is downward and backward. Secondly, if the vehicle's intended parking point and parking line cannot fall within the field of view of the aforementioned cameras, then cameras need to be installed at appropriate locations on the vehicle so that the vehicle's intended parking point and parking line fall within the field of view of those cameras.

[0071] It should be noted that in step S22 above, the extension of the parking line and the vehicle width line is also to ensure that they fall within the field of view of the camera.

[0072] Figure 5 Side view diagram showing the installation of cameras on the centerline at the front and rear of the vehicle. Figure 6 This is a top-down view of the field of view of the cameras on the centerline directly in front of and behind the vehicle.

[0073] In step S24, the camera sensor data from step S23 can be acquired through the V4L2 video acquisition interface to perform image acquisition and display it on the display device in the driver's cab; auxiliary lines and auxiliary points are drawn using the internal drawing functions of QT software.

[0074] Specifically, the calibration interface is as follows: Figure 7 As shown in the diagram, the solid lines represent the parking lines, vehicle width lines, and designated parking spots as seen on the display device. Auxiliary lines and points are drawn near the images of these markings on the display device. The auxiliary points are similar to the designated parking spot markings, being circular with a cross in the center and an arrow pointing outwards. The auxiliary lines and points are drawn as dashed lines, with a line width of 5-10 pixels. The length of the auxiliary lines extends to the edge of the display screen. The radius of the auxiliary points and the length of the arrow lines are adjusted according to the size of the display screen to ensure clear visibility.

[0075] Then, using the directional buttons (up / down / left / right) on the display screen, move the auxiliary lines and auxiliary points (dashed lines) one by one until they completely coincide with the images of the parking lines, vehicle width lines, and predetermined parking points (solid lines) on the ground markings on the display screen. That is, the vehicle width auxiliary lines completely coincide with the vehicle width lines, the parking auxiliary lines completely coincide with the parking lines, the center of the auxiliary points completely coincides with the center of the predetermined parking point markings, and the arrow lines of the auxiliary points coincide with the arrow marking lines extending from the predetermined parking point markings, and the arrow directions of both are the same. At this point, the calibration is considered successful, and the calibration of the parking auxiliary lines, vehicle width auxiliary lines, and auxiliary points is completed.

[0076] In step S24, the calibration parameters are the position coordinate data of the auxiliary lines and auxiliary points on the image.

[0077] It should be noted that an assisted driving system can be configured during step S2 when creating and storing the calibration parameter file. The assisted driving system includes a hardware system and a software system. Specifically, the hardware system includes a video acquisition unit, a display unit, and a main control unit. The video acquisition unit includes several camera sensors, and the display unit includes a display device installed in the driver's cab that does not interfere with driving and is easy to view while driving. The display device's screen supports touch and button functions, allowing the driver to switch modes, perform positioning calibration, and access various configuration functions. The software system includes a Linux operating system, a V4L2 video acquisition framework, and QT software, which are installed and run on the main control unit. Figure 2 This is a schematic diagram of the components of an assisted driving system.

[0078] When setting up an assisted driving system, step S2 also includes the following steps;

[0079] Step S25: Click the save button to save the calibration parameter file, completing the calibration and saving process;

[0080] Step S26: Exit calibration mode.

[0081] It should be noted that when using the assisted driving positioning software for the first time, the auxiliary lines and auxiliary points must first be calibrated and the calibration parameters stored. When the fixed parking point changes, it can be recalibrated. This method supports storing multiple fixed parking point information and selecting any one of them.

[0082] In steps S4 and S5, precise positioning is achieved by combining auxiliary lines and auxiliary points with the fixed parking points marked on the ground. First, the vehicle is moved to the vicinity of the parking point. The driver locates the arrow markings on the ground in the video display and aligns the arrow markings with the direction of the vehicle's movement to determine the vehicle's direction. Next, the vehicle is moved so that the parking auxiliary lines in the video display and the arrow markings on the ground are aligned, determining the vehicle's path. Then, the vehicle is moved closer to the fixed point until the center of the auxiliary point in the video display coincides with the center of the fixed parking point marking on the ground, indicating that the designated fixed point has been reached, and the vehicle is braked in time. The purpose of the vehicle width auxiliary lines is to clearly show people or obstacles in the vehicle's trajectory, aiding in driving safety.

[0083] Specifically, the main control unit of the driver assistance system collects vehicle chassis control information through the CAN communication interface to automatically switch the video on the display device. When a forward gear is detected, the video on the display device automatically switches to the front camera; when a reverse gear is detected, the video on the display device automatically switches to the rear camera.

[0084] This invention enables precise positioning without human intervention. The width of the auxiliary line is 5-10 pixels, and the camera resolution is 720p. Calculated using a scale method, the actual size of each pixel is 0.1cm-0.5cm. Theoretically, the maximum positioning error is 5cm when the parking auxiliary line coincides with the ground marking line. With a vehicle length of at least 10 meters, the directional error of the vehicle's orientation is <0.3°, significantly improving positioning accuracy. Furthermore, the video feed from the vehicle width auxiliary line allows the driver to understand the surrounding environment, enhancing driving safety.

[0085] Example 1

[0086] A precise positioning method for special vehicles based on image vision technology, i.e., a parking method for special vehicles, is proposed. This special transport vehicle is equipped with an auxiliary driving video system, including hardware devices such as camera sensors, a main control unit, and a display device installed in the driver's cab. The hardware system runs a Linux operating system. The main control unit acquires image data through the camera, uses the V4L2 video acquisition framework to obtain the video stream, uses QT library functions to draw on the video screen, and acquires vehicle chassis control information through the CAN communication interface to achieve automatic video switching.

[0087] Precise positioning requirements: The center of the rear of the vehicle must be positioned 100cm from a certain point with an error of less than 10cm and an orientation error of less than 0.5°.

[0088] The fixed location is at the rear of the vehicle, and the driver cannot directly observe the stopping point. The specific implementation steps of the precise positioning method for special vehicles based on image vision technology are as follows:

[0089] Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings.

[0090] The designated parking spot is located 100cm from the center of the rear of the vehicle. The designated parking spot is marked by a circle with a cross in the middle, a radius of 15cm, and the center of the circle is 100cm from the center of the rear of the vehicle to be parked. The arrow points in the direction of the front of the vehicle, and the arrow marking line is 100cm long and 2cm wide.

[0091] Step S2: When using the assisted driving positioning software for the first time, the auxiliary lines and auxiliary points must be calibrated and the calibration parameters must be stored.

[0092] Specifically, the vehicle is parked at a location, and the vehicle width line, the predetermined parking point, and the parking line are marked on the ground. Real-time video information of the ground is collected and pushed to a display device. Then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, the auxiliary lines corresponding to the vehicle width line marked on the ground, the auxiliary points corresponding to the predetermined parking point, and the parking auxiliary lines corresponding to the parking line are marked in the video on the display device to form a layer image. Then, a calibration parameter file is created and stored.

[0093] Detailed steps:

[0094] Step S21: Park the vehicle at a location where the ground level deviation is 5°;

[0095] Step S22: The calibration personnel first place plumb lines at the center of the front and rear of the vehicle. Mark the points where the plumb lines touch with chalk, and connect the two marks with red electrical tape to create the vehicle's longitudinal center line, which is also the parking line. At the two widest points of the vehicle, measure with a tape measure and mark two width lines parallel to the longitudinal centerline with red electrical tape. Extend the parking line and width lines in both directions along a straight line, exceeding the front and rear of the vehicle by 300cm each. Then, measure 100cm outward from the plumb line at the rear of the vehicle along the longitudinal centerline; this is the designated parking point. The designated parking point is marked as a circle with a cross in the center, a radius of 15cm, and the center of the circle is 100cm from the center of the rear of the vehicle to be parked. An arrow pointing from the designated parking point, with the arrow pointing towards the front of the vehicle and coinciding with the parking line, is 100cm long and 2cm wide.

[0096] Step S23: Install cameras (approximately 120cm high) on the centerline directly in front of and behind the vehicle. The front camera is tilted 20° forward and downward, and the rear camera is tilted 20° backward and downward. The vehicle width lines, the predetermined parking points, and the parking lines marked on the ground are within the field of view of the cameras. The display device is installed in the cockpit, and the position of the display device is convenient for the driver to observe.

[0097] Step S24: Start and run the assisted driving software system. Based on the vehicle width line, the predetermined parking point, and the coordinates of the parking line in the video, mark the corresponding vehicle width auxiliary line, auxiliary points, and parking auxiliary line. During calibration, the dashed lines in the video represent the parking auxiliary line and vehicle width auxiliary line to be calibrated. Move the auxiliary lines (dashed lines) one by one using the directional keys (up / down / left / right) until they completely coincide with the parking line and vehicle width line marked on the ground. The calibration is considered successful, and the calibration of the parking auxiliary line and vehicle width auxiliary line is completed. The dashed circular arrow icon in the video represents the auxiliary point. Move the auxiliary point until its center and arrow completely coincide with the predetermined parking point mark on the ground to complete the auxiliary point calibration.

[0098] Step S25: Click the save button to save the calibration parameter file and complete the calibration;

[0099] Step S26: Exit calibration mode;

[0100] Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in Step S2, and simultaneously collect real-time video information from the ground. Push the data to the display device, which simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points, and parking auxiliary lines, as well as the real-time collected ground image. Step S4: Move the vehicle so that the fixed parking point marker is behind the vehicle. The driver relies on the rear camera for guidance. When the vehicle is switched to reverse, the system automatically switches to the rear video view. Locate the arrow marker on the ground in the video view on the display device, align the arrow marker direction with the vehicle's direction, and ensure that the arrow marker line is in a straight line with the parking auxiliary line in the layer image to determine the vehicle's travel path.

[0101] Step S5: Continue moving the vehicle until the auxiliary point in the layer image displayed on the display device coincides with the fixed-point parking mark on the real-time video ground, indicating that the vehicle has reached the designated fixed point position. Brake and complete the fixed-point parking.

[0102] Vehicle width guide lines are used to display people or obstacles on the vehicle's trajectory, helping to improve driving safety.

[0103] Figure 8 This is a schematic diagram of a rear-view fixed-point parking video from Example 1. The video footage in the diagram is captured by the rear-view camera.

[0104] After calculation, the error was 3cm and the orientation error was 0.1°, achieving precise positioning and successful parking.

[0105] Example 2

[0106] A precise positioning method for special vehicles based on image vision technology, i.e., a parking method for special vehicles, is proposed. This special transport vehicle is equipped with an auxiliary driving video system, including hardware devices such as camera sensors, a main control unit, and a display device installed in the driver's cab. The hardware system runs a Linux operating system. The main control unit acquires image data through the camera, uses the V4L2 video acquisition framework to obtain the video stream, uses QT library functions to draw on the video screen, and acquires vehicle chassis control information through the CAN communication interface to achieve automatic video switching.

[0107] Precise positioning requirements: The vehicle's center of gravity must be positioned at a specific point (the vehicle's center of gravity is known and located on the vehicle's longitudinal centerline), with an error requirement of <10cm and an orientation error requirement of <0.5°.

[0108] The fixed location is at the bottom of the vehicle, and the driver and commander cannot directly observe the stopping point. The specific implementation steps of the precise positioning method for special vehicles based on image vision technology are as follows:

[0109] Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings.

[0110] The designated parking spot is the vertical projection of the vehicle's center of gravity onto the ground. The designated parking spot is marked with a circle, with a cross in the center and a radius of 20cm. The center of the circle is the vertical projection of the vehicle's center of gravity onto the ground. The arrow points in the direction of the vehicle's front. The arrow marking line is 80cm long and 1cm wide.

[0111] Step S2: When using the assisted driving positioning software for the first time, the auxiliary lines and auxiliary points must be calibrated and the calibration parameters must be stored.

[0112] Specifically, the vehicle is parked at a location, and the vehicle width line, the predetermined parking point, and the parking line are marked on the ground. Real-time video information of the ground is collected and pushed to a display device. Then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, the auxiliary lines corresponding to the vehicle width line marked on the ground, the auxiliary points corresponding to the predetermined parking point, and the parking auxiliary lines corresponding to the parking line are marked in the video on the display device to form a layer image. Then, a calibration parameter file is created and stored.

[0113] Detailed steps:

[0114] Step S21: Park the vehicle at a location where the ground level deviation is 4°.

[0115] Step S22: The calibration personnel first place plumb lines at the center of the front and rear of the vehicle, marking the points where the plumb lines touch with chalk. Connect the two marks with red electrical tape to create the vehicle's longitudinal centerline, which is also the parking line. At the widest points of the vehicle's lateral direction, measure with a tape measure and mark two width lines parallel to the longitudinal centerline with red electrical tape. Extend the parking line and width lines in both directions along a straight line, exceeding the front and rear of the vehicle by 200cm each. Then, find the vertical projection of the vehicle's center of gravity on the parking line, which is the vehicle's designated parking point. The designated parking point is marked as a circle with a cross in the center, with a radius of 20cm. The center of the circle is the vertical projection of the vehicle's center of gravity onto the ground. An arrow pointing from the designated parking point is aligned with the parking line, pointing in the direction of the vehicle's front. The arrow marking line is 80cm long and 1cm wide.

[0116] Step S23: Install cameras (approximately 120cm high) on the centerline directly in front of and behind the vehicle. The front camera is tilted 20° downwards and forwards, and the rear camera is tilted 20° downwards and backwards. Install a camera 150cm in front of the center of gravity on the longitudinal centerline of the vehicle's bottom. The camera is tilted 20° downwards and backwards. The vehicle width lines, predetermined parking points, and parking lines marked on the ground are within the field of view of the cameras. The display device is installed in the driver's cabin, and its position is convenient for the driver to observe.

[0117] Step S24: Start and run the assisted driving software system, and mark the vehicle width auxiliary line, auxiliary point and parking auxiliary line according to the coordinate position of the vehicle width line, the predetermined parking point and the parking line in the video.

[0118] During calibration, the dashed lines in the video represent the parking and vehicle width auxiliary lines to be calibrated. Use the directional buttons (up / down / left / right) to move the auxiliary lines (dashed lines) one by one until they completely align with the parking and vehicle width lines marked on the ground. Once this is achieved, the calibration is considered successful, and the parking and vehicle width auxiliary lines are calibrated. The dashed circular arrow icons in the video represent auxiliary points. Move these auxiliary points until their centers and arrows completely align with the designated parking point markings on the ground. This completes the auxiliary point calibration.

[0119] Step S25: Click the save button to save the calibration parameter file and complete the calibration;

[0120] Step S26: Exit calibration mode;

[0121] Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in step S2, and at the same time collect real-time video information of the ground and push it to the display device. The display device simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points and parking auxiliary lines, as well as the real-time collected ground image.

[0122] Step S4: Move the vehicle so that the fixed parking point marker is in front of the vehicle. The driver uses the front camera to guide the movement. When the vehicle is switched to forward gear, the system will automatically switch to the front video screen. Locate the arrow marker on the ground in the video screen of the display device, and make the direction of the arrow marker consistent with the direction of the vehicle. The arrow marker line and the parking auxiliary line in the layer image are in a straight line to determine the vehicle's movement path. Then move the vehicle forward so that the fixed parking point marker is below the vehicle. The driver then uses the camera at the bottom of the vehicle to guide the fine-tuning of the movement, so that the arrow marker line and the parking auxiliary line are always in a straight line to determine the vehicle's movement path.

[0123] Step S5: Continue moving the vehicle until the auxiliary point in the layer image displayed on the display device coincides with the fixed-point parking mark on the real-time video ground, indicating that the vehicle has reached the designated fixed point position. Brake and complete the fixed-point parking.

[0124] Vehicle width guide lines are used to display people or obstacles on the vehicle's trajectory, helping to improve driving safety.

[0125] Figure 9 This is a schematic diagram of a center-of-gravity parking video in Example 2. The video footage in the diagram is captured by the front camera and the bottom camera of the vehicle.

[0126] After calculation, the error was 3.6cm and the orientation error was 0.23°, achieving precise positioning and successful parking.

[0127] Example 3

[0128] A precise positioning method for special vehicles based on image vision technology, i.e., a parking method for special vehicles, is proposed. This special transport vehicle is equipped with an auxiliary driving video system, including hardware devices such as camera sensors, a main control unit, and a display device installed in the driver's cab. The hardware system runs a Linux operating system. The main control unit acquires image data through the camera, uses the V4L2 video acquisition framework to obtain the video stream, uses QT library functions to draw on the video screen, and acquires vehicle chassis control information through the CAN communication interface to achieve automatic video switching.

[0129] Precise positioning requirements: The vehicle must be parked with its right side aligned with a specific point (the vehicle must be 30cm from the point, the right front wheel axle must be aligned with the point, and the front of the vehicle must face the direction of the arrow on the marker). The error requirement is <10cm, and the orientation error requirement is <0.5°.

[0130] The fixed location is on the right side of the vehicle, and the driver cannot directly observe the parking point. The specific implementation steps of the precise positioning method for special vehicles based on image vision technology are as follows:

[0131] Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings.

[0132] The designated parking spot is directly opposite the right front wheel axle of the vehicle to be parked, 30cm away from the vehicle. The designated parking spot is marked with a circle, with a cross in the middle, a radius of 25cm, and the center of the circle is directly opposite the right front wheel axle of the vehicle, 30cm away from the vehicle; the arrow points in the direction of the front of the vehicle, the arrow marking line is 70cm long, and the marking line is 1.5cm wide.

[0133] Step S2: When using the assisted driving positioning software for the first time, the auxiliary lines and auxiliary points must be calibrated and the calibration parameters must be stored.

[0134] Specifically, the vehicle is parked at a location, and the vehicle width line, the predetermined parking point, and the parking line are marked on the ground. Real-time video information of the ground is collected and pushed to a display device. Then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, the auxiliary lines corresponding to the vehicle width line marked on the ground, the auxiliary points corresponding to the predetermined parking point, and the parking auxiliary lines corresponding to the parking line are marked in the video on the display device to form a layer image. Then, a calibration parameter file is created and stored.

[0135] Detailed steps:

[0136] Step S21: Park the vehicle at a location where the ground level deviation is 3°.

[0137] Step S22: The calibration personnel first place plumb lines at the center of the front and rear of the vehicle, marking the points where the plumb lines touch with chalk. Connect the two marked points with red electrical tape to create the vehicle's longitudinal centerline. At the widest points of the vehicle's lateral direction, measure with a tape measure and mark two width lines parallel to the longitudinal centerline with red electrical tape. Extend the parking line and width lines in both directions along a straight line, exceeding the front and rear of the vehicle by 400cm each. Then, outside the right-side width line, draw a line parallel to the vehicle's longitudinal centerline and width line, 30cm away from the right-side width line. This is the parking line. Find the point on the parking line directly opposite the center of the right front wheel axle; this is the vehicle's designated parking point. The designated parking spot is marked with a circle, a cross in the center, a radius of 25cm, and the center of the circle is directly opposite the right front wheel axle of the vehicle, 30cm away from the vehicle; an arrow pointing from the designated parking spot is in the direction of the front of the vehicle and coincides with the parking line, the arrow line is 70cm long, and the line width is 1.5cm;

[0138] Step S23: Install cameras (approximately 120cm high) on the centerline directly in front of and behind the vehicle. The front camera is tilted 20° downwards and forwards, and the rear camera is tilted 20° downwards and backwards. Install a wide-angle camera above the right front wheel axle, with the camera facing to the right and tilted downwards at 30°. The vehicle width lines, the designated parking point, and the parking line marked on the ground should be within the camera's field of view. The display device is installed inside the driver's cabin, positioned for easy observation by the driver.

[0139] Step S24: Start and run the assisted driving software system, and mark the vehicle width auxiliary line, auxiliary point and parking auxiliary line according to the coordinate position of the vehicle width line, the predetermined parking point and the parking line in the video.

[0140] During calibration, the dashed lines in the video represent the parking and vehicle width auxiliary lines to be calibrated. Use the directional buttons (up / down / left / right) to move the auxiliary lines (dashed lines) one by one until they completely align with the parking and vehicle width lines marked on the ground. Once this is achieved, the calibration is considered successful, and the parking and vehicle width auxiliary lines are calibrated. The dashed circular arrow icons in the video represent auxiliary points. Move these auxiliary points until their centers and arrows completely align with the predetermined parking point markings on the ground. This completes the auxiliary point calibration.

[0141] Step S25: Click the save button to save the calibration parameter file and complete the calibration;

[0142] Step S26: Exit calibration mode;

[0143] Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in step S2, and at the same time collect real-time video information of the ground and push it to the display device. The display device simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points and parking auxiliary lines, as well as the real-time collected ground image.

[0144] Step S4: Move the vehicle so that the designated parking point marker is on the right side of the vehicle. The driver uses the camera on the right side of the vehicle for guidance. When a forward gear is detected, the video display automatically switches to the front camera; when a reverse gear is detected, the video display automatically switches to the rear camera. Locate the arrow marker on the ground in the video display and ensure that the arrow marker's direction is consistent with the direction the vehicle is heading. The arrow marker line should be aligned with the parking auxiliary line in the layer image to determine the vehicle's travel path.

[0145] Step S5: Continue moving the vehicle until the auxiliary point in the layer image displayed on the display device coincides with the fixed-point parking mark on the real-time video ground, indicating that the vehicle has reached the designated fixed point position. Brake and complete the fixed-point parking.

[0146] Vehicle width guide lines are used to display people or obstacles on the vehicle's trajectory, helping to improve driving safety.

[0147] Figure 10 This is a schematic diagram of the side-to-side fixed-point parking video footage of Example 3. The video footage in the figure is captured by the front camera and the right-side camera of the vehicle.

[0148] After calculation, the error was 4cm and the orientation error was 0.2°, achieving precise positioning and successful parking.

[0149] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An image vision technology-based assisted driving precision positioning method, characterized in that, Includes the following steps: Step S1: Set up ground markings in the planned parking area. The ground markings include fixed parking spot markings and arrow markings extending from the fixed parking spot markings. Step S2: Park the vehicle at any location, mark the vehicle width line, the predetermined parking point, and the parking line on the ground, collect real-time video information from the ground, push it to the display device, and then, based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line marked on the ground in the video on the display device, mark the auxiliary line of the vehicle width line marked on the ground, the auxiliary point corresponding to the predetermined parking point, and the parking auxiliary line corresponding to the parking line in the video on the display device, form a layer image, and save it as a calibration parameter file; Step S3: Drive the vehicle to the vicinity of the planned parking area, run the calibration parameter file in step S2, and at the same time collect real-time video images of the ground and push them to the display device. The display device simultaneously displays the layer image with the vehicle width auxiliary lines, auxiliary points and parking auxiliary lines, as well as the real-time video images of the ground. Step S4: Move the vehicle, locate the ground markings set in the planned parking area in the real-time video image, and make the arrow markings align with the direction of the vehicle's front, and ensure that the arrow markings are in a straight line with the parking auxiliary lines in the layer image to determine the vehicle's travel path. Step S5: Continue moving the vehicle until the auxiliary points in the layer image displayed on the display device coincide with the fixed-point parking location markers in the real-time ground video image, thus completing the fixed-point parking.

2. The method according to claim 1, characterized in that, In step S1, the fixed parking point is the vertical projection of the vehicle's center of gravity onto the ground, or a point around the vehicle at a certain distance from the vehicle.

3. The method according to claim 1, characterized in that, In step S1, the fixed parking spot marker is circular with a cross in the middle, the center of the circle being the fixed parking spot, and the radius is 15-25cm; the arrow marker extending from the fixed parking spot marker points in the direction of the vehicle's front, the arrow marker line is 70-100cm long, and the marker line is 1-2cm wide.

4. The method according to claim 3, characterized in that, The radius of the fixed parking spot marker is preferably 20cm; the length of the arrow marker line is preferably 80cm; and the width of the marker line is preferably 2cm.

5. The method according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Park the vehicle at any location; Step S22: First, mark the longitudinal centerline and width line of the vehicle. Then, accurately mark the predetermined parking point and parking line of the vehicle. The predetermined parking point and the fixed parking point in step S1 are completely consistent with the distance and orientation of the vehicle. Step S23: Install a camera sensor on the vehicle so that the vehicle width line, the predetermined parking point and the parking line marked on the ground are within the field of view of the camera. Step S24: Based on the coordinate positions of the vehicle width line, the predetermined parking point, and the parking line in the video image captured by the camera sensor, the vehicle width auxiliary line, auxiliary point, and parking auxiliary line are calibrated accordingly, forming a layer image and storing it as a calibration parameter file, thus completing the calibration of the parking auxiliary line, vehicle width auxiliary line, and auxiliary point.

6. The method according to claim 5, characterized in that, In step S21, the ground level deviation of the vehicle parking location is controlled within 5°.

7. The method according to claim 5, characterized in that, In step S22, when marking the longitudinal centerline of the vehicle, a plumb line is placed at the center of the front and rear of the vehicle, and a mark is made at the point where the plumb line touches the ground. The two mark points are connected to form the longitudinal centerline of the vehicle. When marking the width lines of a vehicle, two straight lines parallel to the longitudinal centerline are marked at the two ends where the vehicle is widest laterally. The parking line is a straight line that passes through the predetermined parking point and is parallel to or coincides with the longitudinal centerline; when the predetermined parking point is on the longitudinal centerline, the parking line coincides with the longitudinal centerline; when the predetermined parking point is not on the longitudinal centerline, the parking line is parallel to the longitudinal centerline.

8. The method according to claim 7, characterized in that, Both the parking line and the vehicle width line must extend in both directions along a straight line, exceeding the length of the vehicle body.

9. The method according to claim 8, characterized in that, The parking line and the vehicle width line extend at least 200cm beyond the front and rear of the vehicle, respectively.

10. The method according to claim 5, characterized in that, In step S23, camera sensors are installed on the centerline directly in front of and behind the vehicle, such that the field of view of the camera sensor directly in front is forward and downward, and the field of view of the camera sensor directly behind is backward and downward. If the vehicle's intended parking point and parking line cannot fall within the field of view of the above two camera sensors, a camera sensor is installed at an appropriate location on the vehicle so that the vehicle's intended parking point and parking line fall within the field of view of the camera sensor.

Images