A vehicle deviation detection method and system based on vision

Abstract

The invention discloses a method and system for detecting vehicle deviation based on vision. The method comprises: setting lane lines, equidistant positioning lines, visual field positioning zebra crossings and reference lines on a test section; receiving an original image of the test section without a vehicle acquired by an image acquisition device and performing distortion correction to obtain an original corrected image; locating zebra crossings according to the visual field in the original corrected image, extracting an area of ​​interest, and drawing virtual equidistant reference lines in the area of ​​interest according to the equidistant positioning lines and the reference lines, and marking coordinates of the virtual equidistant reference lines; receiving a test image of a test vehicle acquired by an image acquisition device and performing distortion correction to obtain a test corrected image; adding the virtual equidistant reference lines to the test corrected image, identifying a wheel pressure point in the test corrected image, calculating the position coordinates of the wheel pressure point in the virtual equidistant reference lines, and determining the deviation amount of the test vehicle; the method can improve the accuracy of vehicle deviation detection.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vision-based vehicle deviation detection method and system. Background Art

[0002] In the autonomous driving system, lane keeping is one of the important functions to ensure the safe driving of the vehicle. The lane keeping function ensures that the vehicle stays in the correct lane by continuously monitoring the position of the vehicle in the lane, and prevents traffic accidents caused by the vehicle deviating from the lane. Especially on highways or complex urban roads, the lane keeping function can significantly reduce the risk of accidents caused by driver distraction or fatigue driving. In long-distance driving or busy traffic environments, the lane keeping function can help drivers reduce the pressure of continuous vehicle control, reduce fatigue, and make the driving process easier and safer. This is crucial to improve driving comfort and reduce the accident rate. The lane keeping function is one of the basic capabilities required to achieve advanced autonomous driving. Through the stable operation of the lane keeping function, the autonomous driving system can handle more complex driving tasks, such as lane changing, turning and obstacle avoidance, laying the foundation for fully autonomous driving.

[0003] Lane keeping is of great significance in autonomous driving. However, if lane keeping has a large deviation, it will cause harm to the car and the driver. Therefore, cars with autonomous driving functions need to undergo lane keeping deviation tests before leaving the factory. Deviation testing refers to testing whether the vehicle can effectively keep the lane under various conditions and correct potential deviations in a simulated or real environment. Through deviation testing, the reliability of the autonomous driving system under different driving conditions can be verified, ensuring that the system can accurately identify lane markings and take corrective measures in time when the vehicle deviates from the lane to prevent accidents. Deviation testing can reveal the possible deficiencies of the autonomous driving system under specific conditions (such as complex road conditions, severe weather conditions or blurred road markings). By analyzing the test results, engineers can optimize the algorithm, improve the robustness and response speed of the system, and enhance the adaptability of autonomous driving vehicles in various environments. Through rigorous deviation testing, manufacturers can ensure that their autonomous driving vehicles perform stably and reliably in various real-life scenarios. This not only improves the safety of the vehicle, but also enhances users' trust in autonomous driving technology, thereby promoting the widespread application of this technology. The data and results of deviation testing provide an important basis for the formulation of safety standards and regulations for autonomous vehicles. This will help form industry consensus, promote the standardized development of autonomous driving technology, and ensure that all autonomous vehicles on the road meet the necessary safety requirements.

[0004] Patent document CN115471801A proposes a method, device, computer equipment and storage medium for measuring the deviation of a vehicle. The method comprises: obtaining a first environmental image of the vehicle at a first moment, and obtaining a first distance between the vehicle and the center line of the road according to the first environmental image; obtaining a second environmental image of the vehicle at a second moment, and obtaining a second distance between the vehicle and the center line of the road according to the second environmental image; obtaining the deviation distance of the vehicle according to the first distance and the second distance; obtaining the longitudinal distance of the vehicle, and obtaining the deviation angle according to the longitudinal distance and the deviation distance. The method requires an image acquisition device to be set on the vehicle. However, in practical applications, it is difficult to set an image acquisition device on the vehicle so that the field of view of the image acquisition device covers the wheels and the markings on the road. Moreover, when the image acquisition device is displaced due to bumps, the detection result is bound to be inaccurate. In addition, the road slope will also affect the detection result, and the image acquisition device needs to be disassembled and assembled for each test, resulting in low detection efficiency.

[0005] Patent document CN101187595A discloses an online automatic detection system for vehicle deviation. On the test section where the vehicle deviates, multiple test devices are set up, which are composed of a gantry, an image sensor and a sensor controller installed in the middle of the gantry, a group of opposing photoelectric switches and a detection control room. The opposing photoelectric switch is composed of a light emitting tube and a photoelectric tube. The detection control room is equipped with a computer, which is connected to the image sensor via optical fiber Ethernet. The image sensor controller is also connected to the photoelectric tube via a signal line. In this scheme, the gantry, sensor, controller, photoelectric switch and other devices have a complex structure and require high installation accuracy, which is not very conducive to the replication of the system. If more test points are set, the cost is high. If fewer test points are used, only a small amount of deviation data can be obtained. Summary of the invention

[0006] The present invention provides a vehicle deviation detection method and system based on vision, which can improve the accuracy of detection results, have high detection efficiency and low production cost.

[0007] A vehicle deviation detection method based on vision, comprising:

[0008] Set lane lines, equidistant positioning lines, visual field positioning zebra crossings and reference lines on the test road section;

[0009] At least one image acquisition device is arranged on the test road section, and the field of view of the image acquisition device covers the lane lines, equidistant positioning lines, field of view positioning zebra crossings and reference lines;

[0010] Receiving the original image of the test road section without a vehicle captured by the image acquisition device and performing distortion correction to obtain an original corrected image;

[0011] Positioning the zebra crossing according to the field of view in the original corrected image, extracting the region of interest, and drawing virtual equidistant reference lines in the region of interest according to the equidistant positioning lines and reference lines, and marking coordinates of the virtual equidistant reference lines;

[0012] Receiving the test image of the test vehicle acquired by the image acquisition device and performing distortion correction to obtain a test corrected image;

[0013] The virtual equidistant reference line is added to the test correction image, and the wheel pressure point in the test correction image is identified, and the position coordinates of the wheel pressure point in the virtual equidistant reference line are calculated to determine the deviation amount of the test vehicle.

[0014] Furthermore, the lane lines are two solid lines, which are parallel to the length direction of the test section and symmetrically distributed along the center line of the length direction of the test section, and the test vehicle travels within the two lane lines;

[0015] The equidistant positioning lines are two dotted lines, which are parallel to the length direction of the test section, and are respectively arranged on the outside of the two lane lines and at equal distances from the lane lines. The length of the line segments in the equidistant positioning lines is equal to the distance between the line segments, and the line segments in the two equidistant positioning lines are aligned one by one.

[0016] Furthermore, a camera positioning area is provided on the test section, and the image acquisition device is installed in the camera positioning area.

[0017] Furthermore, the number of the field of vision positioning zebra crossings is n+1 groups, where n is the number of image acquisition devices, and each group of the field of vision positioning zebra crossings includes four solid lines of fixed length, which are parallel to the length direction of the test section and are respectively arranged outside the two equidistant positioning lines, and two field of vision positioning zebra crossings are arranged on each side, and are symmetrically distributed along the center line of the length direction of the test section, and the distance between each two adjacent groups of field of vision positioning zebra crossings is equal; the end of the field of vision positioning zebra crossing close to the starting point of the test vehicle is defined as the head end, and the end away from the starting point of the test vehicle is defined as the tail end, and the head end is aligned with the endpoint of the line segment in the equidistant positioning line, and in two adjacent groups of field of vision positioning zebra crossings, the distance between the head end of one group of field of vision positioning zebra crossings to the head end of the other group of field of vision positioning zebra crossings is an integer multiple of the length of the line segment in the equidistant positioning line;

[0018] The field of view of each image acquisition device includes two sets of field of view positioning zebra crossings;

[0019] The zebra crossing is located according to the field of view in the original rectified image, and the region of interest is extracted, including:

[0020] Two groups of visual field positioning zebra crossings in the original corrected image are identified, and an area between a head end of a first group of visual field positioning zebra crossings close to a starting point of a test vehicle and a head end of a second group of visual field positioning zebra crossings far from a starting point of the test vehicle is determined as an area of ​​interest.

[0021] Further, the reference line is a solid line arranged at intervals of different colors, the solid line segments of each color have the same length, and the reference line is perpendicular to the equidistant positioning line;

[0022] The number of the reference lines is 2*n, where n is the number of image acquisition devices, different reference lines are spaced by a preset distance, the two ends of the reference line extend beyond the two equidistant positioning lines respectively, and the field of view of each image acquisition device includes two reference lines;

[0023] Drawing a virtual equidistant reference line in the region of interest according to the equidistant positioning line and the reference line comprises:

[0024] Identifying solid line segments of different colors in the reference line;

[0025] Draw a plurality of horizontal equidistant reference lines, so that the projection lines of the horizontal equidistant reference lines on the actual test section are perpendicular to the lane lines, and the spacing between the projection lines of the plurality of horizontal equidistant reference lines on the actual test section is equal to the length of the line segment in the equidistant positioning line;

[0026] Draw a plurality of vertical equidistant reference lines, so that the projection lines of the vertical equidistant reference lines on the actual test road section are parallel to the lane lines, and the spacing between the projection lines of the plurality of vertical equidistant reference lines on the actual test road section is equal to the length of a single color solid line segment in the reference line;

[0027] A plurality of the horizontal equidistant reference lines and the vertical equidistant reference lines constitute the virtual equidistant reference lines.

[0028] Further, marking coordinates of the virtual equidistant reference lines includes:

[0029] In the grid formed by the plurality of horizontal equidistant reference lines and the vertical equidistant reference lines, the upper left vertex of the upper leftmost grid is selected as the coordinate origin;

[0030] Determine the width and height of each grid according to the length of the solid line segment in the reference line and the length of the line segment in the equidistant positioning line;

[0031] According to the coordinate origin, the width and height of each grid, the coordinates of the upper left vertex of each grid are calculated and marked.

[0032] Further, calculating the position coordinates of the wheel pressure point on the virtual equidistant reference line to determine the deviation amount of the test vehicle includes:

[0033] Calculate a first distance from the left side of the grid where the wheel pressing point is located, and a second distance from the upper side of the grid where the wheel pressing point is located;

[0034] Calculating the position coordinates of the wheel pressing point according to the coordinates of the upper left vertex of the grid where the wheel pressing point is located and the first distance and the second distance;

[0035] The position coordinates of multiple wheel pressure points are obtained, and a mileage-offset function curve of the test vehicle is drawn. The deviation amount of the test vehicle is determined according to the mileage-offset function curve.

[0036] Further, the position coordinates of the wheel pressure point are (X, Y), wherein X=w+w0, Y=h+h0, (w, h) are the coordinates of the upper left vertex of the grid where the wheel pressure point is located, w0 is the first distance, and h0 is the second distance;

[0037] The position coordinates of the multiple wheel pressure points are (X_0, Y_0), (X_1, Y_1), ... (X_ num-1 ), Y_ num-1 ), where num is the number of wheel pressure points;

[0038] The first wheel rolling point position coordinates (X_0, Y_0) as the starting point, with (Y_ i -Y_0) is the horizontal coordinate, (X_ i -X_0) is the vertical coordinate, and the mileage-offset function curve is drawn, where i=1,2,3,...num-1.

[0039] Furthermore, the number of image acquisition devices on the test section is two or more, and the number of the camera positioning areas is the same as the number of the image acquisition devices;

[0040] The camera positioning area is arranged at one side of the test section and is located in front of the corresponding field of view positioning zebra crossing, and the distances between adjacent camera positioning areas are equal.

[0041] A vision-based vehicle deviation detection system includes a test road section, at least one image acquisition device, a processor, and a storage device;

[0042] The storage device stores a plurality of instructions, and the processor is used to read the instructions and execute the above method.

[0043] The vision-based vehicle deviation detection method and system provided by the present invention have at least the following beneficial effects:

[0044] (1) An external image acquisition device is used to obtain images of the test vehicle and the test road section to calculate the deviation of the vehicle. Since the deviation is calculated on the outside of the vehicle, it is objective and intuitive. Compared with the case where the camera is installed on the vehicle, the influence of the vehicle's own occlusion, bumpy displacement and road slope on the deviation calculation is avoided, thereby improving the accuracy of the test results;

[0045] (2) After the system of the present invention is debugged, the deviation of any vehicle can be tested. There is no need to replace the equipment or move the camera position every time the test vehicle is changed. The operation is simple, especially when it is necessary to measure multiple vehicles of different models, which can improve the detection efficiency.

[0046] (3) The system of the present invention consists of only three parts: a test section, an image acquisition device, and a computer, which reduces the cumulative error caused by too many devices. In addition, the equipment cost is low, which effectively reduces the production cost. The creation of the test section and the installation of the image acquisition device are simple, the calculation process is easy to understand, and the entire system is easy to reproduce.

[0047] (4) The vehicle can quickly calculate the deviation of the vehicle at multiple locations while driving on the test road, which makes up for the deficiency that most deviation detection systems can only test the deviation at a small number of locations, and provides a basis for subsequent result analysis and evaluation of autonomous driving performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The present invention provides a flow chart of an embodiment of a method for detecting vehicle deviation based on vision.

[0049] Figure 2 A schematic diagram of an embodiment of a test section in the vision-based vehicle deviation detection method provided by the present invention.

[0050] Figure 3 for Figure 2 Enlarged view of part A.

[0051] Figure 4 A schematic diagram of an embodiment of the relationship between the lengths of the equidistant positioning line and the visual field positioning zebra crossing in the vision-based vehicle deviation detection method provided by the present invention.

[0052] Figure 5 A schematic diagram of an embodiment of the field of view coverage of the image acquisition device in the vision-based vehicle deviation detection method provided by the present invention.

[0053] Figure 6 A schematic diagram of an embodiment of a reference line in the vision-based vehicle deviation detection method provided by the present invention.

[0054] Figure 7A schematic diagram of an embodiment of calculating the coordinates of the wheel pressure point in the vision-based vehicle deviation detection method provided by the present invention.

[0055] Figure 8 A schematic diagram of a test road section in an application scenario of the vision-based vehicle deviation detection method provided by the present invention.

[0056] Fig. 9 A schematic diagram of a mileage-offset function curve obtained in an application scenario of the vision-based vehicle deviation detection method provided by the present invention.

[0057] Fig.10 A schematic diagram of an embodiment of a vision-based vehicle deviation detection device provided by the invention. DETAILED DESCRIPTION

[0058] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0059] refer to Figure 1 In some embodiments, a method for detecting vehicle deviation based on vision is provided, comprising:

[0060] S1. Set lane lines, equidistant positioning lines, visual field positioning zebra crossings and reference lines on the test road section;

[0061] S2, setting at least one image acquisition device on the test section, and making the field of view of the image acquisition device cover the lane lines, equidistant positioning lines, field of view positioning zebra crossings and reference lines;

[0062] S3, receiving the original image of the test road section without a vehicle acquired by the image acquisition device and performing distortion correction to obtain an original corrected image;

[0063] S4, locating the zebra crossing according to the field of view in the original corrected image, extracting the region of interest, and drawing virtual equidistant reference lines in the region of interest according to the equidistant positioning lines and reference lines, and marking the coordinates of the virtual equidistant reference lines;

[0064] S5, receiving the test image of the test vehicle acquired by the image acquisition device and performing distortion correction to obtain a test corrected image;

[0065] S6. Add the virtual equidistant reference line to the test correction image, identify the wheel pressure point in the test correction image, calculate the position coordinates of the wheel pressure point in the virtual equidistant reference line, and determine the deviation amount of the test vehicle.

[0066] Specifically, in step S1, refer to Figure 2 and Figure 3The lane lines Q are two solid lines, which are parallel to the length direction of the test section P, symmetrically distributed along the center line of the length direction of the test section P, and set at the center of the test section P. The test vehicle S travels within the two lane lines Q.

[0067] The distance between the two lane lines Q should be close to the distance between the left and right wheels of the test vehicle S, and should not be too small or too large. The lane line Q serves as a reference line for the test vehicle during autonomous driving. If the test vehicle deviates from the lane line Q during driving, the control algorithm will be activated to adjust the forward direction.

[0068] The equidistant positioning lines L are two dotted lines, which are parallel to the length direction of the test section P. They are respectively set on the outside of the two lane lines Q and are at equal distances from the lane lines Q. The length of the line segments in the equidistant positioning lines L is equal to the distance between the line segments, and the line segments in the two equidistant positioning lines L are aligned one by one. The function of the equidistant positioning lines is to serve as the basis for the direction of the lane lines when drawing the equidistant reference lines, as well as the reference for the reference lines that are attached later.

[0069] refer to Figure 4 , assuming that the length of the line segment in the equidistant positioning line L is y, the distance between adjacent line segments is also y.

[0070] Further, refer to Figures 2 to 4 A camera positioning area D is also provided on the test section P, and the image acquisition device is installed in the camera positioning area D.

[0071] Furthermore, in step S2, at least one image acquisition device is set on the test section. If there are multiple image acquisition devices, each image acquisition device acquires an image of part of the test section. Since each image acquisition device acquires an image corresponding to its own test section, the image includes an overlapping portion of the test section corresponding to itself and the test section corresponding to an adjacent image acquisition device. Since the maximum range of deviation of the test vehicle will not exceed the two equidistant positioning lines, the reference line only needs to cover the area between the two equidistant positioning lines and does not need to cover the entire image.

[0072] Furthermore, in step S3, an original image of the test road section without a vehicle captured by the image acquisition device is received, and distortion correction is performed on the original image to obtain an original corrected image.

[0073] Since the image acquisition device shoots from near to far, the lane lines in the distance of the image will be severely distorted. At this time, it is necessary to use the existing correction function to correct the radial distortion and tangential distortion of the image to ensure that the equidistant positioning lines appear as two straight lines on the image.

[0074] Further, in some embodiments, the number of the visual field positioning zebra crossings is n+1 groups, where n is the number of image acquisition devices;

[0075] refer to Figure 3 Each group of visual field positioning zebra crossings W includes four solid lines of fixed length, which are parallel to the length direction of the test section P and are respectively arranged outside the two equidistant positioning lines L. Two visual field positioning zebra crossings are arranged on each side and are symmetrically distributed along the center line of the length direction of the test section P. The distance between each two adjacent groups of visual field positioning zebra crossings is equal; the end of the visual field positioning zebra crossing close to the starting point of the test vehicle is defined as the head end, and the end far from the starting point of the test vehicle is defined as the tail end. The head end is aligned with the endpoint of the line segment in the equidistant positioning line. In two adjacent groups of visual field positioning zebra crossings, the distance between the head end of one group of visual field positioning zebra crossings and the head end of the other group of visual field positioning zebra crossings is an integer multiple of the length of the line segment in the equidistant positioning line.

[0076] refer to Figure 4 In some embodiments, each group of visual field positioning zebra crossings W should be aligned with each other along the lane line direction. In the length direction of the test section, the distance between each group of visual field positioning zebra crossings W is equal. In two adjacent groups of visual field positioning zebra crossings, the distance rangeY between the head end of one group of visual field positioning zebra crossings and the head end of the other group of visual field positioning zebra crossings is an integer multiple of the line segment length y in the equidistant positioning line, that is, rangeY=a*y, where a is an integer. In order to facilitate the calculation of the position of the test vehicle on the image corresponding to the lane line direction in the actual scene, the head end of each group of visual field positioning zebra crossings W should be aligned with the end point of the line segment on the equidistant positioning line L as much as possible.

[0077] The camera positioning area D is set on one side of the test section P and is located in front of the corresponding field of view positioning zebra crossing W. The distances between adjacent camera positioning areas D are equal. The field of view positioning zebra crossing serves as a reference for the corresponding image acquisition device to adjust the field of view.

[0078] The "front" and "rear" mentioned in this embodiment are based on the direction in which the test vehicle is traveling.

[0079] refer to Figure 5 The image acquisition device is installed in the camera positioning area D, and the field of view of the image acquisition device is adjusted so that the field of view of the image acquisition device covers the field of view positioning zebra crossings on both sides. On this basis, the field of view of the image acquisition device covers the lane line Q, the equidistant positioning line L and the reference line H.

[0080] In some embodiments, the image acquisition device can be installed in the camera positioning area D by means of a bracket, and the field of view of the image acquisition device can be adjusted by adjusting the height and angle of the image acquisition device by means of the bracket. If there are multiple image acquisition devices, for each image acquisition device, the lane line area between the head end of a certain field of view positioning zebra crossing and the head end of an adjacent field of view positioning zebra crossing in front of the field of view positioning zebra crossing is regarded as its corresponding test section. If there are n image acquisition devices, the entire test section is divided into n corresponding test sections.

[0081] The field of view of each image acquisition device includes two sets of field of view positioning zebra crossings.

[0082] Specifically, in step S4, the zebra crossing is located according to the field of view in the original rectified image, and the region of interest is extracted, including:

[0083] Two groups of visual field positioning zebra crossings in the original corrected image are identified, and an area between a head end of a first group of visual field positioning zebra crossings close to a starting point of a test vehicle and a head end of a second group of visual field positioning zebra crossings far from a starting point of the test vehicle is determined as an area of ​​interest.

[0084] In some embodiments, an existing neural network model may be used to identify the visual field positioning zebra crossings in the original rectified image, and the area enclosed by them is the region of interest.

[0085] Further, refer to Figure 6 , the reference line H is a solid line set at different color intervals, the length of the solid line segment of each color is the same, all are x, and the reference line H is perpendicular to the equidistant positioning line L.

[0086] The number of reference lines is 2*n, where n is the number of image acquisition devices, different reference lines H are spaced at a preset distance, both ends of the reference line H extend beyond the two equidistant positioning lines, and the field of view of each image acquisition device includes two reference lines;

[0087] In some embodiments, the reference line H may be composed of a plurality of red and white stripes of length equal to x arranged alternately, wherein the total length of the red and white stripes should be equal and greater than the distance between the two equidistant positioning lines L, and the width may be appropriate. The reference line H needs to be perpendicular to the lane line Q, both ends of the reference line H need to be located outside the two equidistant positioning lines L, and each reference line H needs to be aligned at both ends, so as to ensure that the red and white stripes at the same position on each reference line are aligned.

[0088] In step S4, drawing a virtual equidistant reference line in the region of interest according to the equidistant positioning line and the reference line includes:

[0089] Identifying solid line segments of different colors in the reference line;

[0090] Draw a plurality of horizontal equidistant reference lines, so that the projection lines of the horizontal equidistant reference lines on the actual test section are perpendicular to the lane lines, and the spacing between the projection lines of the plurality of horizontal equidistant reference lines on the actual test section is equal to the length of the line segment in the equidistant positioning line, that is, y;

[0091] Draw a plurality of vertical equidistant reference lines, so that the projection lines of the vertical equidistant reference lines on the actual test section are parallel to the lane lines, and the spacing between the projection lines of the plurality of vertical equidistant reference lines on the actual test section is equal to the length of a single color solid line segment in the reference line, that is, x;

[0092] A plurality of the horizontal equidistant reference lines and the vertical equidistant reference lines constitute the virtual equidistant reference lines.

[0093] Specifically, assuming that the number of solid line segments of different colors in the reference line is b, the number of vertical equidistant reference lines is b+1, and the number of horizontal equidistant reference lines is a+1. Affected by the angle and height of the image acquisition device, in the application scenario of multiple image acquisition devices, the angles and lengths of the equidistant reference lines within the field of view of different image acquisition devices are different, but the vertical equidistant reference lines at each corresponding position of all image acquisition devices are on the same straight line on the actual road surface, and each horizontal equidistant reference line is parallel to each other and equidistant on the road surface of the actual test section. In addition, since the objects on the image appear to be small in the distance and large in the near, the multiple horizontal equidistant reference lines / vertical equidistant reference lines on the image are not necessarily parallel, so it is required that the projection line of the horizontal equidistant reference line on the actual test section is perpendicular to the lane line, and the projection line of the vertical equidistant reference line on the actual test section is parallel to the lane line. The region of interest on the current image is divided into a*b grids, each grid is x wide and y high.

[0094] Furthermore, in step S4, marking coordinates of the virtual equidistant reference lines includes:

[0095] In the grid formed by the plurality of horizontal equidistant reference lines and the vertical equidistant reference lines, the upper left vertex of the upper leftmost grid is selected as the coordinate origin;

[0096] Determine the width and height of each grid according to the length of the solid line segment in the reference line and the length of the line segment in the equidistant positioning line;

[0097] According to the coordinate origin, the width and height of each grid, the coordinates of the upper left vertex of each grid are calculated and marked.

[0098] In some embodiments, each grid is x wide and y high. Assuming there are n image acquisition devices, the original rectified images corresponding to each image acquisition device are image_i (i=0, 1, 2, ..., n-1). From the upper left to the lower right, the coordinates of the upper left vertex of each grid should be:

[0099] (a*i*x,0), (a*i*x,y) ... (a*i*x, y*(b-1))

[0100] ((a*i+1)*x,0), ((a*i+1)*x,y) ... ((a*i+1)*x, y*(b-1)) ...

[0102] ((a*(i+1)-1)*x,0), ((a*(i+1)-1)*x,y) ... ((a*(i+1)-1)*x, y*(b-1)).

[0103] Furthermore, in step S5, a test image of the test vehicle captured by the image acquisition device is received, and distortion correction is performed on the test image to obtain a test corrected image. In some embodiments, radial distortion and tangential distortion are corrected using an existing correction function to ensure that the equidistant positioning lines appear as two straight lines on the image.

[0104] Furthermore, in step S6, the virtual equidistant reference lines are added to the test corrected image, and the wheel pressure point in the test corrected image is identified. In some embodiments, an existing neural network model can be used to identify the wheel of the test vehicle in the test corrected image, preferably the right front wheel of the test vehicle. The right front wheel is framed and the center of the lower edge of the identification frame is determined as the wheel pressure point.

[0105] Further, in step S6, the position coordinates of the wheel pressure point on the virtual equidistant reference line are calculated to determine the deviation amount of the test vehicle, including:

[0106] Calculate a first distance from the left side of the grid where the wheel pressing point is located, and a second distance from the upper side of the grid where the wheel pressing point is located;

[0107] Calculating the position coordinates of the wheel pressing point according to the coordinates of the upper left vertex of the grid where the wheel pressing point is located and the first distance and the second distance;

[0108] The position coordinates of multiple wheel pressure points are obtained, and a mileage-offset function curve of the test vehicle is drawn. The deviation amount of the test vehicle is determined according to the mileage-offset function curve.

[0109] Specifically, refer to Figure 7 , each grid can be approximated into a parallelogram. Draw parallel lines between the upper and lower sides of the grid at the wheel pressure point. Assume that the first distance between the wheel pressure point and the left side is w0, the second distance between the wheel pressure point and the upper side is h0, and the coordinates of the upper left point of the grid where the wheel pressure point is located are (w, h), then the position coordinates (X, Y) of the current wheel pressure point are recorded as X=w+w0, Y=h+h0.

[0110] The position coordinates of the multiple wheel pressure points are (X_0, Y_0), (X_1, Y_1), ... (X_ num-1 ), Y_ num-1 ), where num is the number of wheel pressure points;

[0111] The first wheel rolling point position coordinates (X_0, Y_0) as the starting point, with (Y_ i -Y_0) is the horizontal coordinate, (X_ i -X_0) is the vertical coordinate, and the mileage-offset function curve is drawn, where i=1,2,3,...num-1.

[0112] Let the maximum offset of the test vehicle during the test be offsetMax, then:

[0113] offsetMax=|max(X_0,X_1,...X_ (num-1) )-X_0|.

[0114] The method provided in this embodiment is further described below through specific application scenarios.

[0115] refer to Figure 8 , Assume that a vehicle deviation test system has 6 camera positioning areas, 7 sets of field of view positioning zebra crossings, and the total length of the test section is 240 meters. In the field of view of an image acquisition device, the distance rangeY between the head end of a set of field of view positioning zebra crossings to the head end of another set of field of view positioning zebra crossings is 40 meters. The length of the line segments in the equidistant positioning line and the distance between the line segments are both 2.5 meters. The distance between the two equidistant positioning lines is 16 meters. The reference line is composed of red solid line segments and white solid line segments set at intervals. The lengths of the red solid line segments and the white solid line segments of the reference line are both 2 meters.

[0116] Assume that the coordinates of the upper left vertex of the grid on the virtual equidistant reference line drawn in the region of interest corresponding to the six image acquisition devices are as follows:

[0117] The coordinates of the upper left points of all grids in the 0th image are:

[0118] (0,0), (0,2) ... (0, 14)

[0119] (2.5,0), (2.5,2) ... (2.5, 14) ...

[0121] (37.5,0), (37.5,2) ... (37.5, 14)

[0122] The coordinates of the upper left points of all grids in the first picture are:

[0123] (40,0), (40,2) ... (40, 14)

[0124] (42.5,0), (42.5,2) ... (42.5, 14) ...

[0126] (77.5,0), (77.5,2) ... (77.5, 14)

[0127] The coordinates of the upper left points of all grids in the second picture are:

[0128] (80,0), (80,2) ... (80, 14)

[0129] (82.5,0), (82.5,2) ... (82.5, 14) ...

[0131] (117.5,0), (117.5,2) ... (117.5, 14)

[0132] The coordinates of the upper left points of all grids in the third picture are:

[0133] (120,0), (120,2) ... (120, 14)

[0134] (122.5,0), (122.5,2) ... (122.5, 14) ...

[0136] (157.5,0), (157.5,2) ... (117.5, 14)

[0137] The coordinates of the upper left points of all grids on the 4th picture are:

[0138] (160,0), (160,2) ... (160, 14)

[0139] (162.5,0), (162.5,2) ... (162.5, 14) ...

[0141] (197.5,0), (197.5,2) ... (197.5, 14)

[0142] The coordinates of the upper left points of all grids on the 5th picture are:

[0143] (200,0), (200,2) ... (200, 14)

[0144] (202.5,0), (202.5,2) ... (202.5, 14) ...

[0146] (237.5,0), (237.5,2) ... (237.5, 14)

[0147] Furthermore, a test image of the test vehicle is acquired and distortion correction is performed on the test image to obtain a test corrected image; the above-mentioned virtual equidistant reference lines are added to the test corrected image, and the wheel pressure point in the test corrected image is identified, and the position coordinates of the wheel pressure point in the virtual equidistant reference lines are calculated.

[0148] Assume that in the first image acquisition device, among all the images acquired, there are 20 images that need to be involved in the calculation, and the coordinates of the wheel pressure points of the 20 images are calculated as follows:

[0149] (9.2, 1), (9.5, 13.5), (9.7, 26.4), (10.1, 39), (10.3, 52.3),

[0150] (10.4, 61), (10.4, 73.3), (10.3, 86.2), (10.2, 100.1), (10.1,112.4),

[0151] (10.0, 124.9), (9.9, 138), (9.9, 150.7), (10, 163.3), (10.2, 176),

[0152] (10.3, 189.9), (10.3, 204), (10.3, 216.6), (10.4, 227.4), (10.4,239.3).

[0153] According to the coordinates of the wheel pressure points, the mileage-offset function curve of the car can be obtained as follows: Fig. 9As shown, the horizontal axis is the relative mileage of the car, the vertical axis is the relative offset of the car, and the coordinates of the point are the coordinates of the above 20 points minus (9.2, 1).

[0154] The maximum offset of the test vehicle is:

[0155] offsetMax=|max(X_0,X_1,...X_ (num-1)) -X_0|=10.4-9.2=1.2 meters.

[0156] refer to Fig.10 In some embodiments, a vision-based vehicle deviation detection device is also provided, comprising:

[0157] The first image processing module 201 is used to receive the original image of the test road section without a vehicle acquired by the image acquisition device and perform distortion correction to obtain an original corrected image;

[0158] A coordinate generation module 202 is used to locate the zebra crossing according to the field of view in the original corrected image, extract the region of interest, and draw virtual equidistant reference lines in the region of interest according to the equidistant positioning lines and reference lines, and mark the coordinates of the virtual equidistant reference lines;

[0159] The second image processing module 203 is used to receive the test image of the test vehicle acquired by the image acquisition device and perform distortion correction to obtain a test correction image;

[0160] The calculation module 204 adds the virtual equidistant reference line to the test correction image, identifies the wheel pressure point in the test correction image, calculates the position coordinates of the wheel pressure point in the virtual equidistant reference line, and determines the deviation amount of the test vehicle.

[0161] Furthermore, the lane lines are two solid lines, which are parallel to the length direction of the test section and symmetrically distributed along the center line of the length direction of the test section, and the test vehicle travels within the two lane lines;

[0162] The equidistant positioning lines are two dotted lines, which are parallel to the length direction of the test section, and are respectively arranged on the outside of the two lane lines and at equal distances from the lane lines. The length of the line segments in the equidistant positioning lines is equal to the distance between the line segments, and the line segments in the two equidistant positioning lines are aligned one by one.

[0163] Furthermore, a camera positioning area is provided on the test section, and the image acquisition device is installed in the camera positioning area.

[0164] Furthermore, the number of the field of vision positioning zebra crossings is n+1 groups, where n is the number of image acquisition devices, and each group of the field of vision positioning zebra crossings includes four solid lines of fixed length, which are parallel to the length direction of the test section and are respectively arranged outside the two equidistant positioning lines, and two field of vision positioning zebra crossings are arranged on each side, and are symmetrically distributed along the center line of the length direction of the test section, and the distances between two adjacent groups of field of vision positioning zebra crossings are equal; the end of the field of vision positioning zebra crossing close to the starting point of the test vehicle is defined as the head end, and the end away from the starting point of the test vehicle is defined as the tail end, and the head end is aligned with the endpoint of the line segment in the equidistant positioning line, and in two adjacent groups of field of vision positioning zebra crossings, the distance between the head end of one group of field of vision positioning zebra crossings to the head end of the other group of field of vision positioning zebra crossings is an integer multiple of the length of the line segment in the equidistant positioning line;

[0165] The field of view of each image acquisition device includes two sets of field of view positioning zebra crossings;

[0166] The coordinate generation module 202 locates the zebra crossing according to the field of view in the original rectified image and extracts the region of interest, including:

[0167] Two groups of visual field positioning zebra crossings in the original corrected image are identified, and an area between a head end of a first group of visual field positioning zebra crossings close to a starting point of a test vehicle and a head end of a second group of visual field positioning zebra crossings far from a starting point of the test vehicle is determined as an area of ​​interest.

[0168] Further, the reference line is a solid line arranged at intervals of different colors, the solid line segments of each color have the same length, and the reference line is perpendicular to the equidistant positioning line;

[0169] The number of the reference lines is 2*n, where n is the number of image acquisition devices, different reference lines are spaced by a preset distance, the two ends of the reference line extend beyond the two equidistant positioning lines respectively, and the field of view of each image acquisition device includes two reference lines;

[0170] The coordinate generation module 202 draws virtual equidistant reference lines in the region of interest according to the equidistant positioning lines and the reference lines, including:

[0171] Identifying solid line segments of different colors in the reference line;

[0172] Draw a plurality of horizontal equidistant reference lines, so that the projection lines of the horizontal equidistant reference lines on the actual test section are perpendicular to the lane lines, and the spacing between the projection lines of the plurality of horizontal equidistant reference lines on the actual test section is equal to the length of the line segment in the equidistant positioning line;

[0173] Draw a plurality of vertical equidistant reference lines, so that the projection lines of the vertical equidistant reference lines on the actual test road section are parallel to the lane lines, and the spacing between the projection lines of the plurality of vertical equidistant reference lines on the actual test road section is equal to the length of a single color solid line segment in the reference line;

[0174] A plurality of the horizontal equidistant reference lines and the vertical equidistant reference lines constitute the virtual equidistant reference lines.

[0175] Furthermore, the coordinate generating module 202 marks the coordinates of the virtual equidistant reference lines, including:

[0176] In the grid formed by the plurality of horizontal equidistant reference lines and the vertical equidistant reference lines, the upper left vertex of the upper leftmost grid is selected as the coordinate origin;

[0177] Determine the width and height of each grid according to the length of the solid line segment in the reference line and the length of the line segment in the equidistant positioning line;

[0178] According to the coordinate origin, the width and height of each grid, the coordinates of the upper left vertex of each grid are calculated and marked.

[0179] Furthermore, the calculation module 204 calculates the position coordinates of the wheel pressure point on the virtual equidistant reference line to determine the deviation amount of the test vehicle, including:

[0180] Calculate a first distance from the left side of the grid where the wheel pressing point is located, and a second distance from the upper side of the grid where the wheel pressing point is located;

[0181] Calculating the position coordinates of the wheel pressing point according to the coordinates of the upper left vertex of the grid where the wheel pressing point is located and the first distance and the second distance;

[0182] The position coordinates of multiple wheel pressure points are obtained, and a mileage-offset function curve of the test vehicle is drawn. The deviation amount of the test vehicle is determined according to the mileage-offset function curve.

[0183] Further, the position coordinates of the wheel pressure point are (X, Y), wherein X=w+w0, Y=h+h0, (w, h) are the coordinates of the upper left vertex of the grid where the wheel pressure point is located, w0 is the first distance, and h0 is the second distance;

[0184] The position coordinates of the multiple wheel pressure points are (X_0, Y_0), (X_1, Y_1), ... (X_ num-1 ), Y_ num-1 ), where num is the number of wheel pressure points;

[0185] The first wheel rolling point position coordinates (X_0, Y_0) as the starting point, with (Y_ i -Y_0) is the horizontal coordinate, (X_ i -X_0) is the vertical coordinate, and the mileage-offset function curve is drawn, where i=1,2,3,...num-1.

[0186] Furthermore, the number of image acquisition devices on the test section is two or more, and the number of the camera positioning areas is the same as the number of the image acquisition devices;

[0187] The camera positioning area is arranged at one side of the test section and is located in front of the corresponding field of view positioning zebra crossing, and the distances between adjacent camera positioning areas are equal.

[0188] Please refer to the method embodiment for the specific principle, which will not be described here in detail.

[0189] In some embodiments, a vision-based vehicle deviation detection system is also provided, comprising a test road section, at least one image acquisition device, a processor, and a storage device;

[0190] The storage device stores a plurality of instructions, and the processor is used to read the instructions and execute the above method.

[0191] The vision-based vehicle deviation detection method and system provided in the above embodiments have at least the following beneficial effects:

[0192] (1) An external image acquisition device is used to obtain images of the test vehicle and the test road section to calculate the deviation of the vehicle. Since the deviation is calculated on the outside of the vehicle, it is objective and intuitive. Compared with the case where the camera is installed on the vehicle, the influence of the vehicle's own occlusion, bumpy displacement and road slope on the deviation calculation is avoided, thereby improving the accuracy of the test results;

[0193] (2) After the system of the present invention is debugged, the deviation of any vehicle can be tested. There is no need to replace the equipment or move the camera position every time the test vehicle is changed. The operation is simple, especially when it is necessary to measure multiple vehicles of different models, which can improve the detection efficiency.

[0194] (3) The system of the present invention consists of only three parts: a test section, an image acquisition device, and a computer, which reduces the cumulative error caused by too many devices. In addition, the equipment cost is low, which effectively reduces the production cost. The creation of the test section and the installation of the image acquisition device are simple, the calculation process is easy to understand, and the entire system is easy to reproduce.

[0195] (4) The vehicle can quickly calculate the deviation of the vehicle at multiple locations while driving on the test road, which makes up for the deficiency that most deviation detection systems can only test the deviation at a small number of locations, and provides a basis for subsequent result analysis and evaluation of autonomous driving performance.

[0196] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A vehicle deviation detection method based on vision, characterized in that: include: Set lane lines, equidistant positioning lines, visual field positioning zebra crossings and reference lines on the test road section; At least one image acquisition device is arranged on the test road section, and the field of view of the image acquisition device covers the lane lines, equidistant positioning lines, field of view positioning zebra crossings and reference lines; Receiving the original image of the test road section without a vehicle captured by the image acquisition device and performing distortion correction to obtain an original corrected image; Positioning the zebra crossing according to the field of view in the original corrected image, extracting the region of interest, and drawing virtual equidistant reference lines in the region of interest according to the equidistant positioning lines and reference lines, and marking coordinates of the virtual equidistant reference lines; Receiving the test image of the test vehicle acquired by the image acquisition device and performing distortion correction to obtain a test corrected image; Adding the virtual equidistant reference line to the test correction image, identifying the wheel pressure point in the test correction image, calculating the position coordinates of the wheel pressure point in the virtual equidistant reference line, and determining the deviation amount of the test vehicle; The lane lines are two solid lines, parallel to the length direction of the test section and symmetrically distributed along the center line of the length direction of the test section, and the test vehicle travels within the two lane lines; The equidistant positioning lines are two dotted lines, which are parallel to the length direction of the test road section, and are respectively arranged outside the two lane lines and at equal distances from the lane lines. The lengths of the line segments in the equidistant positioning lines are equal to the distances between the line segments, and the line segments in the two equidistant positioning lines are aligned one by one; The number of the field of vision positioning zebra crossings is n+1 groups, where n is the number of image acquisition devices. Each group of the field of vision positioning zebra crossings includes four solid lines of fixed length, which are parallel to the length direction of the test section and are respectively arranged outside the two equidistant positioning lines. Two field of vision positioning zebra crossings are arranged on each side and are symmetrically distributed along the center line of the length direction of the test section. The distance between every two adjacent groups of field of vision positioning zebra crossings is equal. The end of the field of vision positioning zebra crossing close to the starting point of the test vehicle is defined as the head end, and the end away from the starting point of the test vehicle is defined as the tail end. The head end is aligned with the endpoint of the line segment in the equidistant positioning line. In two adjacent groups of field of vision positioning zebra crossings, the distance between the head end of one group of field of vision positioning zebra crossings and the head end of the other group of field of vision positioning zebra crossings is an integer multiple of the length of the line segment in the equidistant positioning line. The reference lines are solid lines arranged at intervals of different colors, the solid line segments of each color have the same length, and the reference lines are perpendicular to the equidistant positioning lines; The number of the reference lines is 2*n, where n is the number of image acquisition devices. Different reference lines are spaced by a preset distance. Both ends of the reference lines extend beyond two equidistant positioning lines respectively. The field of view of each image acquisition device includes two reference lines.

2. The method according to claim 1, characterized in that A camera positioning area is also provided on the test section, and the image acquisition device is installed in the camera positioning area.

3. The method according to claim 1, characterized in that The field of view of each image acquisition device includes two sets of field of view positioning zebra crossings; The zebra crossing is located according to the field of view in the original rectified image, and the region of interest is extracted, including: Two groups of visual field positioning zebra crossings in the original corrected image are identified, and an area between a head end of a first group of visual field positioning zebra crossings close to a starting point of a test vehicle and a head end of a second group of visual field positioning zebra crossings far from a starting point of the test vehicle is determined as an area of ​​interest.

4. The method according to claim 1, characterized in that: Drawing a virtual equidistant reference line in the region of interest according to the equidistant positioning line and the reference line comprises: Identifying solid line segments of different colors in the reference line; Draw a plurality of horizontal equidistant reference lines, so that the projection lines of the horizontal equidistant reference lines on the actual test section are perpendicular to the lane lines, and the spacing between the projection lines of the plurality of horizontal equidistant reference lines on the actual test section is equal to the length of the line segment in the equidistant positioning line; Draw a plurality of vertical equidistant reference lines, so that the projection lines of the vertical equidistant reference lines on the actual test road section are parallel to the lane lines, and the spacing between the projection lines of the plurality of vertical equidistant reference lines on the actual test road section is equal to the length of a single color solid line segment in the reference line; A plurality of the horizontal equidistant reference lines and the vertical equidistant reference lines constitute the virtual equidistant reference lines.

5. The method according to claim 4, characterized in that Marking coordinates of the virtual equidistant reference lines includes: In the grid formed by the plurality of horizontal equidistant reference lines and the vertical equidistant reference lines, the upper left vertex of the upper leftmost grid is selected as the coordinate origin; Determine the width and height of each grid according to the length of the solid line segment in the reference line and the length of the line segment in the equidistant positioning line; According to the coordinate origin, the width and height of each grid, the coordinates of the upper left vertex of each grid are calculated and marked.

6. The method according to claim 5, characterized in that Calculating the position coordinates of the wheel pressure point on the virtual equidistant reference line to determine the deviation of the test vehicle includes: Calculate a first distance from the left side of the grid where the wheel pressing point is located, and a second distance from the upper side of the grid where the wheel pressing point is located; Calculating the position coordinates of the wheel pressing point according to the coordinates of the upper left vertex of the grid where the wheel pressing point is located and the first distance and the second distance; The position coordinates of multiple wheel pressure points are obtained, and a mileage-offset function curve of the test vehicle is drawn. The deviation amount of the test vehicle is determined according to the mileage-offset function curve.

7. The method according to claim 6, characterized in that The position coordinates of the wheel pressure point are (X, Y), where X=w+w0, Y=h+h0, (w, h) are the coordinates of the upper left vertex of the grid where the wheel pressure point is located, w0 is the first distance, and h0 is the second distance; The position coordinates of the multiple wheel pressure points are (X_0, Y_0), (X_1, Y_1), ... (X_ num-1 ), Y_ num-1 ), where num is the number of wheel pressure points; The first wheel rolling point position coordinates (X_0, Y_0) as the starting point, with (Y_ i -Y_0) is the horizontal coordinate, (X_ i -X_0) is the vertical coordinate, and the mileage-offset function curve is drawn, where i=1,2,3,...num-1.

8. The method according to claim 2, characterized in that: The number of image acquisition devices on the test section is two or more, and the number of the camera positioning areas is the same as the number of the image acquisition devices; The camera positioning area is arranged at one side of the test section and is located in front of the corresponding field of view positioning zebra crossing, and the distances between adjacent camera positioning areas are equal.

9. A vision-based vehicle deviation detection system, characterized in that: It includes a test section, at least one image acquisition device, a processor and a storage device; The storage device stores a plurality of instructions, and the processor is used to read the instructions and execute the method according to any one of claims 1-8.

Images