Dark environment vehicle following auxiliary driving method and system based on blackout lamp recognition

Abstract

The invention belongs to the technical field of vehicle auxiliary driving, and particularly relates to a dark environment vehicle following auxiliary driving method and system based on blackout lamp recognition, and the method comprises the steps: obtaining the minimum safety distance between a following vehicle and a following vehicle; acquiring a rear blackout lamp image of the followed vehicle based on an image acquisition device arranged in front of the followed vehicle; extracting contour features of the obtained rear blackout lamp image, and generating contour feature points; performing inter-frame image feature matching according to the generated contour feature points, and solving relative poses of the following vehicle and the following vehicle by combining a feature matching result; according to the obtained relative pose and the minimum safe distance, the speed and direction of the following vehicle are adjusted in real time, and vehicle following auxiliary driving in the dark environment is completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle assisted driving, and particularly relates to a vehicle following assisted driving method and system based on identification of air defense lights in a dark environment. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] With the improvement of the intelligent level, assisted driving and driverless technologies have developed vigorously. Vehicle following technology is widely used in intelligent transportation and intelligent transportation. On the premise of ensuring the driving safety of the first vehicle in front, the vehicle following technology can free the hands of the drivers of the rear vehicles. Especially in a dark environment, it can reduce the probability of frequent accidents caused by fatigue driving of drivers, reduce the number of drivers in the fleet, save human resource costs, and improve labor utilization rate. The premise of realizing the vehicle following technology is to solve the relative pose between the front and rear vehicles. According to the relative pose, the control law can be designed to realize the vehicle following technology between vehicles, so as to achieve assisted driving or even driverless driving.

[0004] Rear air defense lights are mostly installed at the rear of military vehicles and are used to drive at night during wartime to defend against reconnaissance by enemy aircraft and are used during blackouts. This can effectively reduce the probability of being discovered by the reconnaissance forces in the sky. By carefully studying the characteristics of air defense lights, it is found that: the light of the rear air defense light can be divided into two major regions. The upper contour region only emits light when braking and decelerating, while the lower contour region is in a constant light-emitting state after the vehicle starts. The light-emitting color of the air defense light is red, which is a very good recognition feature in a dark environment and provides a very good visual feature for using a camera to realize the vehicle following technology between vehicles.

[0005] The core of the vehicle following technology based on the identification of air defense lights in a dark environment is to solve the relative pose. Currently, the commonly used sensors for solving the relative pose are lidar and cameras; lidar collects point cloud data in the surrounding environment, extracts feature points and feature regions from the data for point cloud matching. Commonly used matching algorithms include Normal Distribution Transformation (NDT) and Iterative Closest Point (ICP). According to the matching results, the relative pose between the two vehicles is calculated; cameras can provide a rich data basis for the vehicle following technology. After the camera collects image data, the corresponding relationship between the features in the front and rear frames is determined through feature extraction and feature matching. Methods such as epipolar geometry, PNP, and ICP can be used to solve the relative pose between the two vehicles.

[0006] LiDAR can work all-weather, but its price is relatively high, which results in relatively low social and economic benefits. The use cost of cameras is low. Although the texture features on the images are reduced in dark environments, which directly restricts the application of cameras in the direction of intelligent driving, the rear air defense lights equipped on vehicles can be used as optical beacons to provide strong texture feature information for cameras in dark environments. Based on the strong optical beacon features of the air defense lights, it becomes possible to use cameras to solve the relative pose between two vehicles in dark environments. Summary of the Invention

[0007] To solve the above problems, the present invention proposes a method and system for vehicle following assisted driving in dark environments based on air defense light recognition. The air defense lights are used as an image feature source because they are extremely easy to recognize in dark environments. The air defense lights are regarded as an inherent component of the vehicle. Combining with the camera projection model and the PNP pose solution method, vehicle following driverless driving is carried out according to the obtained relative positions of the vehicles.

[0008] According to some embodiments, the first solution of the present invention provides a method for vehicle following assisted driving in dark environments based on air defense light recognition, adopting the following technical solutions:

[0009] A method for vehicle following assisted driving in dark environments based on air defense light recognition, comprising:

[0010] Obtain the minimum safety distance between the following vehicle and the vehicle being followed;

[0011] Based on the image acquisition device arranged in front of the following vehicle, obtain the image of the rear air defense light of the vehicle being followed;

[0012] Extract the contour features of the obtained rear air defense light image to generate contour feature points;

[0013] Perform inter-frame image feature matching according to the generated contour feature points, and combine the feature matching results to solve the relative pose between the following vehicle and the vehicle being followed;

[0014] According to the obtained relative pose and the minimum safety distance, adjust the speed and direction of the following vehicle in real time to complete vehicle following assisted driving in dark environments.

[0015] As a further technical limitation, use mask recognition to obtain the red area features of the rear air defense light image, and detect the rear air defense light contour according to the obtained image red area features and the Laplace operator to obtain the rear air defense light contour features.

[0016] As a further technical limitation, the generated contour feature points at least include feature point key points and feature point descriptors; according to the obtained rear air defense light contour features, calculate the minimum circumscribed matrix of the rear air defense light contour, calculate the center point of the minimum circumscribed matrix, and use the obtained center point as the feature point key point; encode the obtained rear air defense light contour features, and the feature point descriptors for reflecting the spatial position relationship between the rear air defense light contours can be obtained.

[0017] Further, perform inter-frame matching on the rear air defense light source contour features between the front and rear frames according to the obtained feature point descriptors to obtain a feature matching result, and complete the association of data information between different frames.

[0018] As a further technical limitation, calculate the relative pose between the following vehicle and the vehicle being followed according to the feature matching result and the PNP pose solution method; during the calculation of the relative pose, determine the camera projection model according to the type of the image acquisition device provided in front of the following vehicle, and determine the calculation method of the relative pose according to the determined camera projection model.

[0019] As a further technical limitation, determine the minimum safe distance between the following vehicle and the vehicle being followed according to the maximum braking distance of the following vehicle; according to the obtained relative pose, determine the current heading angle and speed of the following vehicle, and in combination with the obtained minimum safe distance, adjust the speed and direction of the following vehicle in real time to complete the vehicle following assisted driving in the dark environment.

[0020] According to some embodiments, the second solution of the present invention provides a vehicle following assisted driving system in a dark environment based on air defense light recognition, and adopts the following technical solutions:

[0021] A vehicle following assisted driving system in a dark environment based on air defense light recognition includes:

[0022] An acquisition module, which is configured to acquire the minimum safe distance between the following vehicle and the vehicle being followed; acquire the rear air defense light image of the vehicle being followed based on the image acquisition device provided in front of the following vehicle;

[0023] An extraction module, which is configured to extract the contour features of the acquired rear air defense light image and generate contour feature points;

[0024] A solution module, which is configured to perform matching of inter-frame image features according to the generated contour feature points, and solve the relative pose between the following vehicle and the vehicle being followed in combination with the feature matching result;

[0025] A following module, which is configured to adjust the speed and direction of the following vehicle in real time according to the obtained relative pose and the minimum safe distance to complete the vehicle following assisted driving in the dark environment.

[0026] According to some embodiments, the third solution of the present invention provides a computer-readable storage medium, adopting the following technical solution:

[0027] A computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the steps in a method for vehicle following assisted driving in a dark environment based on anti-aircraft light recognition as described in the first solution of the present invention.

[0028] According to some embodiments, the fourth solution of the present invention provides an electronic device, adopting the following technical solution:

[0029] An electronic device includes a memory, a processor, and a program stored on the memory and running on the processor. When the processor executes the program, it implements the steps in a method for vehicle following assisted driving in a dark environment based on anti-aircraft light recognition as described in the first solution of the present invention.

[0030] According to some embodiments, the fifth solution of the present invention provides a computer program product, adopting the following technical solution:

[0031] A computer program product includes software code, and the program in the software code executes the steps in a method for vehicle following assisted driving in a dark environment based on anti-aircraft light recognition as described in the first solution of the present invention.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] The present invention ingeniously applies the light-emitting characteristics of the rear anti-aircraft light. The anti-aircraft light is used as an image feature source because it is extremely easy to be recognized in a dark environment. The anti-aircraft light is used as an inherent component of the vehicle, combined with the camera projection model and the PNP pose solution method, and vehicle following driverless driving is carried out according to the obtained relative position of the vehicles.

[0034] The present invention extracts the light source contour to generate feature points according to the light-emitting characteristics of the rear anti-aircraft light, completes the inter-frame matching of the feature points by combining the position relationship coding descriptor, combines the camera projection model, the IMU pre-integration principle and the PNP algorithm to solve the relative pose between the front and rear vehicles, and finally designs a PID controller or an LQR control algorithm according to the relative pose to realize the vehicle following technology between the vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings forming a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments and descriptions thereof of this embodiment are used to explain this embodiment and do not constitute an improper limitation to this embodiment.

[0036] Figure 1 It is a schematic structural diagram of the rear anti-aircraft light in the first embodiment of the present invention;

[0037] Figure 2 It is a flowchart of a vehicle following assisted driving method in the dark environment based on anti-aircraft light recognition in the first embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the detailed steps of a vehicle following assisted driving method in the dark environment based on anti-aircraft light recognition in the first embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of the triangulation principle in the first embodiment of the present invention;

[0040] Figure 5 It is a structural block diagram of a vehicle following assisted driving system in the dark environment based on anti-aircraft light recognition in the second embodiment of the present invention. Detailed implementation manners

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are all exemplary and are intended to provide further descriptions of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0043] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0044] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0045] Embodiment 1

[0046] The first embodiment of the present invention introduces a vehicle following assisted driving method in the dark environment based on anti-aircraft light recognition.

[0047] As Figure 2 shown, a vehicle following assisted driving method in the dark environment based on anti-aircraft light recognition includes:

[0048] Obtain the minimum safety distance between the following vehicle and the vehicle being followed;

[0049] Based on the image acquisition device arranged in front of the following vehicle, obtain the image of the rear anti-aircraft light of the vehicle being followed;

[0050] Extract the contour features of the acquired rear anti-aircraft light image to generate contour feature points;

[0051] Match the inter-frame image features based on the generated contour feature points, and solve the relative pose of the following vehicle and the followed vehicle in combination with the feature matching results;

[0052] According to the obtained relative pose and the minimum safety distance, adjust the speed and direction of the following vehicle in real time to complete the vehicle following assisted driving in the dark environment.

[0053] The specific vehicle following assisted driving process is as Figure 3 shown, including the following steps:

[0054] Step S01: Set the minimum safety distance between the two vehicles to ensure the safety of the two vehicles or multiple vehicles during the following process;

[0055] Step S02: Calibrate the camera and calibrate the external parameters of the camera and the IMU;

[0056] Step S03: In the dark environment, collect the image information of the rear air defense light of the followed vehicle and input the IMU information of the following vehicle;

[0057] Step S04: Process the original image, that is, use a filtering algorithm (median filtering / Gaussian filtering / mean filtering, etc.) to reduce image noise, and use morphological operations to reduce the scattering phenomenon;

[0058] Step S05: Color space conversion, and perform color segmentation on the HSV color space;

[0059] Step S06: Air defense light source detection and contour extraction: Define the red HSV color range, create a mask according to the defined red HSV range, and use the mask to extract the red area from the original image and perform contour detection and extraction;

[0060] Step S07: Generate the contour feature points of the rear air defense light; specifically, extract the minimum circumscribed rectangle of each contour feature, and use the center point of the minimum circumscribed rectangle as the key point of the contour feature point; encode each contour feature, and the encoding consists of two digits: the first digit of the descriptor encoding is used to characterize the upper and lower relationships between the contour features, starting from 0 to 9 and increasing; the second digit of the descriptor encoding is used to characterize the left and right relationships between the contour features, starting from 0 to 9 and increasing, and the encoding of each contour represents the descriptor of the contour feature;

[0061] Step S08: Inter-frame matching; that is, perform matching between two frames on the feature points according to the descriptors of the contour feature points to complete the data association between the inter-frame image features;

[0062] Step S09: Solve the relative pose; that is, according to the feature matching results, solve the relative pose between the two vehicles based on the camera projection model and the PNP pose solution algorithm;

[0063] Step S10: Design a PID controller or an LQR control algorithm according to the relative pose information to adjust the speed and direction of the following vehicle, and implement the car-following technology;

[0064] Step S11: Complete the braking and deceleration action according to the braking warning signal.

[0065] As one or more embodiments, in step S01, to ensure the safety of the vehicle following process, the minimum following distance, that is, the minimum safety distance, between the following vehicle and the vehicle being followed is set according to the maximum braking distance of the following vehicle.

[0066] As one or more embodiments, in step S02, the Zhang Zhengyou calibration method is used to calibrate the monocular camera / binocular camera, solve the internal parameter matrix and distortion matrix of the binocular / monocular camera. If a binocular camera is selected, its projection matrix is solved; if a monocular camera and an IMU are selected, the external parameter matrix between the monocular camera and the IMU is solved; to ensure the accuracy of solving the relative pose.

[0067] When a binocular camera is selected, only the Zhang Zhengyou calibration method is needed for camera calibration. According to the idea of the Zhang Zhengyou calibration method, an image data set of the camera calibration board is collected. There are two most commonly used camera calibration boards, one is the checkerboard, and the other is the circular grid. The checkerboard is the most popular and commonly used pattern design. Several parameters of the camera model can be obtained according to the camera calibration:

[0068] Camera internal parameter matrix:

[0069]

[0070] where f x and f y represent the focal lengths of the camera, and c x and c y represent the coordinates of the principal point of the camera.

[0071] Camera distortion matrix:

[0072] [k1 k2 k3 p1 p2];

[0073] where k1, k2, and k3 are radial distortion parameters, and p1, p2 are tangential distortion parameters.

[0074] The camera internal parameter matrix is used to describe the geometric properties of the camera, and the camera distortion matrix is used to describe the distortion situation of the camera.

[0075] Using the camera distortion matrix, correct the radial distortion and tangential distortion of the points on the normalized plane:

[0076]

[0077] Among them, x corrected and y corrected are the normalized image coordinates after distortion, and r is the distance from the image pixel point to the center point of the image.

[0078] Project the corrected point onto the pixel plane through the internal parameter matrix to obtain the correct position of the point on the image:

[0079]

[0080] Among them, u and v are the pixel plane coordinates.

[0081] Then solve the external parameter matrix of the camera to obtain the scale relationship between the camera and the visual coordinates.

[0082] When a monocular camera is selected, camera calibration is performed on the monocular camera, similar to binocular camera calibration; and the Kalibr offline calibration method is used to jointly calibrate the monocular camera and the IMU, and the following can be obtained:

[0083] (1) Roughly estimate the time delay between the camera and the imu;

[0084] (2) Obtain the initial rotation between the imu-camera, and some necessary initial values: gravitational acceleration, gyroscope bias;

[0085] (3) Great optimization, non-linear optimization of the cost function, including all corner reprojection errors, imu accelerometer and gyroscope measurement errors, bias random walk noise, so as to ensure the accuracy of solving the relative pose.

[0086] As one or more implementation manners, in step S03, a binocular camera of the following vehicle is used to collect binocular image information of the rear anti-aircraft lamp of the vehicle being followed, or a monocular camera is used to collect image information of the rear anti-aircraft lamp of the vehicle being followed and obtain the IMU information of the following vehicle, providing a data basis for the subsequent relative pose solution.

[0087] As one or more implementation manners, in step S04, median filtering / Gaussian filtering / mean filtering is used to perform noise reduction processing on the image, reduce the noise in the digital image and remove redundant interference information, and use morphological operations to reduce the interference of scattering on the characteristics of the anti-aircraft lamp. The common image morphological operations are erosion and dilation. Erosion is to remove some burrs and details of the image, and dilation is to expand some burrs and details of the image; first erode the image, and then dilate the eroded image, that is, the opening operation is used for the scattering phenomenon in the image.

[0088] As one or more embodiments, in step S05, the color space of the image dataset is converted, and the image is converted from the BGR color space to the HSV color space for color segmentation to highlight the red light source feature of the air defense light. The conversion formula is as follows:

[0089]

[0090] v = max;

[0091] Where r is red, g is green, b is blue, max is the maximum of r, g, and b; min is the minimum of r, g, and b; h is the hue, s is the chroma (saturation), and v is the brightness.

[0092] As one or more embodiments, in step S06, by defining the HSV range of red, two masks are created, one for red in the range of 0 - 10 degrees and the other for red in the range of 160 - 180 degrees. The saturation and brightness thresholds are set according to the actual situation, and then they are merged into a final mask; the mask is used to identify the red area feature of the air defense light from the original image.

[0093] The Laplace operator is used to detect the contour features of the air defense light, and finally the OpenCV library function findContours is used for extraction. The definition formula of the Laplace operator is as follows:

[0094]

[0095] That is, the sum of the second-order differentials in the x and y directions. Generalized to the discrete domain, it is:

[0096] In the x direction:

[0097]

[0098] In the y direction:

[0099]

[0100] Then:

[0101]

[0102] What the standard Laplace operator does is actually to subtract the central value from the sum of the values of the center point's upper, lower, left, and right in a 3x3 neighborhood.

[0103] As one or more embodiments, in step S07, this embodiment adopts the rear air defense light as shown in Figure 1 The four groups of contour areas below are in the always-on state after the vehicle starts, while the upper contour area only lights up when the vehicle brakes and decelerates.

[0104] The generated contour feature points at least include feature point key points and feature point descriptors;

[0105] For each extracted contour feature, calculate its Minimum Bounding Rectangle (MBR), solve the center point of the minimum bounding rectangle, and use it as the key point of the contour feature.

[0106] Assume that after contour feature extraction, the rectangular frame enclosing the contour has the upper left corner coordinates (x, y), width width, and height height. Then the geometric center of the rectangle is:

[0107] Center point x coordinate = x + width / 2;

[0108] Center point y coordinate = y + height / 2.

[0109] Encode each contour feature as a descriptor of the feature point to describe the contour feature; this encoding consists of two digits and aims to reflect the spatial position relationship between contours; specifically:

[0110] (1) Up and down relationship

[0111] The first digit of the descriptor encoding is used to characterize the up and down relationship between contour features. Based on the y coordinate value in the center points of the contour features, select the contour feature with the smallest y coordinate value as the reference point. Assume the coordinates of the base point are: (x1, y1), and the key point of another contour feature is (x2, y2). If y1 - y2 < σ, it is on the same horizontal plane, and at this time, the first digit of the descriptor is set to 0; if y1 - y2 ≥ σ, it is on different horizontal planes, and at this time, the first digit of the descriptor of the reference point is set to 0, and the first digit of the descriptor corresponding to (x2, y2) is set to 1; where σ is the distance constraint. The first digit of the descriptor is shown in Table 1.

[0112] Table 1 The first digit of the descriptor

[0113]

[0114] (2) Left and right relationship

[0115] The second digit of the descriptor coding is used to characterize the left - right relationship between contour features. Given the up - down relationship of the known contour, if the first digit of the descriptor appears as 1, it indicates that the air defense light contour has two upper and lower layers, and the following vehicle is in a braking state. Record the braking warning signal (Q) at this time. Remove the light source contour features encoded with 0 as the first digit of the descriptor (extract but do not participate in inter - frame feature matching), and change the contour features with the first digit of the coding being 1 to the number 0. Based on the x - coordinate value in the center point of the contour features, code according to the x - coordinate values of the contour key points from small to large. The coding starts from 0 and increments, completing the expression of the left - right relationship of the contour features, and finally generating the feature point descriptor for matching; the descriptor is shown in Table 2.

[0116] Table 2 Descriptor

[0117]

[0118] Through the above - mentioned coding method, a unique two - digit coding is generated for each contour, and this coding serves as the descriptor of the contour feature. This descriptor is not only concise and clear but also can effectively reflect the relative position relationship between contours, thus facilitating the subsequent contour feature matching work.

[0119] As one or more embodiments, in step S08, frame - to - frame matching of the air defense light source contour features between the front and rear frames is completed according to the descriptor information of the contour features. Specifically:

[0120] Assume that the descriptor of contour feature A in the reference frame is 0x, and the descriptor of contour feature B in the current frame is 0y. If xx∧yy = 00 (where ^ is the bit - wise exclusive - OR), it indicates that contour feature A in the reference frame is contour feature B in the current frame, and the task of data association is completed.

[0121] As one or more embodiments, in step S09, according to the feature matching result, the PNP algorithm is used to solve the relative pose based on the camera imaging model. Specifically:

[0122] Monocular camera + IMU:

[0123] Use the matched feature point pairs of the first two frames for epipolar constraint; calculate the homography (H) matrix or the fundamental (F) matrix to solve the inter - frame motion.

[0124] The relative pose solved by epipolar constraint is as follows:

[0125] Assume that the matched feature points of the front and rear two frames are p1, p2, and the corresponding homogeneous pixel coordinates are respectively:

[0126]

[0127] Among them, P is the spatial position corresponding to p1 and p2 in different frames.

[0128] If x1 = K -1 p1, x2 = K -1 p2, then x1 and x2 are the coordinates of two pixel points on the corresponding normalized plane. Substituting them in, we get x2 = Rx1 + t;

[0129] Multiply both sides by t^ on the left and take the cross product with t; that is, t∧x2 = t∧Rx1; Multiply both sides by to get Substitute p1 and p2 back in, that is Denote E = t∧R, we get Recover R and t from E, that is E = U∑V T ∑ = diag(a, 0, 0).

[0130] The translation calculated from IMU pre-integration = scale factor * the translation calculated by the pure vision sensor + the rotation calculated by the pure vision sensor * the intrinsic translation between sensors to calculate the scale factor of the monocular camera and complete the initialization of the monocular camera; Use the obtained 3D coordinates and the feature point coordinates in the current frame to calculate the pose of the camera in the spatial coordinates using the PNP algorithm; The scale factor is

[0131] Among them, is the translation calculated from MU pre-integration;

[0132] is the translation calculated by the pure vision sensor;

[0133] is the rotation calculated by the pure vision sensor;

[0134] is the intrinsic translation between sensors;

[0135] After completing the initialization of the monocular camera, use the obtained 3D coordinates and the feature point coordinates in the current frame to calculate the pose of the camera in the spatial coordinates using the PNP algorithm.

[0136] Stereo camera:

[0137] Use the feature point pairs matched in the left and right eye images of the first frame, and solve the spatial coordinates (i.e., 3D coordinates) of the feature points according to the triangulation principle as shown in Figure 4 ; Take the left eye camera as the main camera; Use the obtained 3D coordinates and the feature point coordinates corresponding to the left eye image in the current frame to calculate the relative pose between the two vehicles using the PNP algorithm.

[0138] Assume that the depth of the feature point P1 matched in the front and rear frames or the left and right frames is s1, and the depth of the feature point P2 is s2. They intersect at a point P, satisfying the equation:

[0139] s1x1 = s2Rx2 + t;

[0140] Multiply both sides of the equation by The obtained equation is:

[0141]

[0142] Solve for s2. Similarly, s1 can be obtained. By this method, the depth information is solved, and thus their spatial coordinates can be determined.

[0143] As one or more embodiments, in step S10, using the relative pose between the two vehicles (including x, y coordinates and heading angle) and the speed of the following vehicle as inputs, design a PID controller or an LQR algorithm to adjust the speed and direction of the following vehicle to implement the following-following technology.

[0144] (1) PID controller settings

[0145] Longitudinal control (distance control):

[0146] Known relative position: e x And the speed v of the following vehicle, the set safety distance is: e taget , then the designed PID controller is:

[0147]

[0148] Among them, Δv x Is the expected change in steering angular velocity.

[0149] Lateral control (steering control):

[0150] Known relative heading angle is e θ And the speed v of the following vehicle, the set expected heading angle is θ taget , then the designed PID controller is Among them, δ desired Is the expected steering angle.

[0151] Brake deceleration setting, complete the brake deceleration action according to the coding characteristics. When the first digit of the contour feature coding appears as 1, that is, there is 1* in the contour feature coding, at this time the vehicle being followed is in a braking state, and the following vehicle makes a brake deceleration braking according to the coding characteristics to ensure the safety distance between the front and rear vehicles.

[0152] (2) LQR control algorithm settings algorithm

[0153] Set the safety distance as e taget, the desired heading angle is θ taget , solve the known where e x is the relative position between the two vehicles, and e θ is the relative heading angle between the two vehicles, and v is the speed of the following vehicle.

[0154] Establish a state-space model:

[0155]

[0156] where is the time delay, k y is the gain of the following control, and a is the acceleration of the following vehicle (which can be directly read by the IMU).

[0157] (3) Design of the LQR controller

[0158] Select appropriate weight matrices Q and R; that is, Q has a higher weight for the distance and the heading angle difference to ensure following safety; R is the weight for the control input, and usually a smaller value is selected to smooth the control.

[0159] Use the Riccati equation to solve the control gain matrix K = R -1 B T P; design the LQR control law u = -Kx;

[0160] Feedback control;

[0161] In each control period, repeat the following steps:

[0162] Calculate the current state x;

[0163] Use the LQR control law to calculate the control input u;

[0164] Update the state of the following vehicle.

[0165] As one or more embodiments, in step S11, the braking action of the following vehicle is completed according to the braking warning signal. When the braking warning signal Q is received, the following vehicle makes a braking and decelerating action to ensure the safety distance between the front and rear vehicles.

[0166] This embodiment ingeniously applies the light-emitting characteristics of the rear air defense lamp. The air defense lamp serves as an image feature source because it is extremely easy to identify in a dark environment. The air defense lamp is used as an inherent component of the vehicle. Combining the camera projection model and the PNP pose solution method, vehicle following for driverless driving is performed based on the obtained relative position of the vehicle; the light source contour is extracted according to the light-emitting characteristics of the rear air defense lamp to generate feature points, and the feature point frame-to-frame matching is completed by combining the position relationship coding descriptor. Combining the camera projection model, the IMU pre-integration principle, and the PNP algorithm to solve the relative pose between the front and rear vehicles. Finally, a PID controller or an LQR control algorithm is designed based on the relative pose to achieve the vehicle following technology between vehicles.

[0167] Embodiment 2

[0168] Embodiment 2 of the present invention introduces a vehicle following assisted driving system in a dark environment based on air defense lamp recognition.

[0169] As Figure 5 shown, a vehicle following assisted driving system in a dark environment based on air defense lamp recognition includes:

[0170] An acquisition module, which is configured to acquire the minimum safety distance between the following vehicle and the vehicle being followed; acquire the image of the rear air defense lamp of the vehicle being followed based on the image acquisition device arranged in front of the following vehicle;

[0171] An extraction module, which is configured to extract the contour features of the acquired rear air defense lamp image to generate contour feature points;

[0172] A solution module, which is configured to perform frame-to-frame image feature matching according to the generated contour feature points, and solve the relative pose between the following vehicle and the vehicle being followed by combining the feature matching results;

[0173] A following module, which is configured to adjust the speed and direction of the following vehicle in real time according to the obtained relative pose and the minimum safety distance, and complete the vehicle following assisted driving in a dark environment.

[0174] The detailed steps are the same as those of a vehicle following assisted driving method in a dark environment based on air defense lamp recognition provided in Embodiment 1, and will not be elaborated here.

[0175] Embodiment 3

[0176] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0177] A computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the steps in a vehicle following assisted driving method in a dark environment based on air defense lamp recognition as described in Embodiment 1 of the present invention.

[0178] The detailed steps are the same as those of a vehicle following assisted driving method in a dark environment based on air defense light recognition provided in Embodiment 1, and will not be elaborated here.

[0179] Embodiment 4

[0180] Embodiment 4 of the present invention provides an electronic device.

[0181] An electronic device includes a memory, a processor, and a program stored on the memory and running on the processor. When the processor executes the program, it implements the steps in a vehicle following assisted driving method in a dark environment based on air defense light recognition as described in Embodiment 1 of the present invention.

[0182] The detailed steps are the same as those of a vehicle following assisted driving method in a dark environment based on air defense light recognition provided in Embodiment 1, and will not be elaborated here.

[0183] Embodiment 5

[0184] Embodiment 5 of the present invention provides a computer program product.

[0185] A computer program product includes software code, and the program in the software code implements the steps in a vehicle following assisted driving method in a dark environment based on air defense light recognition as described in Embodiment 1 of the present invention.

[0186] The detailed steps are the same as those of a vehicle following assisted driving method in a dark environment based on air defense light recognition provided in Embodiment 1, and will not be elaborated here.

[0187] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented in various computer languages, for example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.

[0188] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0189] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0191] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0192] Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

[0193] The above description is only the preferred embodiments of this embodiment and is not intended to limit this embodiment. For those skilled in the art, this embodiment can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this embodiment shall be included within the protection scope of this embodiment.

Claims

1. A vehicle following assisted driving method in a dark environment based on air defense light recognition, characterized in that Including: Obtain the minimum safe distance between the following vehicle and the leading vehicle; Obtain the rear air defense light image of the leading vehicle based on the image acquisition device arranged in front of the following vehicle; Extract the contour features of the obtained rear air defense light image to generate contour feature points; Perform inter-frame image feature matching according to the generated contour feature points, and solve the relative pose of the following vehicle and the leading vehicle in combination with the feature matching result; According to the obtained relative pose and the minimum safe distance, adjust the speed and direction of the following vehicle in real time to complete the vehicle following assisted driving in the dark environment.

2. The vehicle following assisted driving method in a dark environment based on air defense light recognition according to claim 1, wherein Adopt mask recognition to identify the red area features of the obtained rear air defense light image, detect the rear air defense light contour according to the obtained image red area features and the Laplace operator, and obtain the rear air defense light contour features.

3. A method for assisting in following a vehicle in a dark environment based on air defense light recognition as described in claim 1, characterized in that, The generated contour feature points at least include feature point key points and feature point descriptors; according to the obtained rear air defense light contour features, calculate the minimum circumscribed matrix of the rear air defense light contour, calculate the center point of the minimum circumscribed matrix, and use the obtained center point as the feature point key point; encode the obtained rear air defense light contour features to obtain the feature point descriptors for reflecting the spatial position relationship between the rear air defense light contours.

4. A method for vehicle following assisted driving in a dark environment based on air defense light recognition as described in claim 3, characterized in that, Perform inter-frame matching on the rear air defense light source contour features between the front and rear frames according to the obtained feature point descriptors to obtain the feature matching result and complete the data information association between different frames.

5. A vehicle following assisted driving method in a dark environment based on air defense light recognition as described in claim 1, characterized in that, Calculate the relative pose between the following vehicle and the leading vehicle according to the feature matching result and the PNP pose solution method; during the calculation of the relative pose, determine the camera projection model according to the type of the image acquisition device arranged in front of the following vehicle, and determine the calculation method of the relative pose according to the determined camera projection model.

6. The vehicle following assisted driving method in a dark environment based on air defense light recognition as described in claim 1, characterized in that, Determine the minimum safe distance between the following vehicle and the leading vehicle according to the maximum braking distance of the following vehicle; determine the current heading angle and speed of the following vehicle according to the obtained relative pose, and in combination with the obtained minimum safe distance, adjust the speed and direction of the following vehicle in real time to complete the vehicle following assisted driving in the dark environment.

7. A vehicle following assisted driving system in a dark environment based on air defense light recognition, characterized in that, Including: An acquisition module configured to obtain the minimum safe distance between the following vehicle and the leading vehicle; Obtain the rear air defense light image of the leading vehicle based on the image acquisition device arranged in front of the following vehicle; An extraction module configured to extract the contour features of the obtained rear air defense light image to generate contour feature points; A solution module configured to perform inter-frame image feature matching according to the generated contour feature points, and solve the relative pose of the following vehicle and the leading vehicle in combination with the feature matching result; A following module configured to adjust the speed and direction of the following vehicle in real time according to the obtained relative pose and the minimum safe distance to complete the vehicle following assisted driving in the dark environment.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method for vehicle following assisted driving in the dark environment based on air defense light recognition as described in any one of claims 1-6.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for vehicle following assisted driving in the dark environment based on air defense light recognition as described in any one of claims 1-6.

10. A computer program product, comprising software code, characterized in that, The program in the software code executes the steps of the vehicle following assisted driving method based on air defense light recognition as described in any one of claims 1-6.

Images