A non-structured curved road safety early warning system and method based on a binocular camera

Abstract

The application discloses a kind of unstructured curved road safety warning system and method based on binocular camera, safety warning system includes binocular camera module, road edge detection module, central processing module, comparison module, communication module, vehicle-mounted signal receiving module, LED screen and voice module, wherein binocular camera module and road edge detection module are connected between and both are connected with central processing module, central processing module is sequentially connected with comparison module, communication module and vehicle-mounted signal receiving module, vehicle-mounted signal receiving module is connected with LED screen and voice module respectively, its method is: first step, arrangement binocular camera and sensing device;Second step, arrangement central processing module and other modules;Third step, running curvature identification algorithm;Fourth step, obtain safety speed;Fifth step, curved road warning system algorithm operation;Beneficial effect: the application can reduce the possibility of accident of unstructured large-curvature curved road, and the practicality is stronger.

Description

Technical Field

[0001] This invention relates to a curve safety warning system and method, and particularly to an unstructured curve safety warning system and method based on a binocular camera. Background Technology

[0002] Currently, with economic progress and rising living standards, car ownership continues to grow. To facilitate people's livelihoods, almost every town and village is equipped with small public buses. However, in rural areas and economically underdeveloped regions (such as mountainous areas with significant elevation differences), the widespread adoption of structured roads will take time, while unstructured roads pose some safety hazards. Unstructured roads lack lane markings and guardrails, and are often narrow, mostly single-lane. This is disadvantageous for passing and turning maneuvers; failure to slow down to a safe speed in these situations could potentially threaten lives and property.

[0003] Most existing curve recognition methods are designed for structured roads, and curve recognition for structured roads can be categorized into those that rely on road infrastructure. Curve recognition technologies that rely on road infrastructure require receiving signals from the equipment to identify the start and end points of the curve, placing high demands on the quality of road construction. Another type of curve recognition technology relies on image data from vehicle-mounted cameras to identify and judge curves. However, this type of method is easily affected by external environmental factors, such as weather, and can exhibit significant errors or even fail to recognize curves at night. This second type of method is currently the mainstream approach. Both types of methods are designed for structured roads with clearly defined lane markings; research on curve recognition for unstructured roads is limited. Summary of the Invention

[0004] The main purpose of this invention is to address the lack of existing research on the identification of unstructured curves with large curvature in rural towns or economically underdeveloped areas, and to provide a safety early warning system and method for unstructured curves based on binocular cameras.

[0005] The unstructured curve safety warning system based on binocular cameras provided by this invention includes a binocular camera module, a road edge detection module, a central processing module, a comparison module, a communication module, an on-board signal receiving module, an LED screen, and a voice module. The binocular camera module and the road edge detection module are connected to the central processing module. The central processing module is sequentially connected to the comparison module, the communication module, and the on-board signal receiving module. The on-board signal receiving module is connected to the LED screen and the voice module. The binocular camera module and the road edge detection module can transmit the collected data to the central processing module in real time. The central processing module processes the received data and then sends it through the comparison module and the communication module to the on-board signal receiving module via a data transmission system. The on-board signal receiving module outputs the data transmitted by the central processing module through the LED screen and the voice module.

[0006] The binocular camera module includes two roadside binocular cameras, each equipped with a sensor to detect the edge points of the camera's field of view. The module acquires images of the external environment, stores the image data, performs preprocessing for image calibration and correction, and performs distance measurement. The road edge detection module extracts features from the road edges, identifies feature points, and outputs coordinate information. The central processing module includes a storage module and calculates the radius and curvature of a circle, using the curvature to calculate a safe speed. A comparison module compares the speed of a vehicle turning a curve with the corresponding safe speed for that curve. A communication module sends information to the vehicle turning the curve. The onboard signal receiving module receives feedback from the receiving system and transmits it to the LED screen and voice module.

[0007] The aforementioned binocular camera module, road edge detection module, central processing module, comparison module, communication module, vehicle signal receiving module, LED screen, and voice module are all assemblies of existing equipment; therefore, their specific models and specifications are not detailed here.

[0008] The working principle of the unstructured curve safety early warning system based on binocular cameras provided by this invention is as follows:

[0009] The unstructured curve safety warning system based on binocular cameras provided by this invention requires two binocular cameras to be installed at the roadside to ensure that the image information of the entire curve can be captured, and that the data information of the two cameras is interconnected. Therefore, a central processing unit is needed to connect the two cameras to achieve data exchange and processing. In this invention, the curve curvature recognition uses the principle of binocular camera ranging. The left camera of the binocular cameras acquires the original image, and the right camera acquires the driving image. Due to parallax, the images acquired by the two cameras are not exactly the same. Therefore, the original image and the driving image first need to be stereo calibrated and stereo corrected, and then stereo matching is performed to obtain depth information. Since this invention also involves road edge detection, two output ports need to be added to the right camera: one for stereo matching and the other as the input image for road edge detection. This invention can be applied to large-curvature unstructured road curves with unknown road information. It will automatically acquire image information and process it to obtain road curvature information. After installation, the two binocular cameras at the roadside first need to be stereo calibrated and stereo corrected to ensure that the images from the left and right cameras are consistent. First, road edge detection is performed to obtain a binary image containing only road edge information. Feature points are then divided along the road edge, and the coordinate information of each feature point is extracted. This coordinate information is added to the stereo matching of a binocular camera, and the depth information of the feature points is obtained using the ranging principle of the binocular camera. Subsequently, the radius of the circle determined by three consecutive feature points is obtained using the principle that three points determine a circle, and its corresponding curvature is calculated. The above operation is repeated iteratively to calculate the curvature information of the circle determined by every three feature points. The optimal entry speed for cornering is then set based on the calculation. The position information of each arc and its corresponding curvature information are recorded and stored in the system. When there is no possibility of oncoming traffic, the system measures the speed of the vehicle cornering and compares it with the safe speed calculated by the system, providing real-time feedback to the vehicle to ensure safe cornering. When there is a possibility of oncoming traffic, the system incorporates the speed information of both vehicles when calculating the safe cornering speed, ensuring safe cornering and safe passing.

[0010] In a first aspect, the invention performs road edge detection based on the image from the right camera of a binocular camera to obtain feature point coordinate information; based on the binocular camera ranging principle, it imports the feature point coordinate information and divides the scale to obtain feature point depth information; based on the feature points, it uses the principle of three points determining a circle to obtain the radius of the circle, and further obtains the corresponding curvature; based on the curvature, it uses mathematical methods to determine the optimal speed for entering the curve, and feeds it back to the vehicle entering the curve to ensure safe cornering; if there is a possibility of oncoming traffic, it monitors the speeds of the two vehicles in real time and feeds back the optimal speeds to ensure safe passing and cornering.

[0011] In a second aspect, the present invention includes a binocular camera module for acquiring images of the external environment, storing image data from the binocular camera, performing preprocessing such as calibration and correction on the binocular images, and performing operations such as distance measurement; a road edge detection module for extracting features from the road edge, dividing feature points, and outputting coordinate information; a data transmission module for enabling data communication between the modules; a central processing module (containing a storage module) for calculating the radius and curvature of a circle, and calculating a safe vehicle speed using the curvature; a comparison module for comparing the vehicle speed when cornering with the corresponding safe vehicle speed for a curve; a communication module for sending information to the vehicle when cornering; a sensing device installed in the binocular camera module for sensing the position of the edge points of the camera's field of view; and an on-board signal receiving module (including an LED screen and a voice module) for receiving information feedback from the system.

[0012] The present invention provides an unstructured curve safety early warning method based on a binocular camera, the method comprising the following steps:

[0013] Step 1: Set up the dual-lens camera and sensor. There are two cameras of the same model, named Camera 1 and Camera 2. The specific steps are as follows:

[0014] Step 1: The high curvature curve is divided into a left-side entry point and a right-side entry point. Camera 1 faces the right-side entry point, and camera 2 faces the left-side entry point.

[0015] Step 2: Adjust the camera's field of view. The maximum field of view on the left side of camera 1 coincides with point D, the starting point of the right curve entrance. This determines point B, the maximum field of view on the right side. Place a sensor here. Similarly, the maximum field of view on the right side of camera 2 coincides with point A, the starting point of the left curve entrance. This determines point C, the maximum field of view on the left side. Place another sensor here. The recognition range of camera 1 is BD, and the recognition range of camera 2 is AC. The entire high-curvature curve is AD, which includes points B and C.

[0016] Step 3: Debug the binocular cameras 1 and 2 and the sensing devices. Perform stereo calibration and stereo correction on the cameras to ensure that the left and right camera images of each binocular camera are consistent. The B point sensing device is sensed by the 2nd camera, and the C point sensing device is sensed by the 1st camera, to ensure that the position information of the B and C points can be recognized by the 2nd and 1st cameras respectively.

[0017] The second step involves deploying the central processing module and other modules, with the specific steps as follows:

[0018] Step 1: Install an integrated iron box between the two cameras, and install various modules inside. Store the limit safe speed at which any curvature will not cause sideslip or rollover in the comparison module.

[0019] Step 2: Connect the modules and the two cameras using the data transmission cable.

[0020] The third step is to run the curvature recognition algorithm. The specific steps are as follows:

[0021] Step 1: Road Edge Detection

[0022] The purpose of this step is to obtain the feature points of the curve in order to identify the curvature;

[0023] Two binocular cameras are used to collect images. Camera 1 captures images of the BD section curve, and camera 2 captures images of the AC section curve. There is an overlapping section BC at both ends of the curve.

[0024] The images captured by the right camera of the two cameras are used as the image input for road edge detection. The road edge detection module is used to extract the feature map. First, the image is preprocessed. Then, the curve edge is identified and fitted. Finally, the feature map is output, which is a binary image with only gray values ​​of 0 and 1.

[0025] After the feature map is extracted, the feature points will be divided. Since there is an overlapping segment BC, the division scale of the entire curve is determined by segment BC. The location information of point C has been sensed in advance. The division scale is as follows:

[0026]

[0027] In the above formula, L is the length of BE, and i is the number of parts into which BE is divided. Optional, i can be 5, 10, or 20. The larger the value of i, the higher the division accuracy, and the more accurate the final curvature will be. Assuming i = 10, then the division scale n = L / 10. Then, segments AB and CD are divided using n = L / 10 as the scale, ultimately obtaining several feature points. The image coordinate information of these feature points is recorded, including the four points ABCD.

[0028] Step 2, Curvature Recognition, as follows:

[0029] Camera 1 measures the vertical distance h from point B. Adding a scale, the distance is iteratively measured forward from point B. The distance from camera 1 to F1 is h+n, the distance to F2 is h+2n, and so on, until the distance to point D is reached. Using the principle that three points determine a circle, the coordinates of each set of three points, along with the distance measurement information, are fed into the central processing module for calculation to obtain the corresponding radius r. i Where i ≥ 1 and is an integer, as detailed below:

[0030] h(B),h+n(F1),h+2n(F2)→r1(B,F1,F2)

[0031] h+n(F1),h+2n(F2),h+3n(F3)→r2(F1,F2,F3) ...

[0033] h+3n(F3),h+4n(F4),h+5n(C)→r4(F3,F4,C) ... ...

[0036] h+(i-1)n(F i-1 ),h+in(F i ),h+(i+1)n(D)→r i (F i-1 ,F i D)

[0037] In the above iterations, n = L / 10. After calculating the radius, the corresponding relationship is used to find the radius r for each radius. i The corresponding curvature ρ i Record all measured curvature ρ of the entire BD segment. i And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint F of every three points. i The location information is used to represent the location and is then passed to the storage module for storage.

[0038] Since the curvature of segment BC has already been measured, only the curvature of segment AB needs to be measured. The vertical distance from camera 2 to point A is H. Adding a scale, extending from point A backward, iteratively measure the distance. The distance from camera 2 to G1 is Hn, the distance from camera 2 to G2 is H-2n, and so on, until the distance to point B is reached. Using the principle that three points determine a circle, the coordinates of each three points and the distance measurement information are input into the central processing module for calculation to obtain the corresponding radius R. j Where j≥1 and is an integer, as follows:

[0039] H(A),Hn(G1),H-2n(G2)→R1(A,G1,G2)

[0040] Hn(G1),H-2n(G2),H-3n(G3)→R2(G1,G2,G3) ... ...

[0043] H-(j-1)n(G j-1 ),H-jn(G j ),H-(j+1)n(B)→R j (G j-1 G j B)

[0044] In the above iterations, n = L / 10. After calculating the radius, the corresponding relationship is used to find the radius R for each radius. j The corresponding curvature ρ j Record all measured curvatures ρ of the entire AB segment. j And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint G of every three points. j The location information is used to represent the location and is then passed to the comparison module for storage.

[0045] Step 4: Determine a safe speed. The specific steps are as follows:

[0046] Step 1: Assemble the curvature measured in segment AB into set P1, the curvature measured in segment BC into set P2, and the curvature measured in segment CD into set P3.

[0047] Step 2: Set the safety factor. Since sections AB and CD are straight sections that gradually turn into curves, the curvature changes are small, so the safety factor range can be set to a small value, represented by k1. However, the large curvature change of the large curvature curve AD is mainly reflected in the middle section BC, so the safety factor range is large, represented by k2.

[0048] Step 3: Determine the safe speed, as follows:

[0049] ① For segment AB, import the curvature values ​​from set P1 into the comparison module for sorting, and extract the maximum curvature value ρ. max1 and the minimum curvature ρ min1 Then the curvature of segment AB is:

[0050]

[0051] Using curvature ρ x1 The corresponding limit speed v1 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k1 (0.5≤k1≤1.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal entry speed for the left-hand corner is:

[0052] V1 = v1 - k1

[0053] ② For segment BC, import the curvature values ​​from set P2 into the comparison module for sorting, and extract the maximum curvature value ρ. max2 and the minimum curvature ρ min2 Then the curvature of segment BC is:

[0054]

[0055] Using curvature ρ x2The corresponding limit speed v2 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k2 (1.5≤k2≤2.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal cornering speed for a mid-section curve with high curvature is:

[0056] V2 = v2 - k2

[0057] ③ For segment CD, import the curvature values ​​from set P3 into the comparison module for sorting, and extract the maximum curvature value ρ. max3 and the minimum curvature ρ min3 Then the curvature of segment CD is:

[0058]

[0059] Using curvature ρ x3 The corresponding limit speed v3 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k1 (0.5≤k1≤1.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal entry speed for the right-hand corner is:

[0060] V3 = v3 - k1;

[0061] Step 5: The curve warning system algorithm operates, taking left-side entry into the curve as the reference. The specific steps are as follows:

[0062] Step 1: Measure vehicle speed. The speed of vehicle 1 is measured by camera 2, and the speed of vehicle 2 is measured by camera 1. Taking vehicle 1 as an example, the speed of vehicle 1 is:

[0063]

[0064] In the above formula, t1 and t2 are the distance measurement times, L1 is the distance from vehicle 1 to camera 2 at time t1, and L2 is the distance from vehicle 1 to camera 2 at time t2.

[0065] Step 2: Determine the speed of the vehicle entering the curve. When v≤V1, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please drive carefully"; when v>V1, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please slow down to V1".

[0066] Step 3: Determine the cornering speed

[0067] ① When there is no possibility of oncoming traffic, that is, when camera 2 does not detect any vehicle entering the curve on the right, and vehicle 1 is located in section AB, compare vehicle speeds:

[0068] When v≤V1, compare v with V2. If v≤V2, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please be careful when turning". If v>V2, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please slow down to V2". Since V2≤V3, vehicle 1 can continue to travel at speed V2 until it exits the curve.

[0069] When v > V1, since V1 ≥ V2, the system compares the speed v of vehicle 1 with the safe speed corresponding to the curvature of the curve where vehicle 1 is located in real time, and uses the communication module to provide real-time feedback to the voice alarm system of vehicle 1 to remind it to slow down.

[0070] ② When there is a possibility of oncoming traffic, that is, when there are two cars in the curve AD, car 1 is still the reference. The speed and position information of car 1 are obtained by camera 2, and the speed and position information of car 2 are obtained by camera 1. The speed of car 1 is v, and the speed of car 2 is V.

[0071] Scenario 1: Car 1 is located in segment AB, and Car 2 is located in segment CD. When v ≤ V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is about to pass, distance is far, please drive carefully" and the LED signal "Oncoming vehicle speed is V". When v > V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is about to pass, distance is far, please slow down to V2-k2" and the LED message "Oncoming vehicle speed is V".

[0072] Scenario 2: Car 1 is located in segment AB, and Car 2 is located in segment BC. When v ≤ V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is approaching, distance is close, please drive carefully" and the LED signal "Oncoming vehicle speed is V". When v > V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is approaching, distance is close, please slow down to V1-k1" and the LED message "Oncoming vehicle speed is V".

[0073] Scenario 3: Car 1 is located in section BC and Car 2 is located in section CD. In this case, v≤V2 must be true. Then, the roadside communication module sends a signal to the vehicle, including the voice signal "Please be careful when turning, you will meet another vehicle soon. The distance is close, please slow down appropriately" and the LED signal "The other vehicle's speed is V".

[0074] In all three situations described above, after safely passing oncoming traffic, continue driving safely out of the curve at the current speed, or follow the safe speed given by the system to exit the curve.

[0075] The beneficial effects of this invention are:

[0076] The present invention provides a safety warning system and method for unstructured curves based on binocular cameras. This is because there is currently little research on dealing with unstructured curves with large curvature. Unstructured curves with large curvature are prone to accidents due to the lack of guardrails, lane markings, and blind spots. The present invention aims to provide a reliable method and system to reduce the possibility of accidents on unstructured curves with large curvature. It is applicable to underdeveloped rural areas and mountainous areas with more extreme road conditions, and has strong practicality. Attached Figure Description

[0077] Figure 1 This is a structural block diagram of the unstructured curve safety early warning system described in this invention.

[0078] Figure 2 This is a schematic diagram of the unstructured curve safety early warning method described in this invention.

[0079] Figure 3 This is a schematic diagram illustrating the application of the unstructured curve safety early warning system described in this invention.

[0080] Figure 4 This is a schematic diagram of the feature point segmentation method described in this invention.

[0081] Figure 5 This is a schematic diagram of the BD segment curvature recognition algorithm described in this invention.

[0082] Figure 6 This is a schematic diagram of the AB segment curvature recognition algorithm described in this invention.

[0083] The annotations in the image above are as follows:

[0084] 1. Binocular camera module; 2. Road edge detection module; 3. Central processing module

[0085] 4. Comparison module; 5. Communication module; 6. Vehicle signal receiving module; 7. LED screen.

[0086] 8. Voice module. Detailed Implementation

[0087] Please see Figures 1 to 6 As shown:

[0088] The unstructured curve safety warning system based on binocular cameras provided by this invention includes a binocular camera module 1, a road edge detection module 2, a central processing module 3, a comparison module 4, a communication module 5, an on-board signal receiving module 6, an LED screen 7, and a voice module 8. The binocular camera module 1 and the road edge detection module 2 are connected to the central processing module 3. The central processing module 3 is sequentially connected to the comparison module 4, the communication module 5, and the on-board signal receiving module 6. The on-board signal receiving module 6 is connected to the LED screen 7 and the voice module 8. The binocular camera module 1 and the road edge detection module 2 can transmit the collected data to the central processing module 3 in real time. The central processing module 3 processes the received data and then sends it to the on-board signal receiving module 6 via the comparison module 4 and the communication module 5 through a data transmission system. The on-board signal receiving module 6 outputs the data transmitted by the central processing module 3 through the LED screen 7 and the voice module 8.

[0089] The binocular camera module 1 includes two binocular cameras on the roadside. Each camera contains a sensor that detects the edge points of the camera's field of view. The binocular camera module 1 acquires images of the external environment, stores the image data from the binocular cameras, performs preprocessing for calibration and correction of the binocular images, and performs distance measurement. The road edge detection module 2 extracts features from the road edges, identifies feature points, and outputs coordinate information. The central processing module 3 contains a storage module and calculates the radius and curvature of a circle. It also calculates the safe speed based on the curvature. The comparison module 4 compares the speed of a vehicle turning a curve with the corresponding safe speed for that curve. The communication module 5 sends information to the vehicle turning a curve. The vehicle signal receiving module 6 receives information from the receiving system and transmits it to the LED screen 7 and the voice module 8.

[0090] The aforementioned binocular camera module 1, road edge detection module 2, central processing module 3, comparison module 4, communication module 5, vehicle signal receiving module 6, LED screen 7, and voice module 8 are all assemblies of existing equipment; therefore, their specific models and specifications are not detailed here.

[0091] The working principle of the unstructured curve safety early warning system based on binocular cameras provided by this invention is as follows:

[0092] The unstructured curve safety warning system based on binocular cameras provided by this invention requires two binocular cameras to be installed at the roadside to ensure that the image information of the entire curve can be captured, and that the data information of the two cameras is interconnected. Therefore, a central processing unit is needed to connect the two cameras to achieve data exchange and processing. In this invention, the curve curvature recognition uses the principle of binocular camera ranging. The left camera of the binocular cameras acquires the original image, and the right camera acquires the driving image. Due to parallax, the images acquired by the two cameras are not exactly the same. Therefore, the original image and the driving image first need to be stereo calibrated and stereo corrected, and then stereo matching is performed to obtain depth information. Since this invention also involves road edge detection, two output ports need to be added to the right camera: one for stereo matching and the other as the input image for road edge detection. This invention can be applied to large-curvature unstructured road curves with unknown road information. It will automatically acquire image information and process it to obtain road curvature information. After installation, the two binocular cameras at the roadside first need to be stereo calibrated and stereo corrected to ensure that the images from the left and right cameras are consistent. First, road edge detection is performed to obtain a binary image containing only road edge information. Feature points are then divided along the road edge, and the coordinate information of each feature point is extracted. This coordinate information is added to the stereo matching of a binocular camera, and the depth information of the feature points is obtained using the ranging principle of the binocular camera. Subsequently, the radius of the circle determined by three consecutive feature points is obtained using the principle that three points determine a circle, and its corresponding curvature is calculated. The above operation is repeated iteratively to calculate the curvature information of the circle determined by every three feature points. The optimal entry speed for cornering is then set based on the calculation. The position information of each arc and its corresponding curvature information are recorded and stored in the system. When there is no possibility of oncoming traffic, the system measures the speed of the vehicle cornering and compares it with the safe speed calculated by the system, providing real-time feedback to the vehicle to ensure safe cornering. When there is a possibility of oncoming traffic, the system incorporates the speed information of both vehicles when calculating the safe cornering speed, ensuring safe cornering and safe passing.

[0093] In a first aspect, the invention performs road edge detection based on the image from the right camera of a binocular camera to obtain feature point coordinate information; based on the binocular camera ranging principle, it imports the feature point coordinate information and divides the scale to obtain feature point depth information; based on the feature points, it uses the principle of three points determining a circle to obtain the radius of the circle, and further obtains the corresponding curvature; based on the curvature, it uses mathematical methods to determine the optimal speed for entering the curve, and feeds it back to the vehicle entering the curve to ensure safe cornering; if there is a possibility of oncoming traffic, it monitors the speeds of the two vehicles in real time and feeds back the optimal speeds to ensure safe passing and cornering.

[0094] In a second aspect, the present invention includes a binocular camera module 1 for acquiring images of the external environment, storing image data from the binocular camera, performing preprocessing such as calibration and correction on the binocular images, and performing operations such as distance measurement; a road edge detection module 2 for extracting features from the road edge, dividing feature points, and outputting coordinate information; a data transmission module for enabling data communication between the modules; a central processing module 3 (containing a storage module) for calculating the radius and curvature of a circle, and calculating a safe vehicle speed using the curvature; a comparison module 4 for comparing the vehicle speed when cornering with the corresponding safe vehicle speed for a curve; a communication module 5 for sending information to the vehicle when cornering; a sensing device installed in the binocular camera module 1 for sensing the position of the edge points of the camera's field of view; and an on-board signal receiving module 6 (including an LED screen and a voice module) for receiving information feedback from the system.

[0095] The present invention provides an unstructured curve safety early warning method based on a binocular camera, the method comprising the following steps:

[0096] Step 1: Set up the dual-lens camera and sensor. There are two cameras of the same model, named Camera 1 and Camera 2. The specific steps are as follows:

[0097] Step 1: The high curvature curve is divided into a left-side entry point and a right-side entry point. Camera 1 faces the right-side entry point, and camera 2 faces the left-side entry point.

[0098] Step 2: Adjust the camera's field of view. The maximum field of view on the left side of camera 1 coincides with point D, the starting point of the right curve entrance. This determines point B, the maximum field of view on the right side. Place a sensor here. Similarly, the maximum field of view on the right side of camera 2 coincides with point A, the starting point of the left curve entrance. This determines point C, the maximum field of view on the left side. Place another sensor here. The recognition range of camera 1 is BD, and the recognition range of camera 2 is AC. The entire high-curvature curve is AD, which includes points B and C.

[0099] Step 3: Debug the binocular cameras 1 and 2 and the sensing devices. Perform stereo calibration and stereo correction on the cameras to ensure that the left and right camera images of each binocular camera are consistent. The B point sensing device is sensed by the 2nd camera, and the C point sensing device is sensed by the 1st camera, to ensure that the position information of the B and C points can be recognized by the 2nd and 1st cameras respectively.

[0100] The second step involves deploying the central processing module 3 and other modules. The specific steps are as follows:

[0101] Step 1: Install an integrated iron box between the two cameras, and install various modules inside. Store the limit safe speed at which any curvature will not cause sideslip or rollover in the comparison module 4.

[0102] Step 2: Connect the modules and the two cameras using the data transmission cable;

[0103] The third step is to run the curvature recognition algorithm. The specific steps are as follows:

[0104] Step 1: Road edge detection:

[0105] The purpose of this step is to obtain the feature points of the curve in order to identify the curvature;

[0106] Two binocular cameras are used to collect images. Camera 1 captures images of the BD section curve, and camera 2 captures images of the AC section curve. There is an overlapping section BC at both ends of the curve.

[0107] The images captured by the right camera of the two cameras are used as the image input for road edge detection. The feature map is extracted using the road edge detection module 2. First, the image is preprocessed. Then, the curve edge is identified and fitted. Finally, the feature map is output, which is a binary image with only gray values ​​of 0 and 1.

[0108] After the feature map is extracted, the feature points will be divided. Since there is an overlapping segment BC, the division scale of the entire curve is determined by segment BC. The location information of point C has been sensed in advance. The division scale is as follows:

[0109]

[0110] In the above formula, L is the length of BE, and i is the number of parts into which BE is divided. Optional, i can be 5, 10, or 20. The larger the value of i, the higher the division accuracy, and the more accurate the final curvature will be. Assuming i = 10, then the division scale n = L / 10. Then, segments AB and CD are divided using n = L / 10 as the scale, ultimately obtaining several feature points. The image coordinate information of these feature points is recorded, including the four points ABCD.

[0111] Step 2, Curvature Recognition, as follows:

[0112] Camera 1 measures the vertical distance h from point B. Adding a scale, the distance is iteratively measured forward from point B. The distance from camera 1 to F1 is h+n, the distance to F2 is h+2n, and so on, until the distance to point D is reached. Using the principle that three points determine a circle, the coordinates of each set of three points, along with the distance measurement information, are fed into the central processing module for calculation to obtain the corresponding radius r. i Where i ≥ 1 and is an integer, as detailed below:

[0113] h(B),h+n(F1),h+2n(F2)→r1(B,F1,F2)

[0114] h+n(F1),h+2n(F2),h+3n(F3)→r2(F1,F2,F3) ...

[0116] h+3n(F3),h+4n(F4),h+5n(C)→r4(F3,F4,C) ... ...

[0119] h+(i-1)n(F i-1 ),h+in(F i ),h+(i+1)n(D)→r i (F i-1 ,F i D)

[0120] In the above iterations, n = L / 10. After calculating the radius, the corresponding relationship is used to find the radius r for each radius. i The corresponding curvature ρ i Record all measured curvature ρ of the entire BD segment. i And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint F of every three points. i The location information is used to represent the location and is then passed to the storage module for storage.

[0121] Since the curvature of segment BC has already been measured, only the curvature of segment AB needs to be measured. The vertical distance from camera 2 to point A is H. Adding a scale, extending from point A backward, iteratively measure the distance. The distance from camera 2 to G1 is Hn, the distance from camera 2 to G2 is H-2n, and so on, until the distance to point B is reached. Using the principle that three points determine a circle, the coordinates of each three points and the distance measurement information are input into the central processing module for calculation to obtain the corresponding radius R. j Where j≥1 and is an integer, as follows:

[0122] H(A),Hn(G1),H-2n(G2)→R1(A,G1,G2)

[0123] Hn(G1),H-2n(G2),H-3n(G3)→R2(G1,G2,G3) ... ...

[0126] H-(j-1)n(G j-1 ),H-jn(G j ),H-(j+1)n(B)→R j (G j-1 G j B)

[0127] In the above iterations, n = L / 10. After calculating the radius, the corresponding relationship is used to find the radius R for each radius. j The corresponding curvature ρ j Record all measured curvatures ρ of the entire AB segment. j And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint G of every three points. j The location information is used to represent the location and is then passed to the comparison module for storage.

[0128] Step 4: Determine a safe speed. The specific steps are as follows:

[0129] Step 1: Assemble the curvature measured in segment AB into set P1, the curvature measured in segment BC into set P2, and the curvature measured in segment CD into set P3.

[0130] Step 2: Set the safety factor. Since sections AB and CD are straight sections that gradually turn into curves, the curvature changes are small, so the safety factor range can be set to a small value, represented by k1. However, the large curvature change of the large curvature curve AD is mainly reflected in the middle section BC, so the safety factor range is large, represented by k2.

[0131] Step 3: Determine the safe speed, as follows:

[0132] ① For segment AB, import the curvature values ​​from set P1 into the comparison module for sorting, and extract the maximum curvature value ρ. max1 and the minimum curvature ρ min1 Then the curvature of segment AB is:

[0133]

[0134] Using curvature ρ x1 The corresponding limit speed v1 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k1 (0.5≤k1≤1.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal entry speed for the left-hand corner is:

[0135] V1 = v1 - k1

[0136] ② For segment BC, import the curvature values ​​from set P2 into the comparison module for sorting, and extract the maximum curvature value ρ. max2 and the minimum curvature ρ min2 Then the curvature of segment BC is:

[0137]

[0138] Using curvature ρ x2The corresponding limit speed v2 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k2 (1.5≤k2≤2.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal cornering speed for a mid-section curve with high curvature is:

[0139] V2 = v2 - k2

[0140] ③ For segment CD, import the curvature values ​​from set P3 into the comparison module for sorting, and extract the maximum curvature value ρ. max3 and the minimum curvature ρ min3 Then the curvature of segment CD is:

[0141]

[0142] Using curvature ρ x3 The corresponding limit speed v3 that prevents skidding and rollover is used to characterize the safe speed. A safety factor k1 (0.5≤k1≤1.5, km / h) is set, and the specific value can be determined according to the vehicle model. Then, the optimal entry speed for the right-hand corner is:

[0143] V3 = v3 - k1;

[0144] Step 5: The curve warning system algorithm operates, taking left-side entry into the curve as the reference. The specific steps are as follows:

[0145] Step 1: Measure vehicle speed. The speed of vehicle 1 is measured by camera 2, and the speed of vehicle 2 is measured by camera 1. Taking vehicle 1 as an example, the speed of vehicle 1 is:

[0146]

[0147] In the above formula, t1 and t2 are the distance measurement times, L1 is the distance from vehicle 1 to camera 2 at time t1, and L2 is the distance from vehicle 1 to camera 2 at time t2.

[0148] Step 2: Determine the speed of the vehicle entering the curve. When v≤V1, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please drive carefully"; when v>V1, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please slow down to V1".

[0149] Step 3: Determine the cornering speed

[0150] ① When there is no possibility of oncoming traffic, that is, when camera 2 does not detect any vehicle entering the curve on the right, and vehicle 1 is located in section AB, compare vehicle speeds:

[0151] When v≤V1, compare v with V2. If v≤V2, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please be careful when turning". If v>V2, the roadside communication module sends a signal to the vehicle, which is the voice signal "Curve ahead, please slow down to V2". Since V2≤V3, vehicle 1 can continue to travel at speed V2 until it exits the curve.

[0152] When v > V1, since V1 ≥ V2, the system compares the speed v of vehicle 1 with the safe speed corresponding to the curvature of the curve where vehicle 1 is located in real time, and uses the communication module to provide real-time feedback to the voice alarm system of vehicle 1 to remind it to slow down.

[0153] ② When there is a possibility of oncoming traffic, that is, when there are two cars in the curve AD, car 1 is still the reference. The speed and position information of car 1 are obtained by camera 2, and the speed and position information of car 2 are obtained by camera 1. The speed of car 1 is v, and the speed of car 2 is V.

[0154] Scenario 1: Car 1 is located in segment AB, and Car 2 is located in segment CD. When v ≤ V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is about to pass, distance is far, please drive carefully" and the LED signal "Oncoming vehicle speed is V". When v > V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is about to pass, distance is far, please slow down to V2-k2" and the LED message "Oncoming vehicle speed is V".

[0155] Scenario 2: Car 1 is located in segment AB, and Car 2 is located in segment BC. When v ≤ V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is approaching, distance is close, please drive carefully" and the LED signal "Oncoming vehicle speed is V". When v > V1, the roadside communication module sends signals to the vehicles, including the voice signal "Curve ahead, oncoming traffic is approaching, distance is close, please slow down to V1-k1" and the LED message "Oncoming vehicle speed is V".

[0156] Scenario 3: Car 1 is located in section BC and Car 2 is located in section CD. In this case, v≤V2 must be true. Then, the roadside communication module sends a signal to the vehicle, including the voice signal "Please be careful when turning, you will meet another vehicle soon. The distance is close, please slow down appropriately" and the LED signal "The other vehicle's speed is V".

[0157] In all three situations described above, after safely passing oncoming traffic, continue driving safely out of the curve at the current speed, or follow the safe speed given by the system to exit the curve.

Claims

1. An unstructured curve safety warning system based on a binocular camera, comprising a binocular camera module, a road edge detection module, a central processing module, a comparison module, a communication module, an on-board signal receiving module, an LED screen, and a voice module, wherein the binocular camera module and the road edge detection module are connected to the central processing module, the central processing module is sequentially connected to the comparison module, the communication module, and the on-board signal receiving module, and the on-board signal receiving module is connected to the LED screen and the voice module, the binocular camera module and the road edge detection module can transmit the collected data to the central processing module in real time, the central processing module processes the received data and then sends the data through the comparison module and the communication module to the on-board signal receiving module via a data transmission system, and the on-board signal receiving module outputs the data transmitted by the central processing module through the LED screen and the voice module, characterized in that: The binocular camera module includes two binocular cameras on the roadside. Each binocular camera is equipped with a sensing device, which is used to sense the position of the edge point of the camera's field of view. The binocular camera module is used to acquire images of the external environment, store image data from the binocular cameras, perform preprocessing for calibration and correction of the binocular images, and perform distance measurement operations. The road edge detection module is used to extract features from the road edges, divide feature points, and output coordinate information; the central processing module contains a storage module, which is used to calculate the radius and curvature of the circle, and calculate the safe speed through the curvature; the comparison module is used to compare the speed of the vehicle turning the corner with the corresponding safe speed for the curve; the communication module sends information to the vehicle turning the corner; the vehicle signal receiving module receives the information from the receiving system and then transmits it to the LED screen and the voice module.

2. A method for unstructured curve safety early warning based on a binocular camera, which is implemented according to claim 1, is characterized in that: The method includes the following steps: Step 1: Set up the dual-lens camera and sensor. There are two cameras of the same model, named Camera 1 and Camera 2. The specific steps are as follows: Step 1: The high curvature curve is divided into a left-side entry point and a right-side entry point. Camera 1 faces the right-side entry point, and camera 2 faces the left-side entry point. Step 2: Adjust the camera's field of view. The maximum field of view on the left side of camera 1 coincides with point D, the starting point of the right curve entrance. This determines point B, the maximum field of view on the right side. Place a sensor here. The maximum field of view on the right side of camera 2 coincides with point A, the starting point of the left curve entrance. This determines point C, the maximum field of view on the left side. Place another sensor here. The recognition range of camera 1 is BD, and the recognition range of camera 2 is AC. The entire high-curvature curve is AD, which includes points B and C. Step 3: Debug the binocular cameras 1 and 2 and the sensing devices. Perform stereo calibration and stereo correction on the cameras to ensure that the left and right camera images of each binocular camera are consistent. The B point sensing device is sensed by the 2nd camera, and the C point sensing device is sensed by the 1st camera, to ensure that the position information of the B and C points can be recognized by the 2nd and 1st cameras respectively. The second step involves deploying the central processing module and other modules, with the specific steps as follows: Step 1: Install an integrated iron box between the two cameras, and install various modules inside. Store the limit safe speed at which any curvature will not cause sideslip or rollover in the comparison module. Step 2: Connect the modules and the two cameras using the data transmission cable; The third step is to run the curvature recognition algorithm. The specific steps are as follows: Step 1: Road edge detection; The purpose of this step is to obtain the feature points of the curve in order to identify the curvature; Two binocular cameras are used to collect images. Camera 1 captures images of the BD section curve, and camera 2 captures images of the AC section curve. There is an overlapping section BC at both ends of the curve. The images captured by the right camera of the two cameras are used as the image input for road edge detection. The road edge detection module is used to extract the feature map. First, the image is preprocessed. Then, the curve edge is identified and fitted. Finally, the feature map is output, which is a binary image with only gray values ​​of 0 and 1. After the feature map is extracted, the feature points will be divided. Since there is an overlapping segment BC, the division scale of the entire curve is determined by segment BC. The location information of point C has been sensed in advance. The division scale is as follows: ； In the above formula, Let BC be the length. The number of parts to divide BC is optional. Options include 5, 10, and 20. The larger the value, the higher the division accuracy, and the more precise the final curvature will be. Assuming... =10, then the division scale Then segments AB and CD are... The scale is divided to obtain several feature points, and the image coordinate information of these feature points is recorded, including four points A, B, C, and D. Step 2, Curvature Recognition, as follows: The vertical distance to point B measured by camera number 1 is: Add a scale division, extend forward from point B, iteratively measure distance from camera 1 to... The distance is Camera No. 1 to The distance is This process continues until the distance to point D is measured. Using the principle that three points determine a circle, the coordinates of each set of three points, along with the distance measurement information, are fed into the central processing module for calculation to obtain the corresponding radius. The details are as follows: ； ； ...； ； ...； ...； ； In the above iterations, After calculating the radius, the corresponding relationship is used to find the radius of each part. Corresponding curvature Record all measured curvatures of the entire BD segment. And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint of every three points. The location information is used to represent the location and is then passed to the storage module for storage. Since the curvature of segment BC has already been measured, only the curvature of segment AB needs to be measured. The vertical distance from camera 2 to point A is... Add a scale division, extend from point A backward, iteratively measure the distance from camera 2 to... The distance is Camera No. 2 to The distance is This process continues until the distance to point B is measured. Using the principle that three points determine a circle, the coordinates of each set of three points, along with the distance measurement information, are fed into the central processing module for calculation to obtain the corresponding radius. The details are as follows: ； ； ...； ...； ； In the above iterations, After calculating the radius, the corresponding relationship is used to find the radius of each part. Corresponding curvature Record all measured curvatures of the entire AB segment. And the position information of the arc segment corresponding to each curvature, the position information of the arc segment is based on the midpoint of every three points. The location information is used to represent the location and is then passed to the comparison module for storage. Step 4: Determine a safe speed. The specific steps are as follows: Step 1: Assemble the curvature measurements of segment AB into a set. The set of curvatures measured in segment BC The set of curvatures measured in segment CD ; Step 2: Set a safety factor. Since sections AB and CD gradually transition from straight sections to curves, ... This indicates that the large curvature change of the AD curve with high curvature is mainly reflected in the middle section BC, therefore the safety factor range is large, and the safety factor is based on... express; Step 3: Determine the safe speed, as follows: ① For segment AB, set The curvature values ​​are imported into the comparison module, sorted, and the maximum curvature value is extracted. and minimum curvature Then the curvature of segment AB is: ； Using curvature The corresponding maximum speed at which skidding and rollover will not occur. To characterize safe vehicle speed, a safety factor is set. The specific value may vary depending on the vehicle model. Therefore, the optimal entry speed for the left-hand corner is: ； ②For segment BC, the set The curvature values ​​are imported into the comparison module, sorted, and the maximum curvature value is extracted. and minimum curvature Then the curvature of segment BC is: ； Using curvature The corresponding maximum speed at which skidding and rollover will not occur. To characterize safe vehicle speed, a safety factor is set. The specific value can be determined depending on the vehicle model. Therefore, the optimal cornering speed for a mid-section curve with high curvature is: ； ③ For segment CD, the set The curvature values ​​are imported into the comparison module, sorted, and the maximum curvature value is extracted. and minimum curvature Then the curvature of segment CD is: ； Using curvature The corresponding maximum speed at which skidding and rollover will not occur. To characterize safe vehicle speed, a safety factor is set. The specific value may vary depending on the vehicle model. Therefore, the optimal entry speed for the right-hand corner is: ； Step 5: The curve warning system algorithm operates, taking left-side entry into the curve as the reference. The specific steps are as follows: Step 1: Measure vehicle speed. The speed of vehicle 1 is measured by camera 2, and the speed of vehicle 2 is measured by camera 1. Taking vehicle 1 as an example, the speed of vehicle 1 is: ； In the above formula, , At the time of distance measurement, for The distance between camera 1 and camera 2 on the timetable. for The distance between camera 1 and camera 2 on the timetable; Step 2: Determine the entry speed into the curve. At this time, the roadside communication module sends a signal to the vehicle, which is a voice signal: "Curve ahead, please drive carefully"; when At this time, the roadside communication module sends a signal to the vehicle, which is a voice signal: "Curve ahead, please slow down." ”; Step 3: Determine the cornering speed; ① When there is no possibility of oncoming traffic, that is, when camera 2 does not detect any vehicle entering the curve on the right, and vehicle 1 is located in section AB, compare vehicle speeds: when At the time, in comparison and ,like Then the roadside communication module sends a signal to the vehicle, which is a voice signal: "Curve ahead, please be careful when turning"; if The roadside communication module then sends a signal to the vehicle, which is a voice signal: "Curve ahead, please slow down." "Driving", due to So car 1 can maintain its speed Drive until you exit the curve; when ,because At this time, the speed of car 1 is compared in real time by the system. The safe speed corresponding to the curvature of the curve where vehicle 1 is located is fed back to the voice alarm system of vehicle 1 in real time via the communication module to remind the driver to slow down. ② When there is a possibility of oncoming traffic, i.e., when there are two vehicles within the curve AD, vehicle 1 is still the reference vehicle. The speed and position information of vehicle 1 are obtained from camera 2, and the speed and position information of vehicle 2 are obtained from camera 1. The speed of vehicle 1 is... Car 2's speed is ; Scenario 1: Car 1 is located in segment AB, and Car 2 is located in segment CD. At this time, the roadside communication module sends signals to the vehicle, including a voice signal "Curve ahead, oncoming traffic is approaching, distance is far, please drive carefully" and an LED signal "Oncoming vehicle speed is..." ";when At this time, the roadside communication module sends signals to the vehicle, including a voice signal: "Curve ahead, oncoming traffic is about to pass, distance is far, please slow down." "and LED information" indicates the other vehicle's speed is ”; Scenario 2: Car 1 is located in segment AB, and Car 2 is located in segment BC. At this time, the roadside communication module sends signals to the vehicle, including a voice signal "Curve ahead, oncoming traffic is approaching, distance is close, please drive carefully" and an LED signal "Oncoming vehicle speed is..." ";when At this time, the roadside communication module sends signals to the vehicle, including a voice signal: "Curve ahead, oncoming traffic is approaching, the distance is close, please slow down." "and LED information" indicates the other vehicle's speed is ”; Scenario 3: Car 1 is in section BC, and Car 2 is in section CD. If the condition is confirmed, the roadside communication module will send signals to the vehicle, including a voice message "Please be careful around the bend, oncoming traffic is approaching, the distance is close, please slow down appropriately" and an LED message "Oncoming vehicle speed is..." ”; In all three situations described above, after safely passing oncoming traffic, continue driving safely out of the curve at the current speed, or follow the safe speed given by the system to exit the curve.

Images