Image processing algorithm and system for roadbed visual positioning of intelligent connected vehicles

By using roadbed cameras and feature point extraction and matching visual positioning algorithms in intelligent connected cars, the problem of insufficient GPS positioning accuracy in complex scenes is solved and high-precision vehicle positioning is achieved.

CN114724096BActive Publication Date: 2025-01-17HARBIN INST OF TECH AT WEIHAI
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202011523093.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-19
Publication Date
2025-01-17
Estimated Expiration
2040-12-19

AI Technical Summary

Technical Problem

The prior art is difficult to realize high-precision positioning of intelligent connected vehicles in complex indoor and outdoor scenarios, especially when GPS positioning accuracy is insufficient or lost.

Method used

The roadbed visual positioning image processing algorithm for intelligent connected vehicles based on roadbed cameras and feature point extraction and matching is used to simultaneously collect images through multiple cameras to perform feature point extraction, convolutional neural network target detection and multi-view feature point matching to achieve three-dimensional Formation of spatial feature point cloud and high-precision positioning of vehicles.

Benefits of technology

It achieves high-precision vehicle positioning in a wide range of indoor and outdoor complex scenes. Compared with traditional GPS positioning methods, it has a wider positioning range and higher positioning accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0002845987140000021
    Figure BDA0002845987140000021
  • Figure BDA0002845987140000022
    Figure BDA0002845987140000022
  • Figure BDA0002845987140000031
    Figure BDA0002845987140000031
Patent Text Reader

Abstract

The present invention relates to a high-precision positioning technology for intelligent connected vehicles and a roadbed visual positioning image processing algorithm and system for intelligent connected vehicles with vehicle-road collaboration. The system comprises a roadbed camera, an edge computing terminal, a roadbed vehicle network terminal and an on-board vehicle network terminal, wherein the roadbed camera comprises more than two cameras, and after the roadbed camera synchronously collects visual images at different angles in the same area, each frame of the image and the image time mark are sent to the edge computing terminal; the edge computing terminal runs the above-mentioned roadbed visual positioning image processing method for intelligent connected vehicles to realize real-time positioning measurement and feature extraction of all vehicles in the field of view, and obtain high-precision vehicle information, including position, running status, vehicle model, and appearance; the vehicle position and running status measurement within the camera field of view can be realized in real time, and with the support of the high-speed communication technology of 5G vehicle network, high-precision positioning information services can be provided for intelligent connected vehicles in real time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical field:

[0001] The present invention relates to the field of intelligent networked vehicles and intelligent transportation systems, and specifically to a high-precision positioning technology for intelligent networked vehicles and a roadbed visual positioning image processing algorithm and system for intelligent networked vehicles with vehicle-road collaboration. Background technology:

[0002] At present, the positioning of driverless cars in outdoor environments mainly relies on high-precision GPS, multi-line laser radar, cameras and other sensors for perception and positioning. High-precision GPS can obtain high-precision global positioning in open outdoor conditions, but there are problems of poor local positioning accuracy or loss of positioning in indoor or outdoor occluded environments, which cannot meet the positioning requirements of driverless cars.

[0003] Map matching positioning technology can also be used for the positioning of driverless cars. When the vehicle is in a structured outdoor or indoor environment, a two-dimensional grid map of the environment can be established in advance. When the vehicle is driving, the laser radar scanning point cloud data is matched with the known two-dimensional grid map to obtain high-precision positioning of the vehicle in the local environment. However, map matching requires the construction of a map of the driving environment in advance and has requirements for driving environment conditions, so it can only be used for local positioning of driverless cars. Summary of the invention:

[0004] In view of the high-precision positioning problem of intelligent connected vehicles in the prior art, the present invention proposes a method that can solve the vehicle positioning problem in a large range of complex indoor and outdoor scenes, while improving the vehicle positioning accuracy. Compared with the traditional GPS positioning method, the present invention has the advantages of a wide positioning range and high positioning accuracy. The method is based on a roadbed camera and feature point extraction and matching to provide an image processing algorithm and system for roadbed visual positioning of intelligent connected vehicles.

[0005] The present invention is achieved by the following measures:

[0006] An image processing algorithm for roadbed visual positioning of intelligent connected vehicles, characterized in that it includes the following steps:

[0007] Step 1: Read the video image of each camera at the current moment;

[0008] Step 2: Separate the foreground and background of the video information from each camera;

[0009] Step 3: Extract feature points and detect targets using a convolutional neural network for the foreground image. Use the Oriented FAST algorithm to extract feature points and detect targets using a convolutional neural network for the foreground images of each camera at the same time (time K).

[0010] Step 4: Match the feature points of each camera image; use the Steered BRIEF multi-eye matching algorithm to match the feature points of multiple images at the same time. When the image rotates, the center of mass of the image block where each key point is located will also rotate, so that the coordinate direction will rotate with the image rotation, that is, rotation invariance is achieved. The specific formula is as follows. First, define a binary operation:

[0011]

[0012] Where p(x) represents the intensity p at x, so the descriptor is a combination of the results of n binary operations, as shown below:

[0013]

[0014] For a descriptor of length n, n matching pairs are selected, defined as a 2×n matrix

[0015]

[0016] After obtaining the rotation direction of the key point, the corresponding rotation matrix is ​​obtained, and then a rotation matching pair matrix is ​​constructed.

[0017] S θ =R θ S

[0018] The descriptor formula is as follows:

[0019] g n (p,θ):=f n (p)|(x i ,y i )∈S θ

[0020] The matching of feature points is achieved by using the Hamming distance less than twice the minimum distance as the screening basis and combining the algorithm to remove false matches. After that, the internal and external parameters of the camera are calibrated to convert the successfully matched feature points from two dimensions to three dimensions, thereby realizing the three-dimensional space solution of the vehicle and forming the corresponding feature point cloud. From the pinhole imaging model, it can be seen that the conversion relationship between the pixel coordinate system and the camera coordinate system is as follows:

[0021]

[0022] f is the focal length, dx and dy are the physical dimensions of each pixel in the x and y directions of the image plane, respectively. The matrix K is the intrinsic parameter matrix of the camera, and its parameters are determined by the parameters of the camera itself. For a point in the known coordinate system in the real space, its coordinates are transferred to the camera coordinate system through rigid body transformation. The rigid body transformation includes rotation transformation and translation transformation. Let the coordinates of the point in the world coordinate system be (X W , YW , Z W ) T , the coordinates of the point in the camera coordinate system are (X C , Y C , Z C ) T The transformation relationship between the two is as follows:

[0023]

[0024] Among them, R is a 3×3 rotation matrix, t is a 3×1 translation vector, [R t] is called the camera's extrinsic parameter matrix. The camera's extrinsic parameter matrix has nothing to do with the camera's properties, and only represents the rigid body transformation of the coordinate system. The relationship between the pixel coordinate system and the world coordinate system is:

[0025]

[0026] Step 5: Fuse the detection results of the convolutional neural network on the image with the feature point cloud data to remove mismatches and flying point data;

[0027] Step 6: Perform multi-view feature point coordinate conversion based on the multi-camera feature point matching results and the internal and external parameters of the camera to obtain a three-dimensional space feature point cloud;

[0028] Step 7: Filter and track multiple targets on the fused data at multiple consecutive moments to obtain high-precision positioning information, operating status, and vehicle appearance feature information of all vehicles in the field of view;

[0029] Step 8: Obtain low-precision GPS / BD positioning information, operating status and vehicle appearance feature information of all vehicles in the field of view through communication between the vehicle network terminal and the roadbed vehicle network terminal, and use the least squares method to match all vehicle information;

[0030] Step 9: According to the matching results, the high-precision positioning information is sent to the corresponding vehicle network terminal to provide high-precision positioning services for the vehicle.

[0031] The scale invariance of the feature points in step 3 of the present invention is achieved by constructing an image pyramid and detecting corner points on each layer of the pyramid, and the rotation of the feature is achieved by the grayscale centroid method. The centroid refers to the center with the grayscale value of the image block as the weight. The specific operation steps are as follows:

[0032] Step 3-1: In a small image block B, define the moment of the image block as:

[0033]

[0034] Step 3-2: Find the centroid of the image block through moments:

[0035]

[0036] Step 3-3: Connect the geometric center O of the image block and the centroid C to get a direction vector So the direction of the feature point can be defined as:

[0037] θ=arctan(m 01 / m 10 )

[0038] The extracted feature points have descriptions of scale and rotation, which greatly improves their robustness in representing different images.

[0039] In step 2 of the present invention, the separation of the foreground and background of the video information of each camera includes: first, smoothing and denoising three consecutive frames of pictures, and then performing frame difference processing respectively, subtracting the k-1 frame of the picture from the k frame of the picture to obtain a binary image D1 (x, y), and then subtracting the k frame of the picture from the k+1 frame of the picture to obtain a binary image D2 (x, y), and finally performing an "and" operation on D1 (x, y) and D2 (x, y) to obtain a three-frame difference method image D (x, y), and the specific formula is expressed as follows:

[0040]

[0041]

[0042]

[0043] Where T is a preset threshold value, which is selected based on experience. If T is too large, the detected target may have large holes or be missed. If T is too small, a lot of noise will appear.

[0044] The present invention also proposes a roadbed visual positioning image processing system for intelligent networked vehicles, which is characterized by comprising a roadbed camera, an edge computing terminal, a roadbed vehicle network terminal and an onboard vehicle network terminal, wherein the roadbed camera comprises more than two cameras, and after the roadbed camera synchronously collects visual images at different angles in the same area, each frame of the image and the image time stamp are sent to the edge computing terminal;

[0045] The edge computing terminal runs the above-mentioned roadbed visual positioning image processing method for intelligent connected vehicles to achieve real-time positioning measurement and feature extraction of all vehicles in the field of view, and obtain high-precision vehicle information, including location, operating status, vehicle model, and appearance;

[0046] The roadbed vehicle networking terminal realizes dynamic networking with the vehicle network terminals of all intelligent connected vehicles within the field of view, and receives the real-time low-precision GPS location, operating status and vehicle appearance information of the intelligent connected vehicles;

[0047] The edge computing terminal obtains the low-precision GPS location, operating status and vehicle appearance information of all intelligent connected vehicles within the networking range from the roadbed vehicle networking terminal, and compares and matches it with all vehicle information obtained by its own visual measurement, thus achieving pairing of high-precision vehicle information and intelligent connected vehicles;

[0048] Based on the pairing results, the road-based Internet of Vehicles terminal sends the high-precision positioning information corresponding to each intelligent connected vehicle to the corresponding on-board network terminal, providing positioning services for the intelligent connected vehicles.

[0049] The present invention adopts multi-eye vision and convolutional neural network algorithms to measure the vehicle position and running status in the camera field of view in real time. With the support of 5G high-speed communication technology, it can provide high-precision positioning information services for intelligent connected vehicles in real time. Description of the drawings:

[0050] Attached Figure 1 It is a flow chart of the present invention.

[0051] Attached Figure 2 It is a system diagram of the present invention. Specific implementation method:

[0052] The present invention will be further described below in conjunction with the accompanying drawings.

[0053] The present invention proposes a roadbed visual positioning system and image processing algorithm serving intelligent connected vehicles, which includes roadbed cameras (multiple), edge computing terminals, roadbed vehicle networking terminals, and vehicle network terminals. The system is characterized in that the camera is used to obtain multi-angle image information in the same area; the edge computing terminal is used to run the core algorithm for image processing and vehicle positioning; the roadbed vehicle networking terminal and the vehicle network terminal are used for information communication in vehicle-road collaboration;

[0054] The image information used for vehicle positioning comes from multiple fixed cameras on the roadbed, and the image acquisition of the cameras is synchronously controlled by the same signal. By extracting, tracking and matching feature points of multi-angle images of the same area, the vehicle can be identified and positioned with high precision.

[0055] Edge computing is used to achieve fast image processing and high-throughput network information transmission to meet the high real-time requirements of intelligent connected vehicles;

[0056] The steps of its core algorithm to implement vehicle positioning service are as follows:

[0057] Step 1: Separate the foreground and background of the video information of each camera and extract all foreground image data;

[0058] Step 2: Extract feature points and perform convolutional neural network target detection on the foreground images of each camera at the same time (time K);

[0059] Step 3: Use a multi-image matching algorithm to match feature points of multiple images at the same time, and perform three-dimensional space solution on the successfully matched feature points to form a feature point cloud;

[0060] Step 4: Fuse the detection results of the convolutional neural network on the image with the feature point cloud data to remove mismatches and flying point data;

[0061] Step 5: Filter and track multiple targets on the fused data at multiple consecutive moments to obtain high-precision positioning information, operating status, and vehicle appearance characteristics of all vehicles in the field of view;

[0062] Step 6: Obtain low-precision GPS / BD positioning information, operating status and vehicle appearance feature information of all vehicles in the field of view through communication between the vehicle network terminal and the roadbed vehicle network terminal, and use the least squares method to match all vehicle information;

[0063] Step 7: According to the matching results, the high-precision positioning information is sent to the corresponding vehicle network terminal to provide high-precision positioning services for the vehicle.

[0064] The main computing parts of the algorithm run on edge computing terminals near the cameras. Each edge computing terminal is responsible for several cameras. All edge computing terminals are connected through a high-speed network to form a distributed edge computing system, which can provide high-precision positioning services for vehicles on the entire road section.

[0065] The visual positioning system and core algorithm described above do not require GPS / BD data. In tunnels or obstructed scenes without GPS / BD signals, the vehicle information and visual positioning system information can still be matched through vehicle speed and appearance information, providing high-precision positioning of intelligent connected vehicles.

[0066] Embodiment 1:

[0067] This example provides an image processing algorithm and system for roadbed visual positioning of intelligent connected vehicles, where the system consists of Figure 1 As shown in the figure, the positioning system mainly consists of four parts: roadbed camera, edge computing terminal, roadbed vehicle network terminal and vehicle-mounted vehicle network terminal; the basic working process of the positioning system is described as follows:

[0068] (1) Roadbed cameras synchronously collect visual images from different angles in the same area;

[0069] (2) The camera sends each frame of image and image time stamp to the edge computing terminal;

[0070] (3) The edge computing terminal runs the core algorithm proposed in the present invention to achieve real-time positioning measurement and feature extraction of all vehicles in the field of view, and obtain high-precision vehicle information, including location, operating status, vehicle model, appearance, etc.;

[0071] (4) The roadbed vehicle networking terminal realizes dynamic networking with the vehicle network terminals of all intelligent connected vehicles within the field of view, and receives the real-time low-precision GPS location, operating status, vehicle appearance and other information of the intelligent connected vehicles;

[0072] (5) The edge computing terminal obtains the low-precision GPS location, operating status, and vehicle appearance information of all intelligent connected vehicles within the networking range from the roadbed vehicle networking terminal, and compares and matches all vehicle information obtained by its own visual measurement to achieve pairing of high-precision vehicle information and intelligent connected vehicles;

[0073] (6) Based on the pairing results, the road-based vehicle networking terminal sends the high-precision positioning information corresponding to each intelligent connected vehicle to the corresponding vehicle network terminal, providing positioning services for the intelligent connected vehicles.

[0074] In the above system composition and working process, its key technical features are as follows:

[0075] (1) The overlapping area of ​​the field of view of multiple roadbed cameras is the area where the system can provide precise positioning services;

[0076] (2) The image acquisition of multiple roadbed cameras has the ability to collect images synchronously;

[0077] (3) The edge computing terminal performs multi-view visual operations on the multi-angle images at the same time to obtain feature point clouds, and matches and tracks the feature point clouds at different times to obtain high-precision three-dimensional spatial positioning;

[0078] (4) The edge computing terminal obtains the appearance characteristics and operating status of the smart vehicle through the Internet of Vehicles, and compares it with the high-precision three-dimensional positioning information obtained through the roadbed camera to achieve vehicle information matching.

[0079] In this example, the image processing algorithm of the roadbed camera runs in the edge computing terminal. The algorithm flow is as follows: Figure 2 The specific steps are as follows:

[0080] Step 1: Read the video image of each camera at the current moment;

[0081] Step 2: Separate the foreground and background of the video information of each camera. First, smooth and denoise the three consecutive frames, and then perform frame difference processing on each frame. Subtract the k-1 frame from the k-th frame to obtain a binary image D1(x, y). Then subtract the k-th frame from the k+1 frame to obtain a binary image D2(x, y). Finally, perform an "AND" operation on D1(x, y) and D2(x, y) to obtain the three-frame difference method image D(x, y). The specific formula is:

[0082]

[0083]

[0084]

[0085] Where T is a preset threshold value, which is selected based on experience. If T is too large, the detected target may have large holes or be missed. If T is too small, a lot of noise will appear.

[0086] Step 3: Extract feature points and detect targets using convolutional neural networks for the foreground image. Use the Oriented FAST algorithm to extract feature points and detect targets using convolutional neural networks for the foreground images of each camera at the same time (time K). The scale invariance of feature points is achieved by constructing an image pyramid and detecting corner points on each layer of the pyramid. The rotation of features is achieved by the grayscale centroid method. The centroid refers to the center of the image block with the grayscale value as the weight. The specific steps are as follows:

[0087] 1. In a small image block B, the moment of the image block is defined as:

[0088]

[0089] 2. The centroid of the image block can be found through the moment:

[0090]

[0091] 3. Connect the geometric center O of the image block and the center of mass C to get a direction vector So the direction of the feature point can be defined as:

[0092] θ=arctan(m 01 / m 10 )

[0093] The feature points extracted by the above method have descriptions of scale and rotation, which greatly improves their robustness in representation between different images.

[0094] Step 4: Match the feature points of each camera image; use the Steered BRIEF multi-view matching algorithm to match the feature points of multiple images at the same time. When the image rotates, the center of mass of the image block where each key point is located will also rotate, so that the coordinate direction will rotate with the image rotation, that is, rotation invariance is achieved. The specific formula is as follows. First, define a binary operation:

[0095]

[0096] Where p(x) represents the intensity p at x, so the descriptor is a combination of the results of n binary operations, as shown below:

[0097]

[0098] For a descriptor of length n, we need to select n matching pairs, defined as a 2×n matrix

[0099]

[0100] After obtaining the rotation direction of the key point, the corresponding rotation matrix can be obtained, and then a rotation matching pair matrix is ​​constructed.

[0101] S θ =R θ S

[0102] The descriptor formula is as follows:

[0103] g n (p,θ):=f n (p)|(x i ,y i )∈S θ

[0104] The matching of feature points is achieved by using the Hamming distance less than twice the minimum distance as the screening basis and combining the algorithm to remove false matches. After that, the internal and external parameters of the camera are calibrated to convert the successfully matched feature points from two dimensions to three dimensions, thereby realizing the three-dimensional space solution of the vehicle and forming the corresponding feature point cloud. From the pinhole imaging model, it can be seen that the conversion relationship between the pixel coordinate system and the camera coordinate system is as follows:

[0105]

[0106] f is the focal length, dx and dy are the physical sizes of each pixel in the x and y directions of the image plane, respectively. The matrix K is the intrinsic parameter matrix of the camera, and its parameters are determined by the parameters of the camera itself.

[0107] For a point in a known coordinate system (world coordinate system) in real space, its coordinates can be transferred to the camera coordinate system through rigid body transformation, which includes rotation transformation and translation transformation. Let the coordinates of the point in the world coordinate system be (X W , Y W , Z W ) T , the coordinates of the point in the camera coordinate system are (X C , Y C , Z C ) T The transformation relationship between the two is as follows.

[0108]

[0109] Among them, R is a 3x3 rotation matrix, t is a 3x1 translation vector, [R t] is called the camera's extrinsic matrix. The camera's extrinsic matrix has nothing to do with the camera's properties, it only represents the rigid body transformation of the coordinate system, then the relationship between the pixel coordinate system and the world coordinate system is:

[0110]

[0111] Step 5: Fuse the detection results of the convolutional neural network on the image with the feature point cloud data to remove mismatches and flying point data;

[0112] Step 6: Perform multi-view feature point coordinate conversion based on the multi-camera feature point matching results and the internal and external parameters of the camera to obtain a three-dimensional space feature point cloud;

[0113] Step 7: Filter and track multiple targets on the fused data at multiple consecutive moments to obtain high-precision positioning information, operating status, and vehicle appearance characteristics of all vehicles in the field of view;

[0114] Step 8: Obtain low-precision GPS / BD positioning information, operating status and vehicle appearance feature information of all vehicles in the field of view through communication between the vehicle network terminal and the roadbed vehicle network terminal, and use the least squares method to match all vehicle information;

[0115] Step 9: According to the matching results, the high-precision positioning information is sent to the corresponding vehicle network terminal to provide high-precision positioning services for the vehicle.

[0116] The high-precision vehicle positioning system based on roadbed cameras proposed in this invention adopts multi-eye vision and convolutional neural network algorithms to measure the vehicle position and running status in the camera field of view in real time. With the support of high-speed communication technology of 5G Internet of Vehicles, it can provide high-precision positioning information services for intelligent connected vehicles in real time.

Claims

1. A roadbed visual positioning image processing algorithm for intelligent connected vehicles, characterized in that: The following steps are involved: Step 1: Read the video image of each camera at the current moment; Step 2: Separate the foreground and background of the video information from each camera; Step 3: Extract feature points and detect targets using a convolutional neural network for the foreground image. Use the Oriented FAST algorithm to extract feature points and detect targets using a convolutional neural network for the foreground images of each camera at the same time (time K). Step 4: Match the feature points of each camera image; use the Steered BRIEF multi-eye matching algorithm to match the feature points of multiple images at the same time. When the image rotates, the center of mass of the image block where each key point is located will also rotate, so that the coordinate direction will rotate with the image rotation, that is, rotation invariance is achieved. The specific formula is as follows. First, define a binary operation: Where p(x) represents the intensity p at x, so the descriptor is a combination of the results of n binary operations, as shown below: For a descriptor of length n, n matching pairs are selected, defined as a 2×n matrix After obtaining the rotation direction of the key point, the corresponding rotation matrix is ​​obtained, and then a rotation matching pair matrix is ​​constructed. S θ =R θ S The descriptor formula is as follows: g n (p,θ):=f n (p)|(x i ,y i )∈S θ The matching of feature points is achieved by using the Hamming distance less than twice the minimum distance as the screening basis and combining it with an algorithm to remove false matches. After that, the internal and external parameters of the camera are calibrated and the successfully matched feature points are converted from two dimensions to three dimensions, thereby solving the three-dimensional space of the vehicle and forming the corresponding feature point cloud. From the pinhole imaging model, we can know that the transformation relationship between the pixel coordinate system and the camera coordinate system is as follows: f is the focal length, dx and dy are the physical dimensions of each pixel in the x and y directions of the image plane, respectively. The matrix K is the intrinsic parameter matrix of the camera, and its parameters are determined by the parameters of the camera itself. For a point in the known coordinate system in the real space, its coordinates are transferred to the camera coordinate system through rigid body transformation. The rigid body transformation includes rotation transformation and translation transformation. Let the coordinates of the point in the world coordinate system be (X W , Y W , Z W ) T , the coordinates of the point in the camera coordinate system are (X C , Y C , Z C ) T The transformation relationship between the two is as follows: Among them, R is a 3×3 rotation matrix, t is a 3×1 translation vector, [R t] is called the camera's extrinsic parameter matrix. The camera's extrinsic parameter matrix has nothing to do with the camera's properties, and only represents the rigid body transformation of the coordinate system. The relationship between the pixel coordinate system and the world coordinate system is: Step 5: Fuse the detection results of the convolutional neural network on the image with the feature point cloud data to remove mismatches and flying point data; Step 6: Perform multi-view feature point coordinate conversion based on the multi-camera feature point matching results and the internal and external parameters of the camera to obtain a three-dimensional space feature point cloud; Step 7: Filter and track multiple targets on the fused data at multiple consecutive moments to obtain high-precision positioning information, operating status, and vehicle appearance feature information of all vehicles in the field of view; Step 8: Obtain low-precision GPS / BD positioning information, operating status and vehicle appearance feature information of all vehicles in the field of view through communication between the vehicle network terminal and the roadbed vehicle network terminal, and use the least squares method to match all vehicle information; Step 9: According to the matching results, the high-precision positioning information is sent to the corresponding vehicle network terminal to provide high-precision positioning services for the vehicle.

2. The image processing algorithm for roadbed visual positioning of intelligent connected vehicles according to claim 1 is characterized in that: The scale invariance of the feature points in step 3 is achieved by constructing an image pyramid and detecting corner points on each layer of the pyramid, while the rotation of the feature is achieved by the grayscale centroid method. The centroid refers to the center with the grayscale value of the image block as the weight. The specific operation steps are as follows: Step 3-1: In a small image block B, define the moment of the image block as: Step 3-2: Find the centroid of the image block through moments: Step 3-3: Connect the geometric center O of the image block and the centroid C to get a direction vector So the direction of the feature point can be defined as: θ=arctan(m 01 / m 10 ) The extracted feature points have descriptions of scale and rotation, which greatly improves their robustness in representing different images.

3. The image processing algorithm for roadbed visual positioning of intelligent connected vehicles according to claim 1, characterized in that: In step 2, the foreground and background separation of the video information of each camera includes: first, smoothing and denoising three consecutive frames of pictures, and then performing frame difference processing respectively, subtracting the k-1 frame from the k frame to obtain a binary image D1(x, y), and then subtracting the k frame from the k+1 frame to obtain a binary image D2(x, y), and finally performing an "AND" operation on D1(x, y) and D2(x, y) to obtain a three-frame difference method image D(x, y). The specific formula is expressed as follows: Where T is a preset threshold value, which is selected based on experience. If T is too large, the detected target may have large holes or be missed. If T is too small, a lot of noise will appear.

4. A roadbed visual positioning image processing system for intelligent connected vehicles, characterized in that: It is equipped with a roadbed camera, an edge computing terminal, a roadbed vehicle network terminal and an onboard vehicle network terminal. The roadbed camera contains more than two cameras. After the roadbed camera synchronously collects visual images from different angles in the same area, it sends each frame of the image and the image time mark to the edge computing terminal; The edge computing terminal runs the roadbed visual positioning image processing method for intelligent connected vehicles as described in any one of claims 1 to 3, realizes real-time positioning measurement and feature extraction of all vehicles in the field of view, and obtains high-precision vehicle information, including location, operating status, vehicle model, and appearance; The roadbed vehicle networking terminal realizes dynamic networking with the vehicle network terminals of all intelligent connected vehicles within the field of view, and receives the real-time low-precision GPS location, operating status and vehicle appearance information of the intelligent connected vehicles; The edge computing terminal obtains the low-precision GPS location, operating status and vehicle appearance information of all intelligent connected vehicles within the networking range from the roadbed vehicle networking terminal, and compares and matches it with all vehicle information obtained by its own visual measurement, thus achieving pairing of high-precision vehicle information and intelligent connected vehicles; Based on the pairing results, the road-based Internet of Vehicles terminal sends the high-precision positioning information corresponding to each intelligent connected vehicle to the corresponding on-board network terminal, providing positioning services for the intelligent connected vehicles.

Citation Information

Patent Citations

  • A vehicle feature extraction and matching method based on video

    CN109409208A

  • In-vehicle circular image and millimeter-wave radar fusion-based intelligent vehicle positioning method

    CN110609311A