A method, system and electronic device for three-dimensional matching of wheel trajectories

Abstract

This invention relates to the field of vehicle assisted driving technology, and discloses a method, system, and electronic device for three-dimensional matching of wheel trajectories. The method includes: acquiring a first image and a second image with a fixed spatial positional relationship, which are synchronously acquired by an image acquisition device; acquiring a current frame image from the first image or the second image; estimating an initial wheel trajectory based on the vanishing point in the current frame image and the vehicle's inherent parameters; iteratively updating the trajectory parameters by combining real-time steering wheel angle and vehicle speed; determining the region of interest and performing feature extraction and matching on the image within the region; optimizing through cost aggregation and extracting a single disparity value along the trajectory path. This method solves the problems of high computational cost, difficult deployment, and difficulty in adapting to dynamic changes in vehicles with fixed regions of interest in large-scale deep learning networks, thereby improving the accuracy and real-time performance of wheel trajectory tracking and providing a more reliable perception basis for trajectory prediction and active control of vehicle assisted driving systems.

Description

Technical Field

[0001] This invention relates to the field of vehicle driver assistance technology, and in particular to a method, system and electronic device for three-dimensional wheel trajectory matching. Background Technology

[0002] With the rapid development of deep learning technology, stereo matching networks have been widely used in the field of 3D vision. However, existing large-scale deep learning stereo matching networks face many challenges in practical applications. These networks typically contain millions of parameters and rely on high-performance GPU clusters or dedicated AI acceleration hardware to complete training and inference, resulting in high overall solution costs and difficulty in meeting deployment requirements in scenarios with high real-time demands. Especially for the magic carpet function in vehicle driver assistance systems, the core focus is on the accurate tracking of wheel trajectories, rather than large-area parallax matching or complex 3D reconstruction of the entire field of view. Using traditional large-scale deep learning networks to handle such tasks not only results in a huge waste of computing resources but also further increases system development costs and timelines due to the need for large amounts of expensive and time-consuming labeled data for training.

[0003] During vehicle operation, the determination of wheel trajectories is influenced by multiple dynamic factors, including steering wheel angle, vehicle speed, and ground surface conditions. This necessitates dynamic adjustment of the region of interest (ROI) to accurately capture the actual wheel trajectory. Current technologies generally employ fixed ROIs for wheel trajectory monitoring, but this method exhibits significant limitations in dynamically changing environments. When a vehicle travels on uneven surfaces, the vertical displacement of the wheels due to vibrations easily exceeds the preset monitoring range; similarly, during cornering, lateral deviations in the wheel trajectory can cause the target to deviate from the fixed ROI. This inflexible monitoring method cannot adapt to the complex changes in real-world driving scenarios, severely restricting the reliability and accuracy of trajectory monitoring.

[0004] Traditional wheel trajectory extraction methods heavily rely on disparity information. They obtain slope and intercept parameters through 3D ground fitting, then calculate the disparity value corresponding to each Y-coordinate, and combine this with steering wheel angle and vehicle speed information to infer the lateral translation of the trajectory. While this process can achieve trajectory tracking under ideal conditions, it faces fundamental technical bottlenecks in scenarios with no or weak disparity. When reliable depth information cannot be obtained through conventional stereo matching, ground geometry parameters are difficult to construct effectively, and wheel position prediction loses its crucial basis, leading to trajectory extraction failure. Such situations necessitate exploring alternative technical paths, such as fusing data from other sensors or introducing lightweight deep learning models, to overcome the limitations of traditional methods in disparity-free conditions.

[0005] In summary, existing technical solutions are insufficient in terms of cost-effectiveness, dynamic adaptability, and robustness in special scenarios, making it difficult to meet the actual engineering needs of wheel trajectory tracking. Therefore, there is an urgent need to provide a technical solution to address these issues. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method, system, and electronic device for three-dimensional matching of wheel trajectories.

[0007] Firstly, the present invention provides a method for three-dimensional matching of wheel trajectories, the technical solution of which is as follows: Acquire a first image and a second image that have a fixed spatial positional relationship, which are synchronously acquired by an image acquisition device; Obtain the current frame image of the vehicle from the first image or the second image, and estimate the initial wheel trajectory parameters based on the vanishing point in the current frame image and the vehicle's inherent parameters; Based on the current steering wheel angle and current vehicle speed, the initial wheel trajectory parameters are updated to obtain the updated wheel trajectory parameters corresponding to the current frame; Based on the updated wheel trajectory parameters and the current steering wheel angle, determine the dynamic region of interest in the current frame image; Based on the first image and the second image, the corresponding features in the dynamic region of interest are extracted respectively to obtain the first feature map and the second feature map; Translate any one of the first and second feature maps along the horizontal direction of the image and match it with the other feature map, then calculate the matching cost. Cost aggregation is performed on the matching costs, and a single disparity value is extracted along the wheel trajectory path within the dynamic region of interest based on the aggregation results. Wheel trajectory information is output based on a single disparity value.

[0008] The beneficial effects of the wheel trajectory 3D matching method of the present invention are as follows: The method of this invention estimates the initial wheel trajectory based on the vanishing point and the vehicle's inherent parameters, and dynamically updates the region of interest by combining the steering wheel angle and vehicle speed. Within the region, feature matching and cost aggregation are performed to extract a single disparity value. This solves the problems of high computational cost, difficult deployment, and difficulty in adapting a fixed region of interest to dynamic changes in the vehicle, which are common in large-scale deep learning networks. It improves the accuracy and real-time performance of wheel trajectory tracking, thereby providing a more reliable perception basis for trajectory prediction and active control of vehicle assisted driving systems.

[0009] Based on the above scheme, the wheel trajectory three-dimensional matching method of the present invention can be further improved as follows.

[0010] In one alternative approach, the vehicle's inherent parameters include: vehicle wheelbase and vehicle track width; the step of estimating initial wheel trajectory parameters based on the vanishing point in the current frame image and the vehicle's inherent parameters includes: Detect straight road lines parallel to the vehicle's direction of travel from the current frame image; Calculate the coordinates of the intersection point of the straight roads, and determine the intersection point coordinates as the vanishing point; The initial wheel trajectory parameters are calculated based on the coordinates of the vanishing point, the vehicle wheelbase, and the vehicle track width.

[0011] The advantages of adopting the above-mentioned optional method are: further determining the vanishing point by detecting road straight lines, and calculating the initial wheel trajectory parameters by combining the vehicle wheelbase and track width, so that the trajectory estimation is based on the vehicle's geometric characteristics and road perspective rules, thereby improving the accuracy and environmental adaptability of the initial trajectory parameters.

[0012] In one alternative approach, updating the wheel trajectory parameters includes: updating the trajectory slope and updating the trajectory intercept; and updating the initial wheel trajectory parameters based on the current steering wheel angle and current vehicle speed to obtain the updated wheel trajectory parameters for the current frame, including: Based on the pre-established mapping relationship, the slope correction and intercept correction are obtained by querying the current steering wheel angle and current vehicle speed. The initial trajectory slope and the slope correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory slope, and the initial trajectory intercept and the intercept correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory intercept.

[0013] The advantages of adopting the above optional method are: further, by directly querying the slope intercept correction amount through the pre-established mapping relationship between steering wheel angle and vehicle speed, the initial trajectory parameters are added to the correction amount to obtain the updated parameters, thereby realizing the rapid correlation between dynamic driving state and trajectory geometric changes and enhancing real-time response capability.

[0014] In one alternative approach, the step of determining the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle includes: The basic trajectory line of the wheel in the current frame image is determined based on the updated trajectory slope and the updated trajectory intercept. Calculate the lateral offset of the basic trajectory line based on the current steering wheel angle; Based on the lateral offset, determine the lateral boundary of the wheel corresponding to the preset vertical position in the current frame image; Based on the installation parameters of the image acquisition device and the vertical resolution of the current frame image, calculate the minimum and maximum longitudinal coordinates of the area where the wheel is located in the current frame image. The region of interest is determined based on the minimum and maximum longitudinal coordinates and the lateral boundary of the wheel.

[0015] The advantages of using the above-mentioned optional method are as follows: further calculate the lateral offset based on the updated trajectory parameters and steering wheel angle, determine the longitudinal coordinate range in combination with the image acquisition device installation parameters, generate a dynamic region of interest including the wheel boundary, and make the monitoring range adaptively adjust with vehicle movement and camera attitude.

[0016] In one alternative approach, the step of translating either the first feature map or the second feature map along the horizontal direction of the image and matching it with the other feature map, and calculating the matching cost, includes: Within the dynamic region of interest, any feature map in the first and second feature maps is sequentially translated along the horizontal direction of the image according to multiple candidate disparity values. At each candidate disparity value, the translated feature map is compared pixel-by-pixel with the corresponding feature of another feature map within the dynamic region of interest. Based on the results of pixel-by-pixel comparisons, the matching cost corresponding to each candidate disparity value is calculated.

[0017] The advantages of adopting the above optional method are: further translating the feature map sequentially according to the candidate disparity value within the dynamic region of interest, and comparing it with another feature map pixel by pixel to calculate the matching cost, thus limiting the stereo matching calculation to the region related to the wheel trajectory and reducing redundant computational overhead.

[0018] In one alternative approach, the steps of cost aggregation of the matching costs and extraction of a single disparity value along the wheel trajectory path within the dynamically interested region based on the aggregation result include: The matching cost corresponding to each candidate disparity value is aggregated to obtain the aggregated cost corresponding to each candidate disparity value. Within the dynamic region of interest, the aggregate cost corresponding to each candidate disparity value is compared along the wheel trajectory path, and the candidate disparity value with the smallest aggregate cost is determined as the single disparity value.

[0019] The advantages of adopting the above optional method are: further aggregating the matching cost of each candidate disparity value, comparing the aggregated cost along the wheel trajectory path and selecting the minimum value as a single disparity value, optimizing disparity selection under trajectory continuity constraints, and improving the stability of disparity estimation.

[0020] In one alternative approach, the three-dimensional coordinates of multiple trajectory points on the wheel trajectory path are calculated based on a single parallax value and the calibration parameters of the image acquisition device. The wheel trajectory information is generated and output based on the three-dimensional coordinates of multiple trajectory points.

[0021] The advantages of adopting the above-mentioned optional method are as follows: by further utilizing a single parallax value in combination with the calibration parameters of the image acquisition device to calculate the three-dimensional coordinates of the trajectory point and generate wheel trajectory information, the pixel-level parallax is converted into physical space trajectory data, providing directly usable spatial position information for vehicle control.

[0022] In one alternative approach, it also includes: Based on the updated trajectory slope, updated trajectory intercept, and current steering wheel angle, calculate the image coordinates of multiple points within the dynamic region of interest; Generate wheel trajectory paths based on image coordinates of multiple location points.

[0023] The advantages of using the above optional method are: further calculating the image coordinates of multiple position points in the dynamic region of interest based on the updated trajectory parameters and steering wheel angle, and generating the wheel trajectory path, providing an accurate trajectory reference for subsequent matching and disparity extraction.

[0024] Secondly, the present invention provides a three-dimensional wheel trajectory matching system, the technical solution of which is as follows: The acquisition module is used to acquire a first image and a second image that have a fixed spatial positional relationship, which are synchronously acquired by the image acquisition device; The estimation module is used to obtain the current frame image of the vehicle from the first image or the second image, and estimate the initial wheel trajectory parameters based on the vanishing point in the current frame image and the inherent parameters of the vehicle. The update module is used to update the initial wheel trajectory parameters based on the current steering wheel angle and the current vehicle speed, so as to obtain the updated wheel trajectory parameters corresponding to the current frame. The determination module is used to determine the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle; The extraction module is used to extract features corresponding to the dynamic region of interest based on the first image and the second image, respectively, to obtain a first feature map and a second feature map; The calculation module is used to translate any one of the first and second feature maps along the horizontal direction of the image and match it with the other feature map, and calculate the matching cost. The aggregation module is used to aggregate the matching costs and extract a single disparity value along the wheel trajectory path in the dynamic region of interest based on the aggregation results. The output module is used to output wheel trajectory information based on a single parallax value.

[0025] The beneficial effects of the wheel trajectory three-dimensional matching system of the present invention are as follows: The system of this invention estimates the initial wheel trajectory based on the vanishing point and the vehicle's inherent parameters, and dynamically updates the region of interest by combining the steering wheel angle and vehicle speed. It then performs feature matching and cost aggregation within the region to extract a single disparity value. This solves the problems of high computational cost, difficult deployment, and the inability of a fixed region of interest to adapt to dynamic changes in the vehicle, which are common in large-scale deep learning networks. As a result, it improves the accuracy and real-time performance of wheel trajectory tracking, thus providing a more reliable perception basis for trajectory prediction and active control of vehicle assisted driving systems.

[0026] Thirdly, the technical solution of an electronic device according to the present invention is as follows: It includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps of the wheel trajectory stereo matching method of the present invention.

[0027] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0028] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a three-dimensional wheel trajectory matching method according to the present invention; Figure 2 A flowchart illustrating the process of stereo matching for a network model; Figure 3 This is a schematic diagram of an embodiment of a wheel trajectory three-dimensional matching system of the present invention; Figure 4 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0029] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0030] Figure 1This diagram illustrates a flowchart of an embodiment of a 3D wheel trajectory matching method provided by the present invention. This method can be executed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal, such as a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the 3D wheel trajectory matching method by having its processor call computer-readable instructions stored in its memory. Figure 1 As shown, it includes the following steps: S1. Acquire a first image and a second image that have a fixed spatial positional relationship, which are synchronously acquired by the image acquisition device.

[0031] In this context, the image acquisition device refers to a device capable of simultaneously acquiring at least two images and maintaining their relative positions. In this embodiment, it is assumed to be a stereo camera or a binocular camera. A fixed spatial position relationship means that the optical centers of the corresponding acquisition devices for the two images maintain a constant relative spatial position during acquisition; for example, the optical centers of the two cameras in a binocular camera device maintain a fixed distance in the horizontal direction. The first image refers to one of a pair of images simultaneously acquired by the image acquisition device; for example, a road image with a resolution of 1280x720 pixels acquired by one camera in a binocular camera. The second image refers to the other image simultaneously acquired by the image acquisition device, which is paired with the first image and has a fixed spatial position relationship; for example, a road image with the same resolution of 1280x720 pixels acquired at the same time by another camera in the same binocular camera.

[0032] S2. Obtain the current frame image of the vehicle from the first image or the second image, or from the left image or the right image, and estimate the initial wheel trajectory parameters based on the vanishing point and the inherent parameters of the vehicle in the current frame image.

[0033] The current frame image refers to the single image selected from the first or second image for processing at the current moment; for example, the first image captured by the stereo camera is selected as the image for which trajectory parameter estimation is needed. The vanishing point refers to the intersection point of lines representing the directions of parallel straight lines in space within the image plane; for example, by detecting two parallel lane lines in the current frame image using Hough transform and calculating their intersection point, a pixel with coordinates (640, 180) is obtained. Vehicle intrinsic parameters refer to parameters describing the fixed geometric dimensions of the vehicle itself; for example, the vehicle's wheelbase is 2.8m and its track width is 1.6m. Initial wheel trajectory parameters refer to parameters describing the initial geometric characteristics of the wheel trajectory, estimated based on the vanishing point of the first frame image and the vehicle intrinsic parameters; for example, the calculated initial trajectory slope k0 is 0.15, and the initial trajectory intercept b0 is 200 pixels.

[0034] S3. Based on the current steering wheel angle and current vehicle speed, update the initial wheel trajectory parameters to obtain the updated wheel trajectory parameters corresponding to the current frame.

[0035] The current steering wheel angle refers to the real-time rotation angle of the vehicle's steering wheel relative to the center of the straight-line position, measured in degrees; for example, a steering wheel angle of 15 degrees. The current vehicle speed refers to the vehicle's current speed, measured in km / h; for example, a speed of 30 km / h. Updating wheel trajectory parameters refers to the new parameters obtained after correcting the initial wheel trajectory parameters based on the current steering wheel angle and current vehicle speed; for example, updating the trajectory slope k. n The value is 0.17, so the trajectory intercept b is updated. n It is 210 pixels.

[0036] S4. Determine the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle.

[0037] The dynamic region of interest (ROI) refers to a region in the image that is dynamically determined based on the updated wheel trajectory parameters and the current steering wheel angle for subsequent stereo matching calculations; for example, a region with a vertical range from y=300 pixels to y=500 pixels and a horizontal range that varies with the y value.

[0038] S5. Based on the first image and the second image, extract the corresponding features in the dynamic region of interest respectively to obtain the first feature map and the second feature map.

[0039] The first feature map refers to a high-dimensional feature representation obtained after performing convolutional neural network feature extraction on the first image; for example, a feature tensor with a size of 160x90 pixels and containing 256 channels output after inputting the first image into the network. The second feature map refers to a high-dimensional feature representation obtained after performing convolutional neural network feature extraction on the second image; for example, a feature tensor with the same size and number of channels output after inputting the second image into the same network. Figure 1 The characteristic tensor of the given form.

[0040] S6. Shift any one of the first and second feature maps along the horizontal direction of the image and match it with the other feature map, and calculate the matching cost.

[0041] Here, the horizontal direction of the image refers to the x-axis direction parallel to the image rows in the image pixel coordinate system. The matching cost is a numerical value used in stereo matching to measure the similarity between the feature map of the left image and the translated feature map of the right image; for example, a value of 45.3 obtained by calculating the sum of the absolute differences of the feature vectors.

[0042] S7. Perform cost aggregation on the matching cost, and extract a single disparity value along the wheel trajectory path in the dynamic region of interest based on the aggregation result.

[0043] Cost aggregation refers to the process of spatially smoothing or contextually optimizing the initial matching cost to reduce noise and incorrect matches. The aggregation result refers to the optimized cost data obtained after the cost aggregation step; for example, the smoothed cost value corresponding to each candidate disparity value at each pixel position. The wheel trajectory path refers to an image coordinate sequence or curve representing the expected path of the wheel center within the dynamic region of interest; for example, a path formed by connecting point coordinates (160, 400), (165, 410), etc. A single disparity value refers to the optimal disparity integer value determined through matching and aggregation on the wheel trajectory path; for example, a disparity value of 25 pixels.

[0044] S8. Output wheel trajectory information based on a single disparity value.

[0045] Among them, wheel trajectory information refers to the final output data that describes the trajectory of the wheel in three-dimensional space; for example, it includes a sequence of three-dimensional coordinate points (1.5, 10.0, 0.0) and (1.6, 10.5, 0.0).

[0046] The technical solution of this embodiment estimates the initial wheel trajectory based on the vanishing point and the vehicle's inherent parameters, and dynamically updates the region of interest by combining the steering wheel angle and vehicle speed. Feature matching and cost aggregation are performed within the region to extract a single disparity value. This solves the problems of high computational cost, difficult deployment, and difficulty in adapting a fixed region of interest to dynamic changes in the vehicle, which are common in large-scale deep learning networks. It improves the accuracy and real-time performance of wheel trajectory tracking, thereby providing a more reliable perception basis for trajectory prediction and active control of vehicle assisted driving systems.

[0047] In one alternative approach, the vehicle's inherent parameters include: vehicle wheelbase and vehicle track width; the step of estimating initial wheel trajectory parameters based on the vanishing point in the current frame image and the vehicle's inherent parameters includes: Use either the left or right image as the current frame image.

[0048] Detect straight road lines parallel to the vehicle's direction of travel from the current frame image.

[0049] The vehicle's direction of travel refers to the direction parallel to road structures such as lane lines in the image, representing the direction the vehicle is moving. A straight road line refers to a linear road feature detected in the image that is parallel to the vehicle's direction of travel; for example, a straight line representing the edge of a lane.

[0050] Calculate the coordinates of the intersection point of the straight roads, and determine the intersection point coordinates as the vanishing point.

[0051] The intersection point coordinates refer to the coordinates of the intersection point of two or more road straight lines in the image pixel coordinate system; for example, the coordinates are (640, 180).

[0052] The initial wheel trajectory parameters are calculated based on the coordinates of the vanishing point, the vehicle wheelbase, and the vehicle track width.

[0053] Specifically, based on the coordinates of the vanishing point, the straight line passing through the vanishing point and parallel to the vertical direction of the image is determined as the vehicle's driving center line; the fixed lateral offset distance of the left and right wheel trajectory lines relative to the vehicle's driving center line is calculated according to half of the vehicle's wheelbase; and combined with the perspective ratio reflected by the vehicle's wheelbase, the equations of two straight lines representing the left and right wheel trajectories in the image plane are derived, thereby obtaining the initial wheel trajectory parameters.

[0054] In the above-mentioned optional methods, the vanishing point is further determined by detecting straight road lines, and the initial wheel trajectory parameters are calculated by combining the vehicle wheelbase and track width. This allows the trajectory estimation to be based on the vehicle's geometric characteristics and the road perspective rules, thereby improving the accuracy and environmental adaptability of the initial trajectory parameters.

[0055] In one alternative approach, updating the wheel trajectory parameters includes: updating the trajectory slope and updating the trajectory intercept; S3 specifically includes: Based on the pre-established mapping relationship, the slope correction and intercept correction are obtained by querying the current steering wheel angle and current vehicle speed.

[0056] The pre-established mapping relationship refers to the correspondence between steering wheel angle and vehicle speed and trajectory parameter corrections, established through experimental calibration; for example, a two-dimensional lookup table with input (15 degrees, 30 km / h) and output (0.02, 10 pixels). The slope correction is the incremental value used to adjust the slope of the wheel trajectory line, denoted as Δk; for example, Δk is 0.02. The intercept correction is the incremental value used to adjust the intercept of the wheel trajectory line, denoted as Δb; for example, Δb is 10 pixels.

[0057] The initial trajectory slope and the slope correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory slope, and the initial trajectory intercept and the intercept correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory intercept.

[0058] The initial trajectory slope refers to the slope parameter in the initial wheel trajectory parameters, denoted as k0; for example, k0 is 0.15. The initial trajectory intercept refers to the intercept parameter in the initial wheel trajectory parameters, denoted as b0; for example, b0 is 200 pixels. The updated trajectory slope refers to the slope parameter in the updated wheel trajectory parameters, denoted as k... n For example, k n The value is 0.17. Updating the trajectory intercept refers to updating the intercept parameter in the wheel trajectory parameters, denoted as b. n For example, b n It is 210 pixels.

[0059] In the above-mentioned optional methods, the slope intercept correction amount can be directly queried through the pre-established mapping relationship between steering wheel angle and vehicle speed. The initial trajectory parameters are added to the correction amount to obtain the updated parameters, thereby realizing the rapid correlation between dynamic driving state and trajectory geometric changes and enhancing real-time response capability.

[0060] In one alternative approach, S4 specifically includes: The basic trajectory line of the wheel in the current frame image is determined based on the updated trajectory slope and the updated trajectory intercept.

[0061] The basic trajectory line refers to the straight line determined in the image based on the updated trajectory slope kn and the updated trajectory intercept bn, which represents the theoretical center line of the wheel.

[0062] Calculate the lateral offset of the basic trajectory line based on the current steering wheel angle.

[0063] The lateral offset refers to the amount of lateral movement of the wheel trajectory relative to the basic trajectory line caused by the steering wheel angle, denoted as Δx; for example, Δx is 20 pixels.

[0064] Based on the lateral offset, determine the lateral boundary of the wheel corresponding to the preset vertical position in the current frame image.

[0065] The weight, or preset vertical position, refers to a pre-selected set of y-coordinate values ​​along the vertical axis of the current frame image used to calculate the wheel boundary; for example, a position with y-coordinate values ​​of [300, 350, 400, 450, 500] pixels. The wheel lateral boundary refers to the range of lateral coordinates corresponding to the wheel trajectory area on the image at a given vertical position. In this embodiment, the wheel lateral boundaries include: the left lateral left boundary, the left lateral right boundary, the right lateral left boundary, and the right lateral right boundary.

[0066] For example, at y=400 pixels, the left wheel is on the left lateral boundary. 150 pixels, left wheel horizontal right edge 170 pixels, right wheel horizontal left edge The value is 210 pixels, and the right wheel's horizontal right edge is [not specified]. It is 230 pixels.

[0067] Based on the installation parameters of the image acquisition device and the vertical resolution of the current frame image, calculate the minimum and maximum longitudinal coordinates of the area where the wheel is located in the current frame image.

[0068] The installation parameters refer to the external geometric parameters describing the installation status of the image acquisition device in physical space; for example, installation parameters include installation height h = 1.2m, installation angle θ = 30 degrees, and vertical viewing angle α = 60 degrees. Vertical resolution refers to the number of pixels in the vertical direction of the image, denoted as H; for example, H is 720 pixels. The minimum vertical coordinate refers to the y-coordinate of the highest point of the wheel trajectory visible in the image, calculated based on the camera's geometric parameters, denoted as y0. min For example, y min The value is 300 pixels. The maximum vertical coordinate refers to the y-coordinate of the lowest point of the wheel trajectory within the visible range of the image, calculated based on the camera's geometric parameters. max For example, y max It is 500 pixels.

[0069] The region of interest is determined based on the minimum and maximum longitudinal coordinates and the lateral boundary of the wheel.

[0070] Specifically, the upper and lower boundaries of the dynamic region of interest in the image are determined based on the minimum and maximum vertical coordinates. The left and right boundaries of the dynamic region of interest at each preset vertical position in the image are determined based on the left and right horizontal boundaries of the left wheel, the right wheel, and the right wheel. These upper and lower boundaries are combined with the left and right boundaries corresponding to each vertical position to delineate and determine a rectangular region in the image whose width varies with the vertical position as the dynamic region of interest.

[0071] In the above-mentioned optional methods, the lateral offset is further calculated based on the updated trajectory parameters and steering wheel angle, and the longitudinal coordinate range is determined in combination with the installation parameters of the image acquisition device to generate a dynamic region of interest that includes the wheel boundary, so that the monitoring range can be adaptively adjusted with the vehicle movement and camera attitude.

[0072] In one alternative approach, S6 specifically includes: Within the dynamic region of interest, any feature map in the first and second feature maps is sequentially translated along the horizontal direction of the image according to multiple candidate disparity values.

[0073] Among them, multiple candidate disparity values ​​refer to a series of possible integer disparity values ​​preset during the stereo matching process, which constitute the search range; for example, the candidate disparity value is [0,1,2,…,50] pixels.

[0074] At each candidate disparity value, the translated feature map is compared pixel-by-pixel with the corresponding feature of another feature map within the dynamic region of interest.

[0075] Pixel-by-pixel comparison refers to the operation of comparing the feature vectors of each pixel that corresponds strictly to a spatial location on two feature maps one by one during the matching cost calculation.

[0076] Based on the results of pixel-by-pixel comparisons, the matching cost corresponding to each candidate disparity value is calculated.

[0077] The matching cost corresponding to the candidate disparity value refers to the total matching cost calculated within the dynamic region of interest for a specific candidate disparity value.

[0078] In the above-mentioned optional approach, the feature map is further translated sequentially according to the candidate disparity value within the dynamic region of interest, and the matching cost is calculated pixel by pixel by comparing it with another feature map. This limits the stereo matching calculation to the region related to the wheel trajectory and reduces redundant computational overhead.

[0079] In one alternative approach, S7 specifically includes: The matching cost corresponding to each candidate disparity value is aggregated to obtain the aggregated cost corresponding to each candidate disparity value.

[0080] The aggregation cost corresponding to the candidate disparity value refers to the new generation value obtained after optimizing the matching cost through the cost aggregation step for a specific candidate disparity value.

[0081] Within the dynamic region of interest, the aggregate cost corresponding to each candidate disparity value is compared along the wheel trajectory path, and the candidate disparity value with the smallest aggregate cost is determined as the single disparity value.

[0082] In the above-mentioned optional methods, the matching cost of each candidate disparity value is further aggregated, the aggregated cost is compared along the wheel trajectory path, and the minimum value is selected as the single disparity value. Under the trajectory continuity constraint, the disparity selection is optimized, and the stability of disparity estimation is improved.

[0083] In one alternative approach, S8 specifically includes: The three-dimensional coordinates of multiple trajectory points on the wheel trajectory path are calculated based on a single parallax value and the calibration parameters of the image acquisition device.

[0084] Here, calibration parameters refer to parameters obtained through the camera calibration process that describe the internal geometry and optical characteristics of the camera; for example, these include the focal length f. x =800 pixels, f y =800 pixels, principal point coordinates c x =640 pixels, c y =360 pixels, and the baseline distance of the binocular camera B=0.2m. The three-dimensional coordinates of the trajectory point refer to the position coordinates (X,Y,Z) of the point on the wheel trajectory path in the physical three-dimensional space; for example, the coordinates of a trajectory point are (1.5m,10.0m,0.0m).

[0085] The wheel trajectory information is generated and output based on the three-dimensional coordinates of multiple trajectory points.

[0086] Specifically, the three-dimensional coordinates of multiple trajectory points are connected in spatial order to construct a data structure describing the continuous motion path of the wheel, forming wheel trajectory information containing position sequences, and the wheel trajectory information is output through a data interface.

[0087] In the above-mentioned optional methods, the three-dimensional coordinates of the trajectory points are calculated by combining a single parallax value with the calibration parameters of the stereo camera and wheel trajectory information is generated. The pixel-level parallax is converted into physical space trajectory data, providing directly usable spatial position information for vehicle control.

[0088] In one alternative approach, it also includes: Based on the updated trajectory slope, updated trajectory intercept, and current steering wheel angle, calculate the image coordinates of multiple points within the dynamically dynamic region of interest.

[0089] The image coordinates of a location point refer to the position (x, y) of a point on the wheel track path in the image pixel coordinate system; for example, the coordinates of a location point are (160, 400).

[0090] Generate wheel trajectory paths based on image coordinates of multiple location points.

[0091] In the above optional methods, the image coordinates of multiple location points in the dynamic region of interest are further calculated based on the updated trajectory parameters and the steering wheel angle, and the wheel trajectory path is generated, providing an accurate trajectory reference for subsequent matching and disparity extraction.

[0092] It should be noted that in another embodiment of the wheel trajectory stereo matching method of this example, the following steps are included: 1) Perform the wheel trajectory mapping table generation step. The wheel trajectory mapping table contains five sets of key data: longitudinal coordinates... The left wheel's lateral left boundary The left wheel is lateral to the right boundary. The right wheel lateral to the left boundary and the right boundary of the right wheel. The wheel trajectory mapping table is in the form of { , , , , For the first frame, road straight lines parallel to the vehicle's direction of travel are detected in the current frame image, and the coordinates of their intersection points are calculated as vanishing points. Based on these vanishing point coordinates, and in conjunction with the vehicle's wheelbase and track width, the initial wheel trajectory parameters, including the initial trajectory slope, are estimated. Intercept of the initial trajectory For the processing of the second frame and subsequent frames, based on temporal continuity, the [method / mechanism] is utilized... Frame trajectory parameters , Combined with the current steering wheel angle and current vehicle speed Update, the calculation formula is as follows and ,in and The correction coefficients, obtained through training on a large amount of measured data, are determined through a pre-established mapping relationship (e.g., a two-dimensional lookup table). The wheel trajectory parameters are updated based on the current frame. and And combined with the current steering wheel angle Calculate wheel lateral offset Thus, the vertical coordinates are obtained. corresponding , , , The numerical values ​​complete the construction of the wheel trajectory mapping table.

[0093] 2) Perform the camera installation parameter calculation steps. Set the installation height of the stereo camera to... The installation angle is The vertical resolution of the current frame image is The vertical field of view of the stereo camera is This corresponds to the installation angle. Image vertical coordinates It can be calculated using the following formula: By analyzing multiple sets of angle data covering the expected road slope, the minimum longitudinal coordinate was statistically obtained. and vertical maximum coordinate This determines the vertical range of the dynamically interested region.

[0094] 3) Perform the step of determining the width and height of the wheel trajectory area. The height range of the dynamic region of interest is... The area width is obtained by calculating the difference between the lateral boundary of the farthest right wheel and the lateral boundary of the farthest left wheel. This difference reflects the wheel spacing, and half of this difference is stored as the half-width value. Based on a given height and width, initialize a dimension as (height, width, height)... The image array (3) is used to store image data within the dynamically dynamically selected region of interest.

[0095] 4) Perform the stereo matching step based on the network model. For example... Figure 2 As shown, firstly, a convolutional neural network is used to extract features from the left and right images respectively, obtaining feature maps for the left and right images. Based on the constructed wheel trajectory mapping table, the corresponding dynamic regions of interest (ROIs) in the left and right images are determined. Cost volume is then calculated within these ROIs: for each candidate disparity value... The feature map in the right image is shifted horizontally. Each pixel is then compared with the corresponding position in the dynamic region of interest of the feature map in the left image. The absolute difference of the features is calculated as the matching cost and accumulated to form the cost volume. After the cost volume is calculated, a cost aggregation operation is performed to smooth and optimize the matching cost. Based on the aggregation result, the candidate disparity values ​​are compared along the wheel trajectory path defined by the wheel trajectory mapping table. The corresponding aggregation cost is used to determine the candidate disparity value with the smallest aggregation cost as the single disparity value. Based on this, the disparity distribution of the wheel trajectory path is restored, thereby achieving accurate tracking of the wheel trajectory.

[0096] This embodiment achieves accurate extraction and continuous tracking of wheel trajectories even under conditions of missing or insufficient parallax information by fusing vehicle motion state parameters and stereo vision information. In the initial frame, the wheel trajectory parameters are initially estimated using image vanishing point calibration and inherent vehicle parameters. In subsequent frame processing, the trajectory parameters are iteratively updated and refined based on the trajectory parameters estimated in the previous frame, combined with real-time steering wheel angle and vehicle speed. As the number of processed frames increases, the wheel trajectory estimation results gradually converge and become more accurate, enabling effective tracking and adaptive adjustment of the trajectory even under non-ideal conditions such as bumpy roads.

[0097] This embodiment calculates a single disparity value through stereo matching, accurately reflecting the actual trajectory of the vehicle's tires during movement. It is suitable not only for straight-line driving but also reliably adapts to trajectory changes in complex driving scenarios such as turning and obstacle crossing. The entire processing flow integrates feature extraction, cost volume construction and aggregation, and disparity restoration, achieving efficient and stable real-time operation with limited computing resources. Even under complex road conditions such as inclement weather, low light, or missing road texture, the solution in this embodiment can still maintain stable tracking of the wheel trajectory.

[0098] This embodiment effectively solves the problem of trajectory extraction difficulties caused by insufficient parallax information and overcomes the limitation of traditional methods where a fixed region of interest cannot adapt to changes in vehicle motion. By dynamically adjusting the position and range of the region of interest and fusing vehicle dynamic parameters such as steering wheel angle and vehicle speed in real time, the real-time performance and accuracy of wheel trajectory monitoring are ensured. Ultimately, high-precision tracking and real-time online adjustment of wheel trajectories are achieved in various complex driving scenarios, improving the reliability and environmental robustness of the vehicle's driver assistance system.

[0099] Figure 3 A schematic diagram of an embodiment of a wheel trajectory three-dimensional matching system 200 provided by the present invention is shown. Figure 3 As shown, the wheel trajectory three-dimensional matching system 200 includes: The acquisition module 201 is used to acquire a first image and a second image that have a fixed spatial positional relationship and are synchronously acquired by the image acquisition device; The estimation module 202 is used to obtain the current frame image of the vehicle from the first image or the second image, and estimate the initial wheel trajectory parameters based on the vanishing point in the current frame image and the inherent parameters of the vehicle. The update module 203 is used to update the initial wheel trajectory parameters based on the current steering wheel angle and the current vehicle speed, so as to obtain the updated wheel trajectory parameters corresponding to the current frame. The determination module 204 is used to determine the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle; Extraction module 205 is used to extract features corresponding to the dynamic region of interest based on the first image and the second image, respectively, to obtain a first feature map and a second feature map; The calculation module 206 is used to translate any one of the first feature map and the second feature map along the horizontal direction of the image and match it with the other feature map, and calculate the matching cost. The aggregation module 207 is used to aggregate the matching costs and extract a single disparity value along the wheel trajectory path in the dynamic region of interest based on the aggregation results. Output module 208 is used to output wheel trajectory information based on a single parallax value.

[0100] In one alternative approach, the vehicle's inherent parameters include: vehicle wheelbase and vehicle track width; the estimation module 202 is specifically used for: Detect straight road lines parallel to the vehicle's direction of travel from the current frame image; Calculate the coordinates of the intersection point of the straight roads, and determine the intersection point coordinates as the vanishing point; The initial wheel trajectory parameters are calculated based on the coordinates of the vanishing point, the vehicle wheelbase, and the vehicle track width.

[0101] In one alternative approach, updating the wheel trajectory parameters includes updating the trajectory slope and updating the trajectory intercept; the update module 203 is specifically used for: Based on the pre-established mapping relationship, the slope correction and intercept correction are obtained by querying the current steering wheel angle and current vehicle speed. The initial trajectory slope and the slope correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory slope, and the initial trajectory intercept and the intercept correction amount in the initial wheel trajectory parameters are added together to obtain the updated trajectory intercept.

[0102] In one alternative approach, module 204 is specifically used for: The basic trajectory line of the wheel in the current frame image is determined based on the updated trajectory slope and the updated trajectory intercept. Calculate the lateral offset of the basic trajectory line based on the current steering wheel angle; Based on the lateral offset, determine the lateral boundary of the wheel corresponding to the preset vertical position in the current frame image; Based on the installation parameters of the image acquisition device and the vertical resolution of the current frame image, calculate the minimum and maximum longitudinal coordinates of the area where the wheel is located in the current frame image. The region of interest is determined based on the minimum and maximum longitudinal coordinates and the lateral boundary of the wheel.

[0103] In an alternative embodiment, the calculation module 206 is specifically used for: Within the dynamic region of interest, any feature map in the first and second feature maps is sequentially translated along the horizontal direction of the image according to multiple candidate disparity values. At each candidate disparity value, the translated feature map is compared pixel-by-pixel with the corresponding feature of another feature map within the dynamic region of interest. Based on the results of pixel-by-pixel comparisons, the matching cost corresponding to each candidate disparity value is calculated.

[0104] In one alternative approach, aggregation module 207 is specifically used for: The matching cost corresponding to each candidate disparity value is aggregated to obtain the aggregated cost corresponding to each candidate disparity value. Within the dynamic region of interest, the aggregate cost corresponding to each candidate disparity value is compared along the wheel trajectory path, and the candidate disparity value with the smallest aggregate cost is determined as the single disparity value.

[0105] In an alternative embodiment, output module 208 is specifically used for: Based on a single parallax value and the calibration parameters of the image acquisition device, calculate the three-dimensional coordinates of multiple trajectory points on the wheel trajectory path; The wheel trajectory information is generated and output based on the three-dimensional coordinates of multiple trajectory points.

[0106] In an alternative approach, it further includes: a generation module; the generation module is used for: Based on the updated trajectory slope, updated trajectory intercept, and current steering wheel angle, calculate the image coordinates of multiple points within the dynamic region of interest; Generate wheel trajectory paths based on image coordinates of multiple location points.

[0107] It should be noted that the beneficial effects of the wheel trajectory 3D matching system 200 provided in the above embodiments are the same as those of the wheel trajectory 3D matching method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0108] The wheel trajectory 3D matching system 200 of the present invention can be a computer program (including program code) running on a computer device. For example, the wheel trajectory 3D matching system 200 of the present invention is an application software that can be used to execute the corresponding steps in the wheel trajectory 3D matching method of the present invention.

[0109] In some embodiments, the wheel trajectory stereo matching system 200 of the present invention can be implemented in a combination of hardware and software. As an example, the wheel trajectory stereo matching system 200 of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the wheel trajectory stereo matching method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0110] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0111] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned wheel trajectory stereo matching methods. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the wheel trajectory stereo matching method shown in any embodiment of the present invention by calling the computer program.

[0112] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0113] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0114] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0115] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0116] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0117] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0118] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0119] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0121] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0122] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0123] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for three-dimensional matching of wheel trajectories, characterized in that, include: Acquire a first image and a second image that have a fixed spatial positional relationship, which are synchronously acquired by an image acquisition device; Obtain the current frame image of the vehicle from the first image or the second image, and estimate the initial wheel trajectory parameters based on the vanishing point in the current frame image and the inherent parameters of the vehicle. Based on the current steering wheel angle and the current vehicle speed, the initial wheel trajectory parameters are updated to obtain the updated wheel trajectory parameters corresponding to the current frame; Based on the updated wheel trajectory parameters and the current steering wheel angle, determine the dynamic region of interest in the current frame image; Based on the first image and the second image, the corresponding features in the dynamic region of interest are extracted to obtain a first feature map and a second feature map. Translate any one of the first and second feature maps along the horizontal direction of the image and match it with the other feature map, and calculate the matching cost. Cost aggregation is performed on the matching cost, and a single disparity value is extracted along the wheel trajectory path within the dynamic region of interest based on the aggregation result; The wheel trajectory information is output based on the single disparity value.

2. The wheel trajectory three-dimensional matching method according to claim 1, characterized in that, The vehicle's inherent parameters include: vehicle wheelbase and vehicle track width; the step of estimating initial wheel trajectory parameters based on the vanishing point in the current frame image and the vehicle's inherent parameters includes: Detect straight road lines parallel to the vehicle's direction of travel from the current frame image; Calculate the coordinates of the intersection point of the straight road lines, and determine the coordinates of the intersection point as the vanishing point; The initial wheel trajectory parameters are calculated based on the coordinates of the vanishing point, the vehicle wheelbase, and the vehicle track width.

3. The wheel trajectory three-dimensional matching method according to claim 1, characterized in that, The updated wheel trajectory parameters include: updating the trajectory slope and updating the trajectory intercept; the step of updating the initial wheel trajectory parameters based on the current steering wheel angle and current vehicle speed to obtain the updated wheel trajectory parameters corresponding to the current frame includes: Based on the pre-established mapping relationship, the slope correction amount and intercept correction amount are obtained by querying the current steering wheel angle and the current vehicle speed; The initial trajectory slope in the initial wheel trajectory parameters is added to the slope correction amount to obtain the updated trajectory slope, and the initial trajectory intercept in the initial wheel trajectory parameters is added to the intercept correction amount to obtain the updated trajectory intercept.

4. The wheel trajectory three-dimensional matching method according to claim 3, characterized in that, The step of determining the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle includes: The basic trajectory line of the wheel in the current frame image is determined based on the updated trajectory slope and the updated trajectory intercept; Calculate the lateral offset of the basic trajectory line based on the current steering wheel angle; Based on the lateral offset, determine the lateral boundary of the wheel corresponding to the preset longitudinal position in the current frame image; Based on the installation parameters of the image acquisition device and the vertical resolution of the current frame image, calculate the minimum and maximum longitudinal coordinates of the area where the wheel is located in the current frame image; The dynamic region of interest is determined based on the minimum longitudinal coordinate, the maximum longitudinal coordinate, and the lateral boundary of the wheel.

5. The wheel trajectory three-dimensional matching method according to claim 4, characterized in that, The steps of translating any one of the first and second feature maps along the horizontal direction of the image and matching it with the other feature map, and calculating the matching cost, include: Within the dynamic region of interest, any one of the first and second feature maps is sequentially translated along the horizontal direction of the image according to multiple candidate disparity values. At each candidate disparity value, the translated feature map is compared pixel-by-pixel with the feature of the other feature map at the corresponding position within the dynamic region of interest. Based on the results of the pixel-by-pixel comparison, the matching cost corresponding to each candidate disparity value is calculated.

6. The wheel trajectory three-dimensional matching method according to claim 5, characterized in that, The step of performing cost aggregation on the matching cost and extracting a single disparity value along the wheel trajectory path within the dynamic region of interest based on the aggregation result includes: The matching cost corresponding to each candidate disparity value is aggregated to obtain the aggregated cost corresponding to each candidate disparity value. Within the dynamic region of interest, the aggregate cost corresponding to each candidate disparity value is compared along the wheel trajectory path, and the candidate disparity value with the smallest aggregate cost is determined as the single disparity value.

7. The wheel trajectory three-dimensional matching method according to claim 6, characterized in that, The step of outputting wheel trajectory information based on the single disparity value includes: Based on the single disparity value and the calibration parameters of the image acquisition device, calculate the three-dimensional coordinates of multiple trajectory points on the wheel trajectory path; The wheel trajectory information is generated and output based on the three-dimensional coordinates of the multiple trajectory points.

8. The wheel trajectory three-dimensional matching method according to any one of claims 4 to 7, characterized in that, Also includes: Based on the updated trajectory slope, the updated trajectory intercept, and the current steering wheel angle, calculate the image coordinates of multiple points within the dynamic region of interest; The wheel trajectory path is generated based on the image coordinates of the multiple location points.

9. A three-dimensional wheel trajectory matching system, characterized in that, include: The acquisition module is used to acquire a first image and a second image that have a fixed spatial positional relationship, which are synchronously acquired by the image acquisition device; The estimation module is used to obtain the current frame image of the vehicle from the first image or the second image, and estimate the initial wheel trajectory parameters based on the vanishing point in the current frame image and the inherent parameters of the vehicle. The update module is used to update the initial wheel trajectory parameters based on the current steering wheel angle and the current vehicle speed to obtain the updated wheel trajectory parameters corresponding to the current frame. The determination module is used to determine the dynamic region of interest in the current frame image based on the updated wheel trajectory parameters and the current steering wheel angle; The extraction module is used to extract features corresponding to the dynamic region of interest based on the first image and the second image, respectively, to obtain a first feature map and a second feature map; The calculation module is used to translate any one of the first feature map and the second feature map along the horizontal direction of the image and match it with the other feature map, and calculate the matching cost. An aggregation module is used to aggregate the matching costs and extract a single disparity value along the wheel trajectory path within the dynamic region of interest based on the aggregation result. The output module is used to output wheel trajectory information based on the single disparity value.

10. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the wheel trajectory stereo matching method as described in any one of claims 1 to 8.

Images