Vehicle tracking control method and device, computer device and storage medium

Abstract

The application relates to a vehicle tracking control method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a current target image of a target vehicle photographed by a camera at a current position, an expected target image corresponding to the target vehicle at an expected position and an external parameter matrix; inputting the current target image and the expected target image into a pre-trained target detection model respectively to obtain a current ROI region and an expected ROI region; obtaining an interaction matrix according to the luminosity Gaussian mixed features of the current ROI region and the expected ROI region; acquiring a change rate of the luminosity Gaussian mixed features with respect to a Gaussian parameter, performing LM optimization on the camera speed according to the interaction matrix and the change rate to obtain an optimized camera speed; processing the optimized camera speed according to the external parameter matrix and an inverse kinematics model to obtain the speed of a vehicle where the camera is located, and realizing vehicle tracking control according to the speed of the vehicle where the camera is located. The method can improve the accuracy and speed of vehicle tracking control.

Description

Technical Field

[0001] This application relates to the field of vehicle tracking and control software management technology, and in particular to a vehicle tracking and control method, device, computer equipment, and storage medium. Background Technology

[0002] In recent years, multi-robot systems have been widely used in various aspects of life and production, including logistics, search and rescue, and autonomous driving. However, these robots often perform repetitive and dangerous activities. For multi-vehicle systems, strong vehicle platooning capabilities can not only liberate productivity and improve work efficiency, but also enhance road safety and traffic capacity. Therefore, it is desirable for unmanned vehicles to have strong vehicle tracking capabilities.

[0003] Current vehicle tracking control schemes primarily employ sensors such as cameras and lidar to detect and track targets. The position and attitude of the tracked target are then estimated in three-dimensional space, and a controller is designed based on this estimation to achieve tracking control. Mapping information from the sensor space to three-dimensional space introduces errors from sensor intrinsic and extrinsic parameter calibration, sensor measurement, and detection / tracking algorithms. The accumulation of these multiple errors leads to poor vehicle tracking performance. Summary of the Invention

[0004] Therefore, it is necessary to provide a vehicle tracking and control method, device, computer equipment, and storage medium to address the aforementioned technical problems.

[0005] A vehicle tracking control method, the method comprising:

[0006] Acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the extrinsic parameter matrix;

[0007] The current target image and the desired target image are respectively input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region;

[0008] Calculate the luminosity Gaussian mixture features of the current ROI region and the desired ROI region respectively to obtain the current image features and the desired image features, and obtain the interaction matrix based on the current image features and the desired image features;

[0009] The rate of change of the photometric Gaussian mixture feature with respect to the Gaussian parameter is obtained, and the camera speed is optimized by LM based on the interaction matrix and the rate of change to obtain the optimized camera speed.

[0010] The optimized camera speed is processed based on the extrinsic parameter matrix and inverse kinematics model to obtain the speed of the vehicle where the camera is located, and vehicle tracking control is achieved based on the speed of the vehicle where the camera is located.

[0011] In one embodiment, the method further includes: updating the current position based on the speed of the vehicle where the camera is located to obtain a new current target image; iteratively updating the speed of the vehicle where the camera is located using the new current target image until the vehicle where the camera is located reaches the desired position, and then stopping the iteration to achieve vehicle tracking control.

[0012] In one embodiment, the method further includes: obtaining an interaction matrix based on the current image features and the desired image features.

[0013]

[0014] Among them, L G For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, α u Let α be the transformation coefficient between the image coordinate system and the pixel coordinate system in the horizontal direction. v The transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction, (x g y g (v) represents the x and y coordinates of a specific photometric Gaussian mixture feature in the image coordinate system. g v g ) represents the x and y coordinates of a specific luminosity Gaussian mixture feature in the image coordinate system, Z represents the depth estimate of the monocular camera, I represents the luminosity of the image (also known as brightness), and λ represents the image luminosity. g The Gaussian parameter is used to adjust convergence.

[0015] In one embodiment, the method further includes: performing LM optimization on the camera speed based on the interaction matrix and the rate of change to obtain the optimized camera speed as follows:

[0016]

[0017]

[0018] Among them, V c The optimized camera speed is λ, the overall error adjustment parameter is μ, and the hybrid parameter for adjusting the LM optimization performance is G. I The photometric Gaussian mixture features corresponding to the current target image I. For the desired target image I * The corresponding Gaussian mixture characteristics of luminosity This is the interaction matrix when the current target image is I.

[0019] In one embodiment, the method further includes: inputting the current target image and the desired target image into a pre-trained YOLO model to obtain the position of the target vehicle on the current target image and the desired target image; and using the KCF method to track the target vehicle on the current target image and the desired target image according to the target vehicle position to obtain the current ROI region and the desired ROI region.

[0020] A vehicle tracking control device, the device comprising:

[0021] The data acquisition module is used to acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the external parameter matrix;

[0022] The target detection module is used to input the current target image and the desired target image into a pre-trained target detection model to obtain the current ROI region and the desired ROI region;

[0023] The interaction matrix calculation module is used to calculate the photometric Gaussian mixture features of the current ROI region and the desired ROI region respectively, to obtain the current image features and the desired image features, and to obtain the interaction matrix based on the current image features and the desired image features;

[0024] The camera speed optimization module is used to obtain the rate of change of the photometric Gaussian mixture feature with the Gaussian parameter, and to perform LM optimization on the camera speed according to the interaction matrix and the rate of change to obtain the optimized camera speed.

[0025] The vehicle tracking control module is used to process the optimized camera speed according to the extrinsic parameter matrix and the inverse kinematics model to obtain the speed of the vehicle where the camera is located, and to achieve target vehicle tracking control based on the speed of the vehicle where the camera is located.

[0026] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0027] Acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the extrinsic parameter matrix;

[0028] The current target image and the desired target image are respectively input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region;

[0029] Calculate the luminosity Gaussian mixture features of the current ROI region and the desired ROI region respectively to obtain the current image features and the desired image features, and obtain the interaction matrix based on the current image features and the desired image features;

[0030] The rate of change of the photometric Gaussian mixture feature with respect to the Gaussian parameter is obtained, and the camera speed is optimized by LM based on the interaction matrix and the rate of change to obtain the optimized camera speed.

[0031] The optimized camera speed is processed based on the extrinsic parameter matrix and inverse kinematics model to obtain the speed of the vehicle where the camera is located, and vehicle tracking control is achieved based on the speed of the vehicle where the camera is located.

[0032] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0033] Acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the extrinsic parameter matrix;

[0034] The current target image and the desired target image are respectively input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region;

[0035] Calculate the luminosity Gaussian mixture features of the current ROI region and the desired ROI region respectively to obtain the current image features and the desired image features, and obtain the interaction matrix based on the current image features and the desired image features;

[0036] The rate of change of the photometric Gaussian mixture feature with respect to the Gaussian parameter is obtained, and the camera speed is optimized by LM based on the interaction matrix and the rate of change to obtain the optimized camera speed.

[0037] The optimized camera speed is processed based on the extrinsic parameter matrix and inverse kinematics model to obtain the speed of the vehicle where the camera is located, and vehicle tracking control is achieved based on the speed of the vehicle where the camera is located.

[0038] The aforementioned vehicle tracking control method, apparatus, computer equipment, and storage medium acquire the current target image of the target vehicle captured by the camera at the current position and the desired target image of the target vehicle at the desired position, thereby utilizing image information to complete tracking control. This effectively expands the application scenarios and enables high-precision vehicle tracking control without global localization. Next, the current target image and the desired target image are input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region. The photometric Gaussian mixture features of the current ROI region and the desired ROI region are calculated respectively to obtain the current image features and the desired image features. By adding a pre-tracking framework to the direct visual servoing, the computational efficiency of vehicle tracking can be effectively improved. An interaction matrix is ​​obtained based on the current image features and the desired image features. The rate of change of the photometric Gaussian mixture features with respect to Gaussian parameters is obtained. LM optimization of the camera speed is performed based on the interaction matrix and the rate of change, which is beneficial to the convergence of the direct visual servoing. This embodiment of the invention has a wider range of applicable scenarios and can improve the accuracy and speed of vehicle tracking control. Attached Figure Description

[0039] Figure 1 This is an application scenario diagram of the vehicle tracking control method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating a vehicle tracking control method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating the calculation of photometric Gaussian mixture features in one embodiment;

[0042] Figure 4 This is a flowchart illustrating a vehicle tracking control method in a specific embodiment;

[0043] Figure 5 This is a structural block diagram of a vehicle tracking control device in one embodiment;

[0044] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] The vehicle tracking control method provided in this application can be applied to, for example... Figure 1The application environment shown is illustrated. The vehicle tracking control unit is equipped with a camera that captures images of the tracked vehicle. Based on the images, the current pose of the vehicle tracking control unit or its relative pose to the tracked vehicle is indirectly calculated. This pose information is then used to drive the vehicle tracking control unit to complete servo tasks. The vehicle tracking control unit and the tracked vehicle can be, but are not limited to, various robots, unmanned vehicles, and manned vehicles.

[0047] In one embodiment, such as Figure 2 As shown, a vehicle tracking control method is provided, including the following steps:

[0048] Step 202: Obtain the current target image of the target vehicle captured by the camera at the current position, as well as the desired target image and extrinsic parameter matrix corresponding to the target vehicle at the desired position.

[0049] This invention utilizes direct visual servoing to continuously approximate the current image with the desired image, achieving vehicle tracking control. Direct visual servoing refers to visual servoing without local feature extraction; it directly uses global image features such as luminance, color, gradient, and depth to construct an error function to design the controller. It's important to note that this invention does not require the coordinate information of the desired position; it only needs the desired image as input and extracts pose changes by utilizing changes in image features, thereby achieving vehicle tracking control. The extrinsic parameter matrix refers to the positional relationship between the camera and the vehicle where the camera is located, facilitating subsequent conversion of camera velocity to vehicle velocity.

[0050] Step 204: Input the current target image and the desired target image into the pre-trained target detection model to obtain the current ROI region and the desired ROI region.

[0051] Object detection models include those based on YOLO (You Only Look Once) and KCF (Kernel Correlation Filter). YOLO is a one-stage algorithm in object detection, using only a single CNN network to directly predict the class and location of different objects, expressed as a Region of Interest (ROI). In object tracking applications, KCF includes a filter template that maximizes the response when applied to the target; the location of the maximum response value is the target's location.

[0052] Step 206: Calculate the luminosity Gaussian mixture features of the current ROI region and the desired ROI region respectively to obtain the current image features and the desired image features, and obtain the interaction matrix based on the current image features and the desired image features.

[0053] Photometric Gaussian mixture features can be understood as another feature representation of an image, such as... Figure 3 The flowchart shown illustrates the calculation of luminance Gaussian mixture features. It replaces the luminance of each pixel with a Gaussian distribution of luminance, and ultimately, the combination of these Gaussian distributions forms the Gaussian mixture, which represents the image. As shown in the formula below, the luminance Gaussian mixture feature G is the final image feature s composed of the combination of the Gaussian distributions E of each pixel I:

[0054] s(r)=G(I,u g ,λ g )

[0055]

[0056] Among them, g(I,u g ,u,λ g ) is a Gaussian mixture density function, where I is one of the independent variables representing the brightness I(u) of all pixels. g It is a specific pixel used to calculate Gaussian luminance blending features, u g =(u g v g Let λ represent the x and y coordinates of a specific Gaussian mixture feature in the image coordinate system, u represent all pixels, and u = (u, v) represent the x and y coordinates in the image coordinate system. g This is a parameter that can affect convergence. The luminance Gaussian mixture feature can be calculated by substituting the brightness of all pixels in the image into the above formula.

[0057] For visual servoing, solving for the interaction matrix is ​​a crucial issue. Because changes in image features implicitly indicate changes in pose, after appropriately selecting image features *s*, it is necessary to derive the relationship between these feature changes and camera velocity in order to choose a suitable solution method to ultimately obtain the camera velocity. This relationship between feature changes and camera velocity is called the interaction matrix. The interaction matrix using normalized image coordinates (x, y) as image features *s* is shown below:

[0058]

[0059]

[0060] in, L represents the change in characteristics. s Let V represent the interaction matrix, V represent the camera velocity, and Z represent the depth estimate of the monocular camera.

[0061] The derivation of the interaction matrix requires adherence to its definition, namely, the relationship between feature changes and camera speed. Therefore, it is necessary to calculate the feature changes:

[0062] First, the derivative of the Gaussian mixture feature of luminosity is taken, and the result is:

[0063]

[0064] in, It is the derivative of the luminosity Gaussian mixture density function with respect to space, and the result is:

[0065]

[0066] Will Substitute the calculation formula The calculation formula yields the final derivative relationship as follows:

[0067]

[0068] Combining the definition of the interaction matrix, we can obtain some expressions in formula (3) to form the interaction matrix L. G :

[0069]

[0070] in, The content can be calculated using the following formula. You can refer to the interaction matrix L s ,

[0071]

[0072] Substituting the derivation results into the calculation formula yields the final interaction matrix:

[0073]

[0074] Among them, L G For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, α u Let α be the transformation coefficient between the image coordinate system and the pixel coordinate system in the horizontal direction. v The transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction, (x g y g ) represents the x and y coordinates of a specific photometric Gaussian mixture feature in the image coordinate system, (u g v g ) represents the x and y coordinates of a specific luminosity Gaussian mixture feature in the image coordinate system, Z represents the depth estimate of the monocular camera, I represents the luminosity of the image (also known as brightness), and λ represents the image luminosity. g The Gaussian parameter is used to adjust convergence.

[0075] Step 208: Obtain the rate of change of the photometric Gaussian mixture features with respect to the Gaussian parameters, and perform LM optimization on the camera speed based on the interaction matrix and the rate of change to obtain the optimized camera speed.

[0076] The LM optimization algorithm is a combination of the steepest descent method and the Gauss-Newton method. When a large optimization parameter is selected, this optimization method is close to the steepest descent method and is often used for situations far from the minimum value. When a small optimization parameter is selected, this optimization method is close to the Gauss-Newton method and can make the optimization result closer to the minimum point.

[0077] To better adjust the Gaussian parameter λ in the photometric Gaussian mixture g To improve the convergence of direct vision servoing, it is also necessary to calculate the rate of change of the photometric Gaussian mixture features with respect to parameters. for:

[0078]

[0079] Step 210: Process the optimized camera speed according to the extrinsic parameter matrix and inverse kinematics model to obtain the speed of the vehicle where the camera is located, and realize vehicle tracking control based on the speed of the vehicle where the camera is located.

[0080] The inverse kinematics model of a vehicle refers to determining the necessary control commands for the vehicle based on the known required speed. The inverse kinematics model of a vehicle is as follows:

[0081] V fl =(v x +v y +(LX+LY)×w z )

[0082] V fr =(v x -v y -(LX+LY)×w z )

[0083] V bl =(v x -v y +(LX+LY)×w z )

[0084] V br =(v x +v y -(LX+LY)×w z )

[0085] Among them, (V) fl V fr V bl V brThe values ​​) represent the linear velocities of the left front wheel, right front wheel, left rear wheel, and right rear wheel of the vehicle, respectively. LX and LY represent the track width between the left and right wheels and the wheelbase between the front and rear wheels, respectively. x Let v be the translation component of the camera velocity in the x-direction. y Let w be the translation component of the camera velocity in the y-direction. z Let be the rotational component of the camera velocity in the z-direction.

[0086] In the aforementioned vehicle tracking control method, the current target image captured by the camera at the current position and the desired target image of the target vehicle at the desired position are acquired. This image information is then used to complete tracking control, effectively expanding the application scenarios and enabling high-precision vehicle tracking control even without global localization. Next, the current target image and the desired target image are input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region. The photometric Gaussian mixture features of the current ROI region and the desired ROI region are calculated respectively to obtain the current image features and the desired image features. By adding a pre-tracking framework to the direct visual servoing, the computational efficiency of vehicle tracking is effectively improved. An interaction matrix is ​​obtained based on the current image features and the desired image features. The rate of change of the photometric Gaussian mixture features with respect to Gaussian parameters is obtained. LM optimization of the camera speed is performed based on the interaction matrix and the rate of change, which is beneficial for the convergence of the direct visual servoing. This embodiment of the invention has a wider range of applicable scenarios and can improve the accuracy and speed of vehicle tracking control.

[0087] In one embodiment, vehicle tracking control based on the speed of the vehicle where the camera is located includes: updating the current position based on the speed of the vehicle where the camera is located to obtain a new current target image; iteratively updating the speed of the vehicle where the camera is located using the new current target image until the vehicle where the camera is located reaches the desired position, and then stopping the iteration to achieve vehicle tracking control.

[0088] In one embodiment, obtaining the interaction matrix based on the current image features and the desired image features includes: obtaining the interaction matrix based on the current image features and the desired image features as follows:

[0089]

[0090] Among them, L G For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, α u Let α be the transformation coefficient between the image coordinate system and the pixel coordinate system in the horizontal direction. v The transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction, (xg y g ) represents the x and y coordinates of a specific photometric Gaussian mixture feature in the image coordinate system, (u g v g ) represents the x and y coordinates of a specific luminosity Gaussian mixture feature in the image coordinate system, Z represents the depth estimate of the monocular camera, I represents the luminosity of the image (also known as brightness), and λ represents the image luminosity. g The Gaussian parameter is used to adjust convergence.

[0091] In one embodiment, performing LM optimization on the camera speed based on the interaction matrix and the rate of change to obtain the optimized camera speed includes: performing LM optimization on the camera speed based on the interaction matrix and the rate of change to obtain the optimized camera speed as follows:

[0092]

[0093]

[0094] Among them, V c G represents the optimized camera speed, λ is the overall error adjustment parameter, μ is the hybrid parameter for adjusting LM optimization performance, and G... I The photometric Gaussian mixture features corresponding to the current target image I. For the desired target image I * The corresponding Gaussian mixture characteristics of luminosity This is the interaction matrix when the current target image is I.

[0095] In one embodiment, inputting the current target image and the desired target image into a pre-trained target detection model to obtain the current ROI region and the desired ROI region includes: inputting the current target image and the desired target image into a pre-trained YOLO model to obtain the position of the target vehicle on the current target image and the desired target image; and using the KCF method to track the target vehicle on the current target image and the desired target image according to the target vehicle position to obtain the current ROI region and the desired ROI region.

[0096] In one specific embodiment, such as Figure 4 The diagram shows a flowchart of a vehicle tracking control method, which includes:

[0097] S10: Input the image I_desire at the desired location, obtain the image I_initial at the current location, and fixed constant parameters.

[0098] The fixed constant parameters include the extrinsic parameters of the vehicle and camera, as well as the vehicle's shape parameters, which facilitate the construction of subsequent equations. I_desire and I_initial represent the desired target image and the current target image, respectively.

[0099] S20: Use YOLO to complete a specific target detection task, obtain the ROI of the target in l_desire and I_initial, and use KCF to track the target and obtain the ROI in the subsequent process.

[0100] After acquiring the raw image information, the first step is to use YOLO+KCF to detect the object to be tracked, and then use KCF to track the specific object to obtain the image region of the specific target for subsequent processing.

[0101] S21: Using the photometric Gaussian mixture formula, the photometric Gaussian mixture features of the ROL region images of l_desire and l_initial are calculated, namely G_desire and G_initial.

[0102] G_desire and G_initial represent the photometric Gaussian mixture features after processing I_desire and I_initial, respectively, i.e., the desired image features and the current image features. After the ROI extraction is completed, the photometric Gaussian mixture is calculated for the image of that region and used as the feature for direct visual serving.

[0103] S22: Calculate the interaction matrix L using the direct vision servoing correlation calculation formula and substituting G_desire and G_initial. s .

[0104] S30: The camera speed is calculated using the LM optimization method.

[0105] After calculating the interaction matrix and the rate of change of the Gaussian parameters, this relationship can be used to solve for the camera velocity. LM optimization transforms the servo control problem into minimizing the loss function c(r):

[0106] c(r)=(s(r)-s(r * )) T (s(r)-s(r * ))

[0107] Where s(r) represents the current image features, s(r * Let ) represent the desired image features. Considering the background of direct vision servoing, when the initial error is large, choosing a large parameter μ makes LM optimization approximate the steepest descent method, which can meet the control speed requirements. Meanwhile, when the error decreases to a certain extent, choosing a small parameter μ makes LM optimization approximate the Gauss-Newton method, which can meet the high-precision control requirements. Finally, the most suitable camera speed V is found through LM optimization. c =(vx v y w z ).

[0108] S31: The vehicle speed is calculated using the extrinsic parameter matrix and the inverse kinematics model to achieve vehicle tracking control.

[0109] After calculating the camera speed, the speed value needs to be processed and converted according to the positional relationship between the camera and the vehicle. At the same time, it is also necessary to combine the inverse kinematics model of the vehicle to finally calculate the vehicle control command that should be executed in order to achieve the required vehicle speed.

[0110] S40: Iterative loop. Achieve high-precision tracking control of the vehicle in a relatively short time.

[0111] Since the image of the vehicle changes each time the above process is executed, the process can be repeated after one loop is completed, thus achieving high-precision tracking control of the vehicle within a certain period.

[0112] In this embodiment, the present invention proposes a novel vehicle tracking control method, namely, direct visual servoing based on photometric Gaussian mixture. Based on this method, certain improvements and optimizations are made by incorporating the vehicle tracking background. This method solves the problem of high-precision vehicle tracking control without global localization, completing tracking control solely using image information, effectively expanding the application scenarios and resolving the contradiction between accuracy and speed in vehicle tracking. By adding a pre-tracking framework to the direct visual servoing, and using the LM optimization method for calculation, both accuracy and speed are improved, alleviating to some extent the convergence difficulty of direct visual servoing. Using the method of the present invention, the control accuracy of image-based vehicle tracking can be effectively improved. Furthermore, by incorporating the pre-tracking background, the computational efficiency of vehicle tracking can be effectively improved. Moreover, the method of the present invention does not require global localization information, has fewer requirements for the application scenarios, and completes vehicle tracking control solely using image information, thus having a wider range of applications.

[0113] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not required to be sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0114] In one embodiment, such as Figure 5 As shown, a vehicle tracking control device is provided, including: a data acquisition module 502, a target detection module 504, an interaction matrix calculation module 506, a camera speed optimization module 508, and a vehicle tracking control module 510, wherein:

[0115] The data acquisition module 502 is used to acquire the current target image of the target vehicle captured by the camera at the current position, as well as the expected target image and extrinsic parameter matrix corresponding to the target vehicle at the expected position;

[0116] The target detection module 504 is used to input the current target image and the desired target image into the pre-trained target detection model to obtain the current ROI region and the desired ROI region;

[0117] The interaction matrix calculation module 506 is used to calculate the luminous Gaussian mixture features of the current ROI region and the desired ROI region respectively, to obtain the current image features and the desired image features, and to obtain the interaction matrix based on the current image features and the desired image features.

[0118] The camera speed optimization module 508 is used to obtain the rate of change of the photometric Gaussian mixture feature with the Gaussian parameter, and to perform LM optimization on the camera speed based on the interaction matrix and the rate of change to obtain the optimized camera speed.

[0119] The vehicle tracking control module 510 is used to process the optimized camera speed according to the extrinsic parameter matrix and the inverse kinematics model to obtain the speed of the vehicle where the camera is located, and to realize the tracking control of the target vehicle based on the speed of the vehicle where the camera is located.

[0120] In one embodiment, the camera is further configured to update its current position based on the speed of the vehicle in which it is located, thereby obtaining a new current target image; and to iteratively update the speed of the vehicle in which it is located using the new current target image until the vehicle in which it is located reaches the desired position, at which point the iteration stops, thereby achieving vehicle tracking control.

[0121] In one embodiment, the interaction matrix calculation module 506 is further configured to obtain the interaction matrix based on the current image features and the desired image features:

[0122]

[0123] Among them, LG For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, α u Let α be the transformation coefficient between the image coordinate system and the pixel coordinate system in the horizontal direction. v The transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction, (x g y g ) represents the x and y coordinates of a specific photometric Gaussian mixture feature in the image coordinate system, (u g v g ) represents the x and y coordinates of a specific luminosity Gaussian mixture feature in the image coordinate system, Z represents the depth estimate of the monocular camera, I represents the luminosity of the image (also known as brightness), and λ represents the image luminosity. g The Gaussian parameter is used to adjust convergence.

[0124] In one embodiment, the camera speed optimization module 508 is further configured to perform LM optimization on the camera speed based on the interaction matrix and the rate of change, to obtain the optimized camera speed as follows:

[0125]

[0126]

[0127] Among them, V c G represents the optimized camera speed, λ is the overall error adjustment parameter, μ is the hybrid parameter for adjusting LM optimization performance, and G... I The photometric Gaussian mixture features corresponding to the current target image I. For the desired target image I * The corresponding Gaussian mixture characteristics of luminosity This is the interaction matrix when the current target image is I.

[0128] In one embodiment, the target detection module 504 is further configured to input the current target image and the desired target image into a pre-trained YOLO model to obtain the position of the target vehicle on the current target image and the desired target image; based on the position of the target vehicle, the KCF method is used to track the target vehicle on the current target image and the desired target image to obtain the current ROI region and the desired ROI region.

[0129] Specific limitations regarding the vehicle tracking control device can be found in the limitations of the vehicle tracking control method described above, and will not be repeated here. Each module in the aforementioned vehicle tracking control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0130] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a vehicle tracking control method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0131] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this invention should be determined by the appended claims.

Claims

1. A vehicle tracking control method, characterized in that, The method includes: Acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the extrinsic parameter matrix; The current target image and the desired target image are respectively input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region; Calculate the luminosity Gaussian mixture features of the current ROI region and the desired ROI region respectively to obtain the current image features and the desired image features, and obtain the interaction matrix based on the current image features and the desired image features; The rate of change of the photometric Gaussian mixture feature with respect to the Gaussian parameter is obtained, and the camera speed is optimized by LM based on the interaction matrix and the rate of change to obtain the optimized camera speed. The optimized camera velocity is processed using the extrinsic parameter matrix and inverse kinematics model to obtain the velocity of the vehicle where the camera is located. Vehicle tracking control is then implemented based on this velocity. The interaction matrix is ​​obtained based on the current image features and the desired image features. in, For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, These are the transformation coefficients between the image coordinate system and the pixel coordinate system in the horizontal direction. Transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction. For specific photometric Gaussian mixture features, the x and y coordinates are represented in the image coordinate system. For specific photometric Gaussian mixture features, the x and y coordinates are represented in the image coordinate system. This is the depth estimate from a monocular camera. The luminance of an image is also called its brightness. The Gaussian parameters are adjustable for convergence. for The corresponding Gaussian distribution of luminosity.

2. The method according to claim 1, characterized in that, The vehicle tracking control based on the speed of the vehicle where the camera is located includes: The current position is updated based on the speed of the vehicle where the camera is located, resulting in a new current target image; The speed of the vehicle where the camera is located is iteratively updated using the new current target image until the vehicle reaches the desired position, at which point the iteration stops, thus achieving vehicle tracking control.

3. The method according to claim 2, characterized in that, Based on the interaction matrix and the rate of change, the camera speed is optimized using the LM method to obtain the optimized camera speed, which includes: Based on the interaction matrix and the rate of change, the camera speed is optimized using the LM method to obtain the optimized camera speed as follows: ， in, For optimized camera speed, This is the overall error adjustment parameter. To adjust the hybrid parameters for optimizing LM performance, For the current target image The corresponding Gaussian mixture characteristics of luminosity For the desired target image The corresponding Gaussian mixture characteristics of luminosity The current target image is Interaction matrix at time.

4. The method according to claim 1, characterized in that, The current target image and the desired target image are respectively input into a pre-trained target detection model to obtain the current ROI region and the desired ROI region, including: The current target image and the desired target image are respectively input into a pre-trained YOLO model to obtain the position of the target vehicle on the current target image and the desired target image; Based on the target vehicle's location, the KCF method is used to track the target vehicle in the current target image and the desired target image respectively, thereby obtaining the current ROI region and the desired ROI region.

5. A vehicle tracking control device, characterized in that, The device includes: The data acquisition module is used to acquire the current target image of the target vehicle captured by the camera at the current position, the desired target image of the target vehicle at the desired position, and the external parameter matrix. The target detection module is used to input the current target image and the desired target image into a pre-trained target detection model to obtain the current ROI region and the desired ROI region; The interaction matrix calculation module is used to calculate the photometric Gaussian mixture features of the current ROI region and the desired ROI region respectively, to obtain the current image features and the desired image features, and to obtain the interaction matrix based on the current image features and the desired image features; The camera speed optimization module is used to obtain the rate of change of the photometric Gaussian mixture feature with the Gaussian parameter, and to perform LM optimization on the camera speed according to the interaction matrix and the rate of change to obtain the optimized camera speed. The vehicle tracking control module is used to process the optimized camera speed according to the extrinsic parameter matrix and the inverse kinematics model to obtain the speed of the vehicle where the camera is located, and to realize the tracking control of the target vehicle according to the speed of the vehicle where the camera is located. The interaction matrix calculation module is also used to obtain the interaction matrix based on the current image features and the desired image features: in, For the overall interaction matrix, This represents the translation component of the interaction matrix in the vehicle's forward and backward directions. Let be the translation component of the interaction matrix in the left-right direction of the vehicle. The interaction matrix in the vehicle's rotational component, These are the transformation coefficients between the image coordinate system and the pixel coordinate system in the horizontal direction. Transformation coefficients between the image coordinate system and the pixel coordinate system in the vertical direction. For specific photometric Gaussian mixture features, the x and y coordinates are represented in the image coordinate system. For specific photometric Gaussian mixture features, the x and y coordinates are represented in the image coordinate system. This is the depth estimate from a monocular camera. The luminance of an image is also called its brightness. The Gaussian parameter is used to adjust convergence.

6. The apparatus according to claim 5, characterized in that, The camera speed optimization module is also used to perform LM optimization on the camera speed based on the interaction matrix and the rate of change, to obtain the optimized camera speed as follows: ， in, For optimized camera speed, This is the overall error adjustment parameter. To adjust the hybrid parameters for optimizing LM performance, For the current target image The corresponding Gaussian mixture characteristics of luminosity For the desired target image The corresponding Gaussian mixture characteristics of luminosity The current target image is Interaction matrix at time, for The corresponding Gaussian distribution of luminosity.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images