Vehicle control method and device, vehicle, medium and product

Abstract

The invention discloses a vehicle control method and device, an electronic device, a vehicle, a computer readable storage medium and a computer program product, and the method comprises the steps: obtaining the position and posture information of the vehicle, and adjusting the position and posture of a camera according to the position and posture information, so as to enable the shooting angle of the vehicle camera to maintain a steady state. Therefore, the pose of the vehicle camera can be adjusted according to the pose information of the vehicle, so that the pose of the vehicle camera can be changed based on the pose information of the vehicle, and the shooting angle of the vehicle camera is kept stable; therefore, the influence of the pose change of the vehicle body on the shooting angle of the vehicle camera can be reduced to a certain extent, and when the pose change of the vehicle body is caused by the factors such as the concave-convex road surface in the vehicle driving process, the vehicle can accurately sense the external environment through the camera with the shooting angle maintaining the steady state, so that the safety of the vehicle is improved. And the stable running of the vehicle is ensured.

Description

Technical Field

[0001] The present application relates to the field of camera technology, and in particular to a vehicle control method, a control device, an electronic device, a vehicle, a computer-readable storage medium, and a computer program product. Background Art

[0002] In related technologies, vehicles can sense the external environment through various sensors, such as using images captured by cameras to confirm obstacles on the road ahead. However, during driving, the vehicle may shake due to uneven road surfaces, causing the camera installed on the vehicle to shake as well, resulting in residual images in the images captured by the camera, which is of poor quality and affects the vehicle's accurate perception of the external environment. Summary of the invention

[0003] The present application provides a vehicle control method, a control device, an electronic device, a vehicle, a computer-readable storage medium, and a computer program product.

[0004] The present application provides a vehicle control method, characterized in that the camera is installed on the vehicle, and the position of the camera is adjustable. The method includes:

[0005] Acquiring position information of the vehicle;

[0006] According to the posture information, the posture of the vehicle camera is adjusted so that the shooting angle of the vehicle camera remains stable.

[0007] In this way, in the implementation mode of the present application, the posture of the vehicle camera can be adjusted according to the posture information of the vehicle, so that the posture of the vehicle camera can be changed based on the posture information of the vehicle, thereby maintaining the shooting angle of the vehicle camera in a steady state, thereby reducing the influence of the posture change of the vehicle body on the shooting angle of the vehicle camera to a certain extent. Furthermore, when the posture of the vehicle body changes due to factors such as uneven road surface during vehicle driving, the vehicle can accurately perceive the external environment through the camera whose shooting angle maintains a steady state, and the smooth driving of the vehicle can be guaranteed.

[0008] In certain embodiments of the present application, the posture information includes an angular velocity of the vehicle in a preset direction, and adjusting the posture of the vehicle camera according to the posture information so that the shooting angle of the vehicle camera maintains a steady state includes:

[0009] Determining an angular change of the vehicle in the preset direction according to the angular velocity;

[0010] According to the angle change, the angle of the vehicle camera in the preset direction is adjusted.

[0011] Thus, in an embodiment of the present application, the angular change of the vehicle in the preset direction can be determined according to the angular velocity of the vehicle in the preset direction, and the angle of the vehicle camera in the preset direction can be adjusted according to the angular change of the vehicle in the preset direction, thereby achieving the adjustment of the vehicle camera posture.

[0012] In certain embodiments of the present application, the method further comprises:

[0013] After adjusting the angle of the vehicle camera, the angle of the vehicle camera in the preset direction is adjusted according to the angle change amount and the adjusted angle of the vehicle camera.

[0014] Thus, in the embodiment of the present application, after adjusting the angle of the vehicle camera, the angle of the vehicle camera in a preset direction can be adjusted according to the angle change and the adjusted angle of the vehicle camera, thereby realizing closed-loop control of the vehicle camera posture.

[0015] In certain embodiments of the present application, the vehicle includes a drive motor connected to the vehicle camera, the drive motor is used to drive the vehicle camera to move to change the posture of the vehicle camera, and the posture of the vehicle camera is adjusted according to the posture information so that the shooting angle of the vehicle camera maintains a steady state, including:

[0016] According to the posture information, the driving motor is controlled to operate so as to adjust the posture of the vehicle camera.

[0017] Thus, in the embodiment of the present application, the drive motor can be controlled to operate according to the posture information of the vehicle to adjust the posture of the vehicle camera, so that the posture adjustment of the vehicle camera can be achieved based on the drive motor.

[0018] In certain embodiments of the present application, the method further comprises:

[0019] The multiple frames of images captured by the camera are compensated to suppress the displacement of pixels in the multiple frames of images.

[0020] Thus, in the embodiment of the present application, on the basis of adjusting the posture of the vehicle camera based on the posture of the vehicle so that the shooting angle of the vehicle camera maintains a steady state, the multiple frames of images captured by the vehicle camera can be compensated to suppress the displacement of pixels in the multiple frames of images, thereby ensuring the quality of the images captured by the vehicle camera.

[0021] In certain embodiments of the present application, compensating the multiple frames of images captured by the camera to suppress the displacement of pixels in the multiple frames of images includes:

[0022] Preprocessing the multiple frames of images to determine the processed multiple frames of images;

[0023] The processed multiple frames of images are compensated to suppress the displacement of pixels in the multiple frames of images.

[0024] Thus, in the implementation manner of the present application, multiple frames of images may be preprocessed to improve the image quality of the multiple frames of images.

[0025] In certain embodiments of the present application, compensating the multiple frames of images captured by the camera to suppress the displacement of pixels in the multiple frames of images includes:

[0026] Determining displacement information of pixels in the multiple frames of images according to the multiple frames of images;

[0027] The multiple frames of images are compensated according to the displacement information to suppress the displacement of pixels in the multiple frames of images.

[0028] Thus, in the embodiment of the present application, the displacement information of the pixels in the multiple frames of images can be determined based on the multiple frames of images, and the multiple frames of images can be compensated based on the displacement information to suppress the displacement of the pixels in the multiple frames of images, thereby achieving pixel displacement compensation.

[0029] In some embodiments of the present application, the displacement information includes an optical flow vector, and determining the displacement information of pixels in the multiple frames of images according to the multiple frames of images includes:

[0030] According to the pixel points in the multiple frames of images, optical flow vectors corresponding to the multiple frames of images are determined.

[0031] Thus, in the embodiment of the present application, the optical flow vectors corresponding to the multiple frames of images can be determined according to the pixel points in the multiple frames of images, so that the displacement information of the pixels in the multiple frames of images can be determined based on the optical flow method.

[0032] In some embodiments of the present application, the displacement information includes a displacement amount, the multiple frames of images include a previous frame of image and a next frame of image taken continuously, and determining the displacement information of pixels in the multiple frames of image according to the multiple frames of image includes:

[0033] Determine, according to the multiple frames of images, a first position of a feature point in the previous frame of image and a second position of the feature point in the next frame of image;

[0034] The displacement amount is determined according to the first position and the second position.

[0035] Thus, in an embodiment of the present application, the first position of a feature point in a previous frame image and the second position of a feature point in a next frame image are determined based on multiple frames of images, and the displacement of pixels in the previous and next frames of images are determined based on the first and second positions of the feature points, thereby achieving the determination of pixel displacement.

[0036] In some embodiments of the present application, the multiple frames of images include a previous frame of image and a next frame of image taken continuously, and the compensating the multiple frames of images according to the displacement information to suppress the displacement of pixels in the multiple frames of images includes:

[0037] According to the displacement information, the pixel value and the pixel position of the image pixel point in the next frame of image are adjusted.

[0038] Thus, in the implementation mode of the present application, the pixel value and pixel position of the image pixel point in the next frame image can be adjusted according to the displacement information, thereby achieving displacement compensation of the pixel point in the next frame image.

[0039] In certain embodiments of the present application, adjusting the pixel value and the pixel position of the image pixel point in the next frame of image according to the displacement information includes:

[0040] According to the displacement information, adjusting the position of the image pixel in the next frame of image;

[0041] The pixel value of the image pixel point is adjusted according to the pixel value of the image pixel point and the pixel value of the reference pixel point, wherein the image pixel point and the reference pixel point are adjacent in the subsequent frame image.

[0042] Thus, in the implementation mode of the present application, the position of the image pixel in the subsequent frame image can be adjusted according to the displacement information, and the pixel value of the image pixel can be adjusted according to the pixel value of the image pixel and the pixel value of the reference pixel, thereby achieving the adjustment of the pixel value and the pixel position of the image pixel in the subsequent frame image.

[0043] An embodiment of the present application provides a control device, the device comprising a transceiver unit and a processing unit;

[0044] The transceiver unit is configured to obtain the position information of the vehicle;

[0045] The processing unit is configured to adjust the posture of the vehicle camera according to the posture information so that the shooting angle of the vehicle camera maintains a steady state.

[0046] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned vehicle control method is implemented.

[0047] An embodiment of the present application provides a vehicle, comprising the above-mentioned electronic device or control device.

[0048] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by one or more processors, the above-mentioned vehicle control method is implemented.

[0049] An embodiment of the present application provides a computer program product, including a computer program / instruction, which implements the above-mentioned vehicle control method when executed by a processor.

[0050] The control device, electronic device, vehicle, computer-readable storage medium and computer program product provided in the embodiments of the present application can adjust the posture of the vehicle camera according to the posture information of the vehicle, so that the posture of the vehicle camera can be changed based on the posture information of the vehicle, thereby maintaining the shooting angle of the vehicle camera in a steady state, thereby reducing the influence of the posture change of the vehicle body on the shooting angle of the vehicle camera to a certain extent. Furthermore, when the posture of the vehicle body changes due to factors such as uneven road surface during vehicle driving, the vehicle can accurately perceive the external environment through the camera whose shooting angle maintains a steady state, and the smooth driving of the vehicle is guaranteed.

[0051] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0053] Figure 1 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0054] Figure 2 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0055] Figure 3 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0056] Figure 4 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0057] Figure 5 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0058] Figure 6 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0059] Figure 7A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0060] Figure 8 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0061] Fig. 9 A schematic flow chart of a vehicle control method in certain embodiments of the present application;

[0062] Fig.10 A schematic diagram of an application scenario in certain embodiments of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0064] It is understandable that in the autonomous driving system, the camera is one of the main perception devices, and the stability of its image directly affects the recognition accuracy and safety of the system. Therefore, the Electronic Image Stabilization (EIS) technology is proposed in the relevant technology, which is a widely used image stabilization technology. Specifically, this technology uses digital signal processing methods to smooth and compensate for the jittery image to achieve image stability.

[0065] Furthermore, the electronic image stabilization technology mainly includes three steps, namely: 1) collecting continuous frame images through the camera. 2) using the optical flow method or feature point matching method to detect the jitter in the image. 3) compensating the detected jitter by moving the image frame to make the output image more stable.

[0066] However, since electronic image stabilization technology mainly relies on image processing algorithms for image smoothing and compensation, the processing effect is limited by the algorithm to a certain extent. Especially in the case of severe jitter, the compensation effect is not good, and afterimages or image blur may occur. At the same time, since complex calculations need to be performed on each frame of the image, such as jitter detection, image compensation, etc., high hardware performance requirements are imposed, which increases the system's computational burden and power consumption. In addition, since each processing step requires a certain execution time, there is a processing delay, which is a serious defect for real-time application scenarios such as autonomous driving, and it is difficult to meet the system's real-time and response speed requirements. In addition, in scenarios where lighting conditions vary greatly or image features are not obvious, the electronic image stabilization algorithm may not be able to effectively detect and compensate for jitter, resulting in a decrease in image stabilization.

[0067] Therefore, although electronic image stabilization technology can improve image stability to a certain extent, it still has many shortcomings in practical applications and cannot meet the high requirements of autonomous driving environments.

[0068] Based on the above problems you may encounter, please refer to Figure 1 The embodiment of the present application provides a method for adjusting a camera, wherein the camera is installed on a vehicle and the position of the camera is adjustable. The method comprises:

[0069] 01: Get the vehicle’s position information;

[0070] 02: According to the posture information, adjust the posture of the vehicle camera to keep the shooting angle of the vehicle camera stable.

[0071] The embodiment of the present application provides a camera adjustment device. The camera adjustment method of the embodiment of the present application can be implemented by the adjustment device of the embodiment of the present application. Specifically, the adjustment device includes a transceiver unit and a processing unit. The transceiver unit is configured to obtain the posture information of the vehicle. The processing unit is configured to adjust the posture of the vehicle camera according to the posture information so that the shooting angle of the vehicle camera maintains a steady state.

[0072] The embodiment of the present application also provides an electronic device, which includes a memory and a processor. The camera adjustment method of the embodiment of the present application can be implemented by the electronic device of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain the posture information of the vehicle, and to adjust the posture of the vehicle camera according to the posture information so that the shooting angle of the vehicle camera maintains a steady state.

[0073] Specifically, in the implementation of the present application, the vehicle (or the electronic device in the vehicle, or the control device in the vehicle) can obtain its own posture information, such as the pitch angle of the vehicle body, etc. Then, the vehicle can adjust the posture of the vehicle camera according to the acquired posture information, so that the posture of the vehicle camera can change with the change of the vehicle posture, thereby ensuring that the camera angle of the vehicle camera can maintain a steady state when the vehicle posture changes, thereby enabling the vehicle camera to stably capture images.

[0074] For example, when the vehicle body is tilted up, the front-view camera can be controlled to tilt downward, so that the shooting angle of the front-view camera is always facing the front of the vehicle, so that the vehicle can perceive the road conditions in front of the vehicle through the front-view camera when the vehicle body is tilted up.

[0075] In this way, in the implementation mode of the present application, the posture of the vehicle camera can be adjusted according to the posture information of the vehicle, so that the posture of the vehicle camera can be changed based on the posture information of the vehicle, thereby maintaining the shooting angle of the vehicle camera in a steady state, thereby reducing the influence of the posture change of the vehicle body on the shooting angle of the vehicle camera to a certain extent. Furthermore, when the posture of the vehicle body changes due to factors such as uneven road surface during vehicle driving, the vehicle can accurately perceive the external environment through the camera whose shooting angle maintains a steady state, and the smooth driving of the vehicle can be guaranteed.

[0076] See also Figure 2 In some embodiments of the present application, the posture information includes the angular velocity of the vehicle in a preset direction, and then, step 02 includes:

[0077] 020: Determine the angular change of the vehicle in the preset direction according to the angular velocity;

[0078] 021: Adjust the angle of the vehicle camera in the preset direction according to the angle change.

[0079] The processing unit of the embodiment of the present application is configured to determine the angle change of the vehicle in a preset direction according to the angular velocity, and adjust the angle of the vehicle camera in the preset direction according to the angle change.

[0080] The processor of the embodiment of the present application is also used to determine the angle change of the vehicle in a preset direction based on the angular velocity, and to adjust the angle of the vehicle camera in the preset direction based on the angle change.

[0081] Specifically, in the embodiment of the present application, the posture information obtained by the vehicle can be used to describe the angular velocity of the vehicle body in one or more directions, and therefore, the vehicle can integrate the angular velocity to determine the angular change of the vehicle body in one or more directions. Finally, the vehicle can adjust the angle of the vehicle camera in the preset direction according to the angular change.

[0082] In one example, the posture information includes the roll angular velocity, pitch angular velocity, and yaw angular velocity of the vehicle.

[0083] In one example, the angle variation includes a roll angle variation, a pitch angle variation, and a yaw angle variation. The roll angle variation is obtained by integrating the roll angular velocity of the vehicle, the pitch angle variation is obtained by integrating the pitch angular velocity variation of the vehicle, and the yaw angle variation is obtained by integrating the yaw angular velocity of the vehicle.

[0084] In one example, when the roll angle of the vehicle changes by an amount A 1° The vehicle can control the roll angle of the vehicle camera to reduce A 1 °, when the vehicle's roll angle is reduced to A 1 ° The vehicle can control the roll angle of the vehicle camera to increase by A 1 °. Similarly, when the pitch angle of the vehicle changes by increasing A 2 ° The vehicle can control the pitch angle of the vehicle camera to decrease by A 2 °, when the pitch angle of the vehicle is reduced to A 2 ° The vehicle can control the pitch angle of the vehicle camera to increase by A 2 °. And, when the yaw angle of the vehicle changes by A 3 ° The vehicle can control the yaw angle of the vehicle camera to reduce A 3 °, when the vehicle's yaw angle is reduced to A 3 ° The vehicle can control the yaw angle of the vehicle camera to increase by A 3 °.

[0085] Thus, in the implementation mode of the present application, the angular change of the vehicle in the preset direction can be determined according to the angular velocity of the vehicle in the preset direction, and the angle of the vehicle camera in the preset direction can be adjusted according to the angular change of the vehicle in the preset direction, thereby realizing the adjustment of the vehicle camera posture.

[0086] In certain embodiments of the present application, the vehicle control method further includes:

[0087] After adjusting the angle of the vehicle camera, the angle of the vehicle camera in a preset direction is adjusted according to the angle change amount and the adjusted angle of the vehicle camera.

[0088] The processing unit of the embodiment of the present application is configured to adjust the angle of the vehicle camera in a preset direction according to the angle change and the adjusted angle of the vehicle camera after adjusting the angle of the vehicle camera.

[0089] The processor of the embodiment of the present application is also used to adjust the angle of the vehicle camera in a preset direction according to the angle change and the adjusted angle of the vehicle camera after adjusting the angle of the vehicle camera.

[0090] Specifically, in the implementation mode of the present application, the vehicle can realize closed-loop posture adjustment of the vehicle camera based on PID (Proportional Integral Derivative) control. Specifically, in the implementation mode of the present application, after the vehicle adjusts the angle of the vehicle camera in the preset direction according to the angle change amount in the preset direction, the vehicle can make a secondary adjustment to the angle of the vehicle camera in the preset direction according to the angle change amount and the adjusted angle of the vehicle camera, thereby ensuring that the angle of the vehicle camera in the preset direction can be accurately compensated.

[0091] For example, when the roll angle of the vehicle changes by A 1 °, the vehicle can generate a control instruction according to the change in roll angle and send the control instruction to the vehicle camera. Then, the vehicle camera reduces A based on the control instruction. 4 °, the vehicle can 1 ° and A 4 ° difference (|A 1 °-A 4 °|) generates a new control instruction and sends the new control instruction to the vehicle camera, so that the vehicle camera can reduce or increase its own roll angle according to the new control instruction, thereby ensuring that the change of its own roll angle is A 1 °.

[0092] Thus, in the embodiment of the present application, after adjusting the angle of the vehicle camera, the angle of the vehicle camera in a preset direction can be adjusted according to the angle change and the adjusted angle of the vehicle camera, thereby realizing closed-loop control of the vehicle camera posture.

[0093] In certain embodiments of the present application, the vehicle includes a driving motor connected to the vehicle camera, and the driving motor is used to drive the vehicle camera to move to change the posture of the vehicle camera body. Then, step 02 includes:

[0094] According to the position information, the drive motor is controlled to adjust the position of the vehicle camera.

[0095] The processing unit of the embodiment of the present application is configured to control the drive motor to operate so as to adjust the posture of the vehicle camera according to the posture information.

[0096] The processor of the embodiment of the present application is also used to control the operation of the drive motor to adjust the posture of the vehicle camera according to the posture information.

[0097] Specifically, in the embodiment of the present application, a driving motor is provided on the vehicle, and the motor is connected to the vehicle camera. The driving motor can drive the vehicle camera body to change the posture of the vehicle camera. Therefore, in the embodiment of the present application, the vehicle can control the driving motor to work so that the driving motor drives the vehicle camera to move, thereby adjusting the posture of the vehicle camera.

[0098] It is understandable that the response speed of the motor is crucial, and the response speed of the motor determines the real-time performance and stability of the entire system. Therefore, in the implementation of the present application, the motor needs to have high torque and high precision to quickly respond to the adjustment command of the controller and accurately adjust the vehicle camera posture.

[0099] In one example, the motor is a servo motor.

[0100] Thus, in the embodiment of the present application, the drive motor can be controlled to operate according to the posture information of the vehicle to adjust the posture of the vehicle camera, so that the posture adjustment of the vehicle camera can be achieved based on the drive motor.

[0101] See also Figure 3 In certain embodiments of the present application, the vehicle control method further includes:

[0102] 03: Compensate the multiple frames of images captured by the vehicle camera to suppress the displacement of pixels in the multiple frames.

[0103] The processing unit of the embodiment of the present application is configured to compensate for the multiple frames of images captured by the vehicle camera to suppress the displacement of pixels in the multiple frames of images.

[0104] The processor of the embodiment of the present application is also used to compensate for the multiple frames of images captured by the vehicle camera to suppress the displacement of pixels in the multiple frames of images.

[0105] Specifically, in order to further ensure the quality of images captured by the vehicle camera, in the implementation mode of the present application, image motion compensation can be performed on the multiple frames of images captured by the vehicle camera to eliminate residual jitter and blur in the image, so as to ensure the high quality and clarity of the image.

[0106] It can be understood that compared with the electronic image stabilization technology in the related art, the embodiment of the present application can adjust the posture of the vehicle camera based on the posture of the vehicle so that the shooting angle of the vehicle camera maintains a steady state, so as to eliminate the residual jitter, blur, etc. in the image taken by the vehicle camera to a certain extent, and on this basis, perform pixel displacement compensation on the image taken by the vehicle camera, so as to further eliminate the residual jitter, blur, etc. in the image, and ensure the image quality.

[0107] Thus, in the embodiment of the present application, on the basis of adjusting the posture of the vehicle camera based on the posture of the vehicle so that the shooting angle of the vehicle camera maintains a steady state, the multiple frames of images captured by the vehicle camera can be compensated to suppress the displacement of pixels in the multiple frames of images, thereby ensuring the quality of the images captured by the vehicle camera.

[0108] See also Figure 4 In certain embodiments of the present application, step 03 includes:

[0109] 030: pre-processing the multiple frame images and determining the processed multiple frame images;

[0110] 031: Compensate the processed multi-frame images to suppress the displacement of pixels in the multi-frame images.

[0111] The processing unit of the embodiment of the present application is configured to pre-process the multiple frames of images, determine the processed multiple frames of images, and compensate the processed multiple frames of images to suppress the displacement of pixels in the multiple frames of images.

[0112] The processor of the embodiment of the present application is also used to pre-process the multiple frames of images, determine the processed multiple frames of images, and compensate the processed multiple frames of images to suppress the displacement of pixels in the multiple frames of images.

[0113] Specifically, in the implementation manner of the present application, multiple frames of images may be preprocessed to improve the image quality of the multiple frames of images, so as to provide a basis for subsequent image motion compensation.

[0114] In one example, the preprocessing includes at least one of noise reduction processing, enhancement processing, and correction processing.

[0115] The noise reduction process refers to removing random noise in the image by using Gaussian filtering, median filtering, etc. It can be understood that Gaussian filtering can smooth the image and reduce the influence of noise on image motion detection.

[0116] Enhancement processing refers to enhancing the contrast and edge details of an image by means of histogram equalization or Laplacian operator. It can be understood that histogram equalization can improve the global contrast of an image and make the details clearer.

[0117] Correction processing refers to color correction, that is, if there is color deviation in the image, it can be corrected by adjusting the white balance to ensure color accuracy.

[0118] It can be understood that after preprocessing, the noise of the image is reduced and the image details are enhanced, thereby providing a reliable basis for subsequent image motion detection and image motion compensation.

[0119] Thus, in the implementation manner of the present application, multiple frames of images may be preprocessed to improve the image quality of the multiple frames of images.

[0120] See also Figure 5 In certain embodiments of the present application, step 03 includes:

[0121] 032: Determine displacement information of pixels in the multiple frame images according to the multiple frame images;

[0122] 033: Compensate multiple frame images according to displacement information to suppress the displacement of pixels in the multiple frame images.

[0123] The processing unit of the embodiment of the present application is configured to determine the displacement information of the pixels in the multiple frames of images according to the multiple frames of images, and compensate the multiple frames of images according to the displacement information to suppress the displacement of the pixels in the multiple frames of images.

[0124] The processor of the embodiment of the present application is also used to determine the displacement information of the pixels in the multiple frames of images based on the multiple frames of images, and to compensate the multiple frames of images based on the displacement information to suppress the displacement of the pixels in the multiple frames of images.

[0125] Specifically, in the embodiment of the present application, the vehicle can determine the displacement information of the pixels in the multiple frames of images, such as the displacement amount, etc., based on the multiple frames of images captured by the vehicle camera. Then, the vehicle compensates the multiple frames of images based on the displacement information of the pixels in the multiple frames of images to suppress the displacement of the pixels in the multiple frames of images.

[0126] Thus, in the embodiment of the present application, the displacement information of the pixels in the multiple frames of images can be determined based on the multiple frames of images, and the multiple frames of images can be compensated based on the displacement information to suppress the displacement of the pixels in the multiple frames of images, thereby achieving pixel displacement compensation.

[0127] In some embodiments of the present application, the displacement information includes an optical flow vector, and then, step 030 includes:

[0128] According to the pixel points in the multiple-frame images, the optical flow vectors corresponding to the multiple-frame images are determined.

[0129] The processing unit of the embodiment of the present application is configured to determine the optical flow vectors corresponding to the multiple frames of images based on the pixel points in the multiple frames of images.

[0130] The processor of the embodiment of the present application is also used to determine the optical flow vectors corresponding to the multiple frames of images based on the pixel points in the multiple frames of images.

[0131] Specifically, in the implementation manner of the present application, the displacement of pixels in adjacent frame images can be determined by the optical flow method, that is, the optical flow vectors corresponding to the multiple frame images are determined by the pixel points in the multiple frame images.

[0132] In one example, the vehicle can call a pre-set program that can implement the Lucas-Kanade optical flow method or the Horn-Schunck optical flow method to determine the optical flow vectors corresponding to multiple frames of images. Among them, the Lucas-Kanade optical flow method is suitable for local small-scale pixel motion detection, while the Horn-Schunck optical flow method can detect global pixel motion.

[0133] Thus, in the embodiment of the present application, the optical flow vectors corresponding to the multiple frames of images can be determined according to the pixel points in the multiple frames of images, so that the displacement information of the pixels in the multiple frames of images can be determined based on the optical flow method.

[0134] See also Figure 6 In some embodiments of the present application, the displacement information includes the displacement amount, and the multiple frames of images include a previous frame of image and a next frame of image taken continuously. Then, step 032 includes:

[0135] 0320: Determine, according to the multiple frames of images, a first position of a feature point in a previous frame of image and a second position of a feature point in a next frame of image;

[0136] 0321: Determine the displacement according to the first position and the second position.

[0137] The processing unit of the embodiment of the present application is configured to determine the first position of the feature point in the previous frame image and the second position of the feature point in the next frame image based on multiple frames of images, and determine the displacement based on the first position and the second position.

[0138] The processor of the embodiment of the present application is also used to determine the first position of the feature point in the previous frame image and the second position of the feature point in the next frame image based on multiple frames of images, and to determine the displacement based on the first position and the second position.

[0139] Specifically, in the implementation mode of the present application, the vehicle can determine the displacement of the pixel points in the two frames of images by the feature point matching method. Specifically, in the implementation mode of the present application, the vehicle can determine the displacement of the same feature point in the two frames of images according to the first position of the feature point in the previous frame of image and the second position of the feature point in the next frame of image, thereby obtaining the displacement information of the pixel points in the two frames of image.

[0140] In an example, the feature points are at least one of SIFT (Scale-invariant feature transform) feature points, SURF (Speeded-Up Robust Features) feature points, and ORB (Oriented FAST and Rotated BRIEF) feature points.

[0141] In one example, the vehicle first extracts a feature point in the previous frame image to determine a first position P of the feature point in the previous frame image. 1 Then, a pixel matching (or similar) to the feature point is determined in the next frame image, and the position of the pixel is used as the second position P of the feature point in the next frame image. 2 Finally, the vehicle can be based on the first position P of the feature point 1 and the second position P 2 , determine the displacement of the feature point, thereby determining the displacement of the same pixel point in two adjacent frames of images, and then obtain the displacement information of the pixels in the two adjacent frames of images.

[0142] Thus, in an embodiment of the present application, the first position of a feature point in a previous frame image and the second position of a feature point in a next frame image are determined based on multiple frames of images, and the displacement of pixels in the previous and next frames of images are determined based on the first and second positions of the feature points, thereby achieving the determination of pixel displacement.

[0143] See also Figure 7 In some embodiments of the present application, the multiple frames of images include a previous frame of image and a next frame of image taken continuously, and then, step 033 includes:

[0144] 0330: According to the displacement information, the pixel value and pixel position of the image pixel point in the next frame image are adjusted.

[0145] The processing unit of the embodiment of the present application is configured to adjust the pixel value and pixel position of the image pixel point in the next frame image according to the displacement information.

[0146] The processor of the embodiment of the present application is also used to adjust the pixel value and pixel position of the image pixel point in the next frame of image according to the displacement information.

[0147] Specifically, in the implementation mode of the present application, the pixel value and pixel position of the pixel point in the next frame image can be adjusted respectively according to the displacement information of the pixels in the previous and next frames of images, thereby compensating for the displacement of the pixel point in the next frame image.

[0148] In one example, the displacement information includes the displacement amount. Then, the vehicle can control the pixel points in the next frame of image to move in the opposite direction according to the displacement amount of the pixel points in the two adjacent frames of image. For example, if the displacement amount of the pixel points in the two adjacent frames of image is "moving 3 pixel units to the left", each pixel point in the next frame of image can be moved 3 pixel units to the right. At the same time, the vehicle can determine the pixel value of each pixel point according to its neighboring pixel points, thereby completing the position adjustment and pixel value adjustment of each pixel point in the next frame of image.

[0149] Thus, in the implementation mode of the present application, the pixel value and pixel position of the image pixel point in the next frame image can be adjusted according to the displacement information, thereby achieving displacement compensation of the pixel point in the next frame image.

[0150] See also Figure 8 In some embodiments of the present application, step 0330 includes:

[0151] 03300: Adjust the position of the image pixel in the next frame of image according to the displacement information;

[0152] 03301: adjusting the pixel value of the image pixel according to the pixel value of the image pixel and the pixel value of the reference pixel, wherein the image pixel and the reference pixel are adjacent in a subsequent frame of the image.

[0153] The processing unit of the embodiment of the present application is configured to adjust the position of the image pixel in the subsequent frame image according to the displacement information, and adjust the pixel value of the image pixel according to the pixel value of the image pixel and the pixel value of the reference pixel, wherein the image pixel and the reference pixel are adjacent in the subsequent frame image.

[0154] The processor of the embodiment of the present application is also used to adjust the position of the image pixel in the subsequent frame image according to the displacement information, and adjust the pixel value of the image pixel according to the pixel value of the image pixel and the pixel value of the reference pixel, wherein the image pixel and the reference pixel are adjacent in the subsequent frame image.

[0155] Specifically, in the implementation mode of the present application, the position of each pixel in the next frame of image can be corrected according to the displacement information of the pixel in the previous and next frames of image, so as to determine the new pixel position of each pixel in the next frame of image. At the same time, in the implementation mode of the present application, the new pixel value of each pixel in the next frame of image can be calculated by an interpolation algorithm. Finally, a new image is combined according to the new pixel position and new pixel value of each pixel in the next frame of image, thereby completing the displacement compensation of the next frame of image.

[0156] In one example, the displacement information is the displacement of pixels in the two frames of images. Furthermore, when the displacement of pixels in the two frames of images is 3 pixel units to the left, for each image pixel in the latter image, the current position of the image pixel can be moved 3 pixel units to the right to determine the new position of the image pixel, thereby completing the position correction of the image pixel.

[0157] In one example, the vehicle calculates the new pixel value of each pixel in the next frame of the image by bilinear interpolation or bicubic interpolation, wherein bilinear interpolation calculates the new pixel value by the weighted average of the surrounding four pixels, while bicubic interpolation calculates the new pixel by the weighted average of the surrounding 16 pixels.

[0158] For example, for any image pixel in the next frame of image, based on the four reference pixels adjacent to the image pixel, namely, the pixel directly above, the pixel directly to the left, the pixel directly below, and the pixel directly to the right, the pixel values ​​of these four reference pixels are weighted, summed, and averaged to obtain a new pixel value for the image pixel, so that the pixel value of the image pixel is adjusted to the new pixel value.

[0159] Thus, in the implementation mode of the present application, the position of the image pixel in the subsequent frame image can be adjusted according to the displacement information, and the pixel value of the image pixel can be adjusted according to the pixel value of the image pixel and the pixel value of the reference pixel, thereby achieving the adjustment of the pixel value and the pixel position of the image pixel in the subsequent frame image.

[0160] To more clearly illustrate the implementation of this application, please refer to Fig. 9 , Fig. 9 1 is a flow chart of a vehicle control method in certain embodiments of the present application. Specifically, Fig. 9 As shown, in an embodiment of the present application, a three-axis gyro stabilization platform is installed in the vehicle, and the three-axis gyro stabilization platform includes a three-axis gyroscope, a stabilization controller and an actuator, and a camera is installed on the three-axis gyro stabilization platform.

[0161] Among them, the three-axis gyroscope is used to detect the vehicle's roll angular velocity, pitch angular velocity and yaw angular velocity in real time. These three angular velocities reflect the vehicle's rotational motion in three directions and are a direct representation of the vehicle's posture change. It can be understood that the three-axis gyroscope has three independent gyroscopes built in, each of which is responsible for detecting the angular velocity in one axis. It can also be understood that by integrating the angular velocity, the angular change of the vehicle in each direction can be obtained.

[0162] It should be noted that in the embodiment of the present application, the three-axis gyroscope transmits the detected angular velocity data to the stability controller in real time. It is understandable that the data transmission frequency needs to be high enough to ensure that the system can respond to changes in the vehicle posture in a timely manner.

[0163] Next, the stabilization controller calculates the compensation angle required for the camera based on the angular velocity data provided by the three-axis gyroscope, thereby offsetting the image jitter caused by the change in vehicle posture. Specifically, the stabilization controller first filters the angular velocity data transmitted by the gyroscope to remove noise and unnecessary high-frequency interference. Then, the vehicle's posture change is estimated using signal processing algorithms such as the Kalman filter. Next, based on the estimation result (i.e., the angle change), the controller calculates the compensation angle of the camera. Finally, a control instruction is generated based on the compensation angle and sent to the actuator (i.e., the motor connected to the camera).

[0164] Next, the actuator drives the camera to make real-time posture adjustments based on the received control instructions to ensure the stability of the shooting angle.

[0165] While driving the camera to adjust its posture in real time, the images captured by the camera are pre-processed such as denoising, enhancement, and color correction, thereby reducing noise in the image and enhancing image details.

[0166] Subsequently, the optical flow method or feature point matching method is used to obtain the movement amount and direction of pixels in adjacent frame images, thereby completing image motion detection.

[0167] Afterwards, the image is corrected using an interpolation algorithm based on the detected amount and direction of movement to eliminate residual jitter and blur and restore image clarity.

[0168] To more clearly illustrate the implementation of this application, please refer to Fig.10 , Fig.10 This is a schematic diagram of an application scenario in some embodiments of the present application. That is, in the embodiments of the present application, corresponding experimental verification can be carried out to determine the implementation effect.

[0169] Static experiment: In a stationary state, the camera on the vehicle continuously collects image data, records the original image and the adjusted image after processing by the three-axis gyro stabilized platform. Then, the original image is compared with the adjusted image, and image quality evaluation indicators such as Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) are used to evaluate the stability and clarity of the adjusted image. Then, the improvement effect of the three-axis gyro stabilized platform on image stability in a stationary state is analyzed, and the data of various evaluation indicators are recorded. Finally, through static experiments, the basic performance of the three-axis gyro stabilized platform without external interference can be preliminarily verified.

[0170] Dynamic experiment: Continuously collect image data at different vehicle speeds (such as 30km / h, 60km / h, 100km / h) and different road conditions (such as flat roads and bumpy roads), record the original images and the processed images after being processed by the three-axis gyro-stabilized platform. Then, use image quality evaluation indicators to compare the original images and stabilized images under different conditions to evaluate the performance of the platform in a dynamic environment. Then, analyze the improvement effect of the three-axis gyro-stabilized platform on image stability under different vehicle speeds and road conditions, and record the data of various evaluation indicators. Finally, through dynamic experiments, the stability performance and adaptability of the three-axis gyro-stabilized platform under actual driving conditions can be fully verified.

[0171] Comparative experiment: Several common traditional image processing technologies (such as electronic image stabilization and digital filtering) are selected as comparison objects to ensure that the test environment and conditions are consistent. Specifically, image data is collected in static and dynamic experiments, and the original image, the image processed by traditional image processing technology, and the image processed by the three-axis gyro stabilized platform are recorded. Then, the image quality evaluation index and the recognition accuracy of the autonomous driving recognition system are used to compare and analyze the image quality and recognition effect after processing by different technologies. Then, by comparing the experimental results, the differences between the three-axis gyro stabilized platform and the traditional image processing technology in image stability, clarity and recognition accuracy are evaluated, and the data of various evaluation indicators are recorded. Finally, through the comparative experiment, the advantages of the three-axis gyro stabilized platform in image stability and recognition accuracy can be intuitively demonstrated, and the effectiveness and superiority of the three-axis gyro stabilized platform can be verified.

[0172] Image clarity analysis: Several groups of original images and images processed by the three-axis gyro-stabilized platform were selected from static and dynamic experiments. Subsequently, the structural similarity SSIM and PSNR of the original and processed images were calculated. Finally, the SSIM and PSNR values ​​of each group of images were summarized, the average and standard deviation were calculated, and the improvement effect of image clarity under different conditions was compared and analyzed. It can be understood that the SSIM value ranges from 0 to 1, and the higher the value, the better the image quality, and the higher the PSNR value, the smaller the image distortion and the higher the quality.

[0173] Recognition accuracy analysis uses the vehicle's autonomous driving system to perform object recognition tests in different scenarios such as daytime, nighttime, rainy days, urban roads, and highways. Specifically, the system's recognition results in each scenario are recorded, including the number of correct recognitions and the total number of tests. Subsequently, the recognition accuracy in each scenario is calculated. Finally, the changes in recognition accuracy before and after using the three-axis gyro-stabilized platform in different scenarios are compared and analyzed to evaluate the improvement of the three-axis gyro-stabilized platform on recognition performance.

[0174] Processing delay analysis: First, record the time required for the three-axis gyro-stabilized platform and the forward image motion compensation algorithm to process each frame of the image, including the total time for data acquisition, processing, and output. Then, calculate the average processing time for multiple measurements to evaluate the real-time performance of the system. Then, analyze the relationship between processing delay and factors such as image resolution and vehicle speed, and evaluate the computational complexity of the algorithm under different conditions. After that, compare the relationship between processing delay and recognition accuracy and image clarity, and comprehensively evaluate the real-time performance and performance of the system. Then, organize all experimental data and comprehensively evaluate the overall effect of the three-axis gyro-stabilized platform. Finally, based on the three indicators of image clarity, recognition accuracy, and processing delay, the practical application value and improvement direction of the three-axis gyro-stabilized platform in autonomous driving recognition are obtained.

[0175] It can be understood that through the experimental steps, the comprehensive effect of the three-axis gyro-stabilized platform can be comprehensively and objectively evaluated to ensure its practical application value in the autonomous driving system.

[0176] The embodiment of the present application also provides a vehicle, the vehicle comprising the above-mentioned electronic device or the above-mentioned control device

[0177] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned vehicle control method is implemented.

[0178] The embodiment of the present application also provides a computer program product, including a computer program / instruction, which implements the above-mentioned vehicle control method when executed by a processor.

[0179] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0180] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0181] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vehicle control method, characterized in that: include: Acquiring position information of the vehicle; According to the posture information, the posture of the vehicle camera is adjusted so that the shooting angle of the vehicle camera remains stable.

2. The method according to claim 1, wherein the position information includes an angular velocity of the vehicle in a preset direction, and adjusting the position of the vehicle camera according to the position information so that the shooting angle of the vehicle camera maintains a steady state, comprising: Determining an angular change of the vehicle in the preset direction according to the angular velocity; According to the angle change, the angle of the vehicle camera in the preset direction is adjusted.

3. The method according to claim 2, further comprising: After adjusting the angle of the vehicle camera, the angle of the vehicle camera in the preset direction is adjusted according to the angle change amount and the adjusted angle of the vehicle camera.

4. The method according to claim 1, characterized in that: The vehicle includes a driving motor connected to the vehicle camera, the driving motor is used to drive the vehicle camera to move so as to change the posture of the vehicle camera, and the posture of the vehicle camera is adjusted according to the posture information so that the shooting angle of the vehicle camera maintains a steady state, including: According to the posture information, the driving motor is controlled to operate so as to adjust the posture of the vehicle camera.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The multiple frames of images captured by the vehicle camera are compensated to suppress the displacement of pixels in the multiple frames of images.

6. The method according to claim 5, characterized in that The compensating the multiple frames of images captured by the vehicle camera to suppress the displacement of pixels in the multiple frames of images includes: Preprocessing the multiple frames of images to determine the processed multiple frames of images; The processed multiple frames of images are compensated to suppress the displacement of pixels in the multiple frames of images.

7. The method according to claim 5, characterized in that The compensating the multiple frames of images captured by the vehicle camera to suppress the displacement of pixels in the multiple frames of images includes: Determining displacement information of pixels in the multiple frames of images according to the multiple frames of images; The multiple frames of images are compensated according to the displacement information to suppress the displacement of pixels in the multiple frames of images.

8. The method according to claim 7, characterized in that The displacement information includes an optical flow vector, and determining the displacement information of pixels in the multiple frames of images according to the multiple frames of images includes: According to the pixel points in the multiple frames of images, optical flow vectors corresponding to the multiple frames of images are determined.

9. The method according to claim 7, characterized in that: The displacement information includes a displacement amount, the multiple frames of images include a previous frame of image and a next frame of image taken continuously, and determining the displacement information of pixels in the multiple frames of image according to the multiple frames of image includes: Determine, according to the multiple frames of images, a first position of a feature point in the previous frame of image and a second position of the feature point in the next frame of image; The displacement amount is determined according to the first position and the second position.

10. The method according to claim 7, characterized in that The multiple frames of images include a previous frame of image and a next frame of image taken continuously, and the compensating the multiple frames of images according to the displacement information to suppress the displacement of pixels in the multiple frames of images includes: According to the displacement information, the pixel value and the pixel position of the image pixel point in the next frame of image are adjusted.

11. The method according to claim 10, characterized in that The step of adjusting the pixel value and the pixel position of the image pixel point in the next frame of image according to the displacement information comprises: According to the displacement information, adjusting the position of the image pixel in the next frame of image; The pixel value of the image pixel point is adjusted according to the pixel value of the image pixel point and the pixel value of the reference pixel point, wherein the image pixel point and the reference pixel point are adjacent in the subsequent frame image.

12. A control device, characterized in that: The device comprises a transceiver unit and a processing unit; The transceiver unit is configured to obtain the position information of the vehicle; The processing unit is configured to adjust the posture of the vehicle camera according to the posture information so that the shooting angle of the vehicle camera maintains a steady state.

13. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 11 is implemented.

14. A vehicle, characterized in that: The vehicle comprises a vehicle camera, and the vehicle further comprises the device of claim 12 or 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 11 is implemented.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images