Camera frame rate adjustment method for vehicle-mounted camera and vehicle thereof

Abstract

This invention relates to the field of vehicle technology, and more particularly to a method, vehicle, device, and medium for adjusting the frame rate of an in-vehicle camera. The method includes: setting an initial frame rate for each camera; acquiring the current vehicle speed; acquiring images outside the vehicle through at least one camera; processing the images using an image algorithm to obtain the moving speed of a target object in the image; calculating the relative speed of the target object based on the current vehicle speed and the moving speed of the target object in the image; calculating the corresponding frame rate for each camera using a frame rate motion weighting model, combining the relative speed of the target object and the initial frame rate of each camera; and adjusting the frame rate of the corresponding camera based on the frame rate. This invention's method for adjusting the frame rate of an in-vehicle camera reduces camera power consumption by dynamically adjusting the frame rate, thereby reducing the power consumption of the entire domain controller and improving the stability and lifespan of the camera operation.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more particularly to a method for adjusting the frame rate of an in-vehicle camera, a vehicle, equipment, and a medium thereof. Background Technology

[0002] Currently, low-power design of automotive domain controllers mainly relies on system-level sleep strategies and dynamic shutdown mechanisms for functional modules. However, power optimization methods for the visual perception modules relied upon by ADAS (Advanced Driver Assistance Systems) remain insufficient. Existing ADAS vision systems employ a fixed frame rate operating mode: upon system power-on, each camera (including forward-looking, panoramic, and surround-view cameras) is initialized and configured, and a fixed image acquisition frame rate is assigned. In actual operation, regardless of changes in the vehicle's driving environment, each camera maintains a constant high frame rate. This "all-weather, all-scenario" high-power operating mode has the following significant drawbacks: First, energy efficiency is low. As the level of intelligent driving increases, the number of ADAS cameras in a single vehicle has increased from 1-2 in the early days to more than 10. The power consumption of the visual perception module accounts for a continuously rising proportion of the total power consumption of the vehicle's domain controller, and in some high-performance computing platforms, this proportion has exceeded 40%. The fixed frame rate working mode causes the camera to maintain a high frame rate acquisition even in scenarios with low requirements for real-time image acquisition, such as low-speed cruising, traffic jam following, and parking waiting, resulting in a large amount of ineffective energy consumption.

[0003] Second, hardware lifespan degradation. Image sensors and signal processing links operate under high load for extended periods, accelerating the accumulation of thermal stress and the aging process of the components, affecting the long-term operational stability and lifespan of the camera. Summary of the Invention

[0004] The technical problem this invention aims to solve is the low energy efficiency, low long-term operational stability, and short service life of existing cameras. This invention provides a method for adjusting the frame rate of an in-vehicle camera. By dynamically adjusting the frame rate, the power consumption of the camera is reduced, thereby reducing the power consumption of the entire domain controller and improving the stability and service life of the camera.

[0005] The technical solution adopted by this invention to solve its technical problem is: a method for adjusting the frame rate of a vehicle-mounted camera, the method comprising the following steps: S1, Initial frame rate settings for each camera; S2, obtain the vehicle's current speed; S3, acquires images outside the vehicle using at least one camera; S4: Process the image using an image algorithm to obtain the moving speed of the target object in the image; S5, calculate the relative speed of the target object based on the vehicle's current driving speed and the target object's moving speed in the image; S6. Calculate the corresponding camera frame rate for each camera by using a frame rate motion weighted model, combining the relative speed of the target object and the initial frame rate of each camera. The S7 adjusts the frame rate of the corresponding camera based on the video frame rate.

[0006] Furthermore, specifically, step S6 includes the following steps: Obtain the current time difference and calculate the current exponential decay factor α of the camera based on the current time difference; The current frame rate of the camera is obtained based on the relative speed of the target object. ; The current adjustment factor is calculated based on the camera's current frame rate and its initial frame rate. f ; Based on the current exponential decay factor α and the current adjustment factor f Calculate the current overall weight ; Based on the current camera frame rate and current overall weight The current smooth frame rate of the camera is calculated by combining the frame rate moving weighted average algorithm. Update the camera's initial frame rate for the next moment based on the camera's current frame rate and initial frame rate.

[0007] Furthermore, specifically, the current formula for calculating the exponential decay factor α is as follows: ; Current regulating factor f The calculation formula is: ; Current overall weight The calculation formula is:

[0008] in, As the current overall weight, f i Where α is the current frame rate of the camera, α is the exponential decay factor, and β is the adjustment intensity. Let λ be the initial frame rate of the camera, and λ be the decay rate. For time difference.

[0009] Furthermore, specifically, the calculation formula for the frame rate moving weighted average algorithm is as follows: .

[0010] Furthermore, specifically, step S4 includes the following steps: The image is detected by an object detection model to obtain the current detection result, which includes the object and the pixel coordinates of the object. The current detection result is compared with the detection result of the previous frame to determine if there is an update; If so, the relative velocity of the target object is calculated based on the pixel coordinates of the target object in the current image and the pixel coordinates of the target object in the previous frame image; If not, the relative velocity of the target object in the previous frame is the relative velocity of the current target object.

[0011] Furthermore, specifically, the cameras include: surround view cameras, front view cameras, and panoramic view cameras.

[0012] Furthermore, specifically, calculating the relative velocity of the target object includes: The relative speed of the target object is determined by averaging the current vehicle speed and the moving speed of the target object in the image.

[0013] A vehicle employing the image frame rate adjustment method for an onboard camera as described above.

[0014] A computer device, comprising: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the camera frame rate adjustment method for an in-vehicle camera as described above.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the image frame rate adjustment method for an in-vehicle camera as described above.

[0016] The beneficial effects of this invention are that the camera frame rate adjustment method for vehicle-mounted cameras calculates the relative speed of the target object based on the vehicle's current driving speed and the moving speed of the target object in the image, and dynamically adjusts the camera frame rate based on the relative speed of the target object. The adjustment accuracy is high, and while reducing the power consumption of the camera, it also reduces the power consumption of the entire domain controller, significantly improving the system's energy efficiency, ensuring the stability of the camera during long-term operation, reducing equipment failures and performance degradation caused by excessive energy consumption, and thus significantly increasing the service life of the camera. Attached Figure Description

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present invention.

[0019] Figure 2 This is a schematic diagram of the hardware structure of a computer device according to Embodiment 2 of the present invention.

[0020] In the diagram, 10 is a computer device; 1002 is a processor; 1004 is a memory; and 1006 is a transmission device. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] Example 1: This invention provides a method for adjusting the frame rate of an in-vehicle camera, such as... Figure 1 As shown, the method includes the following steps: S1. Set the initial frame rate for each camera, including a surround view camera, a front view camera, and a panoramic view camera. Further, the initial frame rate of the surround view camera and the panoramic view camera is set to 30 frames / second, and the initial frame rate of the front view camera is set to 60 frames / second. S2, obtain the vehicle's current speed; obtain the vehicle's current speed from the vehicle's electronic control unit (ECU) via the CAN network protocol.

[0023] S3, acquires images outside the vehicle using at least one camera; S4, processing the image using image algorithms to obtain the moving speed of the target object in the image; specifically including: The image is detected by an object detection model, and the current detection result is obtained. The current detection result includes the object and the pixel coordinates of the object. The current detection result is compared with the detection result of the previous frame to determine if there is an update; If so, the relative velocity of the target object is calculated based on the pixel coordinates of the target object in the current image and the pixel coordinates of the target object in the previous frame image; If not, the relative velocity of the target object in the previous frame is the relative velocity of the current target object.

[0024] S5. Based on the vehicle's current speed and the target's moving speed in the image, calculate the relative speed of the target. Specifically, calculate the average of the vehicle's current speed and the target's moving speed in the image to determine the target's relative speed. S6 calculates the corresponding camera frame rate for each camera using a frame rate-weighted motion model, combining the relative velocity of the target object and the initial frame rate of each camera; specifically: Obtain the current time difference, and calculate the current exponential decay factor α of the camera based on the current time difference. The calculation formula is as follows:

[0025] ,in, Let λ be the time difference and λ be the decay rate. The larger λ is, the faster the decay and the faster the weight of historical data decreases; the smaller λ is, the slower the decay and the slower the weight of historical data decreases.

[0026] The current frame rate of the camera is obtained based on the relative speed of the target object. As in one specific embodiment, based on the current frame rate of the camera frame rate The calculation formula is:

[0027] in, The relative velocity of the current target object. The current adjustment factor is calculated based on the camera's current frame rate and initial frame rate. f The calculation formula is:

[0028] in, f i This represents the current frame rate of the camera. This is the camera's current initial frame rate.

[0029] Based on the current exponential decay factor α and the current adjustment factor f Calculate the current overall weight The calculation formula is:

[0030] Where β is the adjustment strength, β determines the sensitivity of weight adjustment when the frame rate deviates from the initial frame rate. The larger β is, the stronger the adjustment, the more the weight increases at high frame rates, and the more the weight decreases at low frame rates. β=0 means no adjustment, and the frame rate does not affect the weight.

[0031] Based on the current camera frame rate and current overall weight The current smooth frame rate of the camera is calculated using a moving weighted average frame rate algorithm; furthermore, the calculation formula for the moving weighted average frame rate algorithm is as follows: .

[0032] The initial frame rate of the camera at the next moment is updated based on the current frame rate and the initial frame rate of the camera. The formula for calculating the initial frame rate of the camera at the next moment is:

[0033] The S7 adjusts the frame rate of the corresponding camera based on the video frame rate.

[0034] Furthermore, step S5 is described in detail as follows: the initial frame rate of the forward-view camera is preset to 30 frames / second, λ=0.5, and β is 0.3. For the first frame of the camera, at t=0.0s, the relative speed of the target object is 30.24 km / h. The corresponding calculated current frame rate of the camera is... =10.2, initially t_prev = None (no previous time value), the time interval is the sampling interval of the calculation period, the default is 0.1S, that is, the time difference Δt = 0.1, calculate the exponential decay factor: = =0.9512.

[0035] The current adjustment factor is calculated based on the camera's current frame rate and its initial frame rate: f= (10.2-30) / 30=-0.66.

[0036] Based on the current exponential decay factor α and the current adjustment factor f Calculate the current overall weight: =0.763.

[0037] Based on the current camera frame rate and current overall weight The current smooth frame rate of the camera is calculated using a frame rate moving weighted average algorithm: = = 10.20 fps.

[0038] Update the window: window_f = [(10.2, 0.763)].

[0039] The initial frame rate of the camera at the next moment is updated based on the current frame rate and the initial frame rate of the camera. The formula for calculating the initial frame rate of the camera at the next moment is: = 0.3×10.2 + 0.7×30.0 = 24.06.

[0040] For the second frame rate calculation of the camera, at t=0.1s, the relative speed of the target object is 31.8 km / h, and the corresponding calculated current frame rate of the camera is... =11.5, the camera's initial frame rate is 24.06, and the calculated time difference Δt = 0.1 -0.0 = 0.1s.

[0041] Calculate the exponential decay factor based on the time difference: = =0.9512.

[0042] The current adjustment factor is calculated based on the camera's current frame rate and its initial frame rate: f= (24.06 -30) / 30 = -0.198.

[0043] Based on the current exponential decay factor α and the current adjustment factor f Calculate the current overall weight: = 0.895.

[0044] Based on the current camera frame rate and current overall weight The current smooth frame rate of the camera is calculated using a frame rate moving weighted average algorithm: = = 10.90fps.

[0045] Update the window: window_f = [(10.2,0.763), (11.5,0.895)].

[0046] The initial frame rate of the camera at the next moment is updated based on the current frame rate and the initial frame rate of the camera. The formula for calculating the initial frame rate of the camera at the next moment is: = 0.3×11.5 + 0.7×24.06 = 24.06.

[0047] The calculation process for the frame rate of the 3rd frame, 4th frame, ..., nth frame of the camera is the same as that for the second frame, and will not be repeated here for the sake of brevity in the manual.

[0048] It should be noted that when the number of windows reaches the upper limit, the data of the earliest frame will be removed. For example, if the number of windows is 5, the data of the earliest frame will be removed.

[0049] In summary, the image frame rate adjustment method for a vehicle-mounted camera in this embodiment calculates the relative speed of the target object based on the vehicle's current driving speed and the moving speed of the target object in the image, and dynamically adjusts the image frame rate based on the relative speed of the target object. The adjustment accuracy is high, and while reducing the power consumption of the camera, it also reduces the power consumption of the entire domain controller, significantly improving the system's energy efficiency, ensuring the stability of the camera during long-term operation, reducing equipment failures and performance degradation caused by excessive energy consumption, and thus significantly increasing the lifespan of the camera.

[0050] Example 2: This invention provides a vehicle that employs the above-described method for adjusting the frame rate of an in-vehicle camera.

[0051] Example 3: This invention provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a video frame rate adjustment method for an in-vehicle camera as provided in the above method embodiments.

[0052] Figure 2 This diagram illustrates a hardware structure of a device for implementing a method for adjusting the frame rate of a vehicle-mounted camera as provided in an embodiment of the present invention. The device may constitute or include the apparatus or system provided in the embodiments of the present invention. Figure 2 As shown, the computer device 10 may include one or more processors 1002 (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer device 10 may also include... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0053] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be wholly or partially embodied in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuit may be a single, independent processing module, or wholly or partially integrated into any other element within the computer device 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0054] The memory 1004 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to a method for adjusting the frame rate of a vehicle-mounted camera in an embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby implementing the aforementioned method. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor, and these remote memories can be connected to the computer device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0055] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer device 10. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0056] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer device 10 (or mobile device).

[0057] Example 4: This invention also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a method for adjusting the frame rate of an in-vehicle camera in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method for adjusting the frame rate of an in-vehicle camera provided in the above method embodiment.

[0058] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0059] Example 5: This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a camera frame rate adjustment method for an in-vehicle camera provided in the various optional embodiments described above.

[0060] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0062] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0063] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for adjusting the frame rate of a vehicle-mounted camera, characterized in that, The method includes the following steps: S1, Initial frame rate settings for each camera; S2, obtain the vehicle's current speed; S3, acquires images outside the vehicle using at least one camera; S4: Process the image using an image algorithm to obtain the moving speed of the target object in the image; S5, calculate the relative speed of the target object based on the vehicle's current driving speed and the target object's moving speed in the image; S6, calculate the corresponding camera frame rate of each camera by combining the relative speed of the target object and the initial frame rate of each camera through the frame rate motion weighting model; The S7 adjusts the frame rate of the corresponding camera based on the video frame rate.

2. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 1, characterized in that, Step S6 specifically includes the following steps: Obtain the current time difference and calculate the current exponential decay factor α of the camera based on the current time difference; The current frame rate of the camera is obtained based on the relative speed of the target object. ; The current adjustment factor is calculated based on the camera's current frame rate and its initial frame rate. f ; Based on the current exponential decay factor α and the current adjustment factor f Calculate the current overall weight ; Based on the current camera frame rate and current overall weight The current smooth frame rate of the camera is calculated by combining the frame rate moving weighted average algorithm. Update the camera's initial frame rate for the next moment based on the camera's current frame rate and initial frame rate.

3. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 2, characterized in that, The current formula for calculating the exponential decay factor α is: ； Current regulating factor f The calculation formula is: ； Current overall weight The calculation formula is: in, As the current overall weight, f i Where α is the current frame rate of the camera, α is the exponential decay factor, and β is the adjustment intensity. Let λ be the initial frame rate of the camera, and λ be the decay rate. For time difference.

4. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 2, characterized in that, The calculation formula for the frame rate moving weighted average algorithm is as follows: 。 5. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 1, characterized in that, Step S4 specifically includes the following steps: The image is detected by an object detection model to obtain the current detection result, which includes the object and the pixel coordinates of the object. The current detection result is compared with the detection result of the previous frame to determine if there is an update; If so, the relative velocity of the target object is calculated based on the pixel coordinates of the target object in the current image and the pixel coordinates of the target object in the previous frame image; If not, the relative velocity of the target object in the previous frame is the relative velocity of the current target object.

6. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 1, characterized in that, The cameras include: surround view cameras, front view cameras, and panoramic view cameras.

7. The method for adjusting the frame rate of a vehicle-mounted camera according to claim 1, characterized in that, Calculating the relative velocity of the target object specifically includes: The relative speed of the target object is determined by averaging the current vehicle speed and the moving speed of the target object in the image.

8. A vehicle, characterized in that, The vehicle employs the camera frame rate adjustment method for an in-vehicle camera as described in any one of claims 1 to 7.

9. A computer device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the camera frame rate adjustment method for a vehicle-mounted camera as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the video frame rate adjustment method for an in-vehicle camera as described in any one of claims 1 to 7.

Images