Vehicle-mounted camera frame rate parameter adjusting method, vehicle-mounted controller and program product

Abstract

The invention relates to the technical field of vehicle control, and provides a vehicle-mounted camera frame rate parameter adjustment method, a vehicle-mounted controller and a computer program product. The method comprises the following steps: acquiring sensing data of a target vehicle in multiple dimensions; wherein the sensing data of different dimensions correspond to different priorities respectively; and adjusting a frame rate parameter of a vehicle-mounted camera of the target vehicle according to the sensing data of the multiple dimensions and the priorities corresponding to the sensing data of the multiple dimensions. By adopting the method, the frame rate parameter of the vehicle-mounted camera can be reasonably adjusted according to the sensing data of the vehicle, so that the frame rate parameter cannot be continuously fixed at a high value, continuous high-power-consumption operation of each vehicle component can be avoided, the power consumption of a battery of the vehicle is saved, and the endurance mileage of the vehicle is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method for adjusting the frame rate parameters of an in-vehicle camera, an in-vehicle controller, and a computer program product. Background Technology

[0002] With the rapid development of vehicle intelligence, the performance and function of in-vehicle cameras are gradually improving. For example, in-vehicle cameras can capture images of the in-vehicle scene to improve occupant safety, while cameras in electronic rearview mirrors can collect images of the rear view of the vehicle and display them to the driver in real time, providing assistance for safe driving. However, existing in-vehicle camera systems typically use fixed and high frame rate parameters, such as video capture frame rate, display refresh rate, and video encoding / decoding rate. These fixed high frame rate parameters cause vehicle components such as in-vehicle cameras, image processing chips, and displays to operate at high power consumption continuously, placing a significant burden on the vehicle battery and affecting the vehicle's driving range. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method for adjusting the frame rate parameters of an in-vehicle camera, an in-vehicle controller, and a computer program product, which can reasonably adjust the frame rate parameters of the in-vehicle camera based on the vehicle's perception data, avoid continuous high power consumption operation of various vehicle components, thereby saving vehicle battery power consumption and improving vehicle range.

[0004] The first aspect of this application provides a method for adjusting the frame rate parameter of an in-vehicle camera, including: Acquire perception data of the target vehicle from multiple dimensions; among which, the perception data of different dimensions correspond to different priorities; Based on multiple dimensions of perception data and their respective priorities, the frame rate parameters of the vehicle's onboard camera are adjusted.

[0005] In the technical solution of this application embodiment, firstly, multiple dimensions of perception data of the target vehicle are acquired, with different dimensions of perception data corresponding to different priorities. Then, based on the multiple dimensions of perception data and their respective priorities, the frame rate parameters of the vehicle's onboard camera are adjusted. This process intelligently perceives multiple dimensions of perception data and pre-assigns different priorities to different dimensions, enabling reasonable adjustment of the onboard camera's frame rate parameters according to the current perception data and their corresponding priorities. For example, assuming the multiple dimensions of perception data include emergency scenarios, driver behavior, and vehicle speed, with emergency scenarios having the highest priority, when the vehicle is not in an emergency scenario, the frame rate parameters of the onboard camera can be adjusted to a moderate value based on driver behavior or vehicle speed. However, when the vehicle enters an emergency scenario, driver behavior and vehicle speed are disregarded, and the frame rate parameters of the onboard camera are directly adjusted to the highest value. As can be seen from the above examples, the technical solution of this application embodiment can reasonably adjust the frame rate parameter according to the current actual situation of the vehicle. In this way, the frame rate parameter will not be fixed at a high value, thus avoiding continuous high power consumption operation of various vehicle components, thereby saving vehicle battery power consumption and improving vehicle range.

[0006] In one implementation of this application, the frame rate parameter of the vehicle-mounted camera of the target vehicle is adjusted according to multiple dimensions of perception data and the priorities corresponding to each dimension of perception data, including: If at least one dimension of the perception data in multiple dimensions satisfies the preset frame rate adjustment conditions, then the perception data of the target dimension with the highest priority is determined from the perception data of at least one dimension. The frame rate parameters are adjusted based on the perceived data of the target dimension.

[0007] In one implementation of this application, the target dimension perception data is the real-time speed of the target vehicle; adjusting the frame rate parameter based on the target dimension perception data includes: Based on the preset vehicle speed frame rate mapping relationship, the optimal frame rate corresponding to the real-time vehicle speed is determined; wherein, the vehicle speed frame rate mapping relationship records the optimal frame rate corresponding to different vehicle speeds respectively; The frame rate parameter is smoothly adjusted from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed.

[0008] In one implementation of this application, the target dimension perception data consists of multi-frame facial images of the driver of the target vehicle; the frame rate parameter is adjusted based on the target dimension perception data, including: For each frame of facial image, the gaze state corresponding to that frame of facial image is determined based on the driver's gaze direction in that frame of facial image; wherein, the gaze state includes gaze and no gaze, which are used to indicate whether the driver is looking at the display area of ​​the vehicle camera; If the proportion of frames in a multi-frame facial image that correspond to a gazed state exceeds the first threshold, then the driver's gaze area is determined to be the display area of ​​the screen; otherwise, the driver's non-gaze area is determined to be the display area of ​​the screen. If it is determined that the driver is looking at the display area, the frame rate parameter is adjusted to the first frame rate; if it is determined that the driver is not looking at the display area, the frame rate parameter is adjusted to the second frame rate, with the first frame rate being higher than the second frame rate.

[0009] In one implementation of this application embodiment, determining the gaze state corresponding to the facial image frame based on the driver's gaze direction in the facial image frame includes: Perform eye region detection on the facial image frame to determine the coordinates of the pupil center point and the corner of the eye; Determine the driver's line of sight vector based on the coordinates of the pupil center point and the corner of the eye; The gaze state corresponding to the facial image in that frame is determined based on the geometric relationship between the gaze vector and the display area.

[0010] In one implementation of this application, the perception data for the target dimension consists of multi-frame images of the exterior scene of the target vehicle; the frame rate parameter is adjusted based on the perception data for the target dimension, including: Based on multiple frames of images of the exterior of the vehicle, determine whether there is a moving target object rapidly approaching the target vehicle; If a moving object is identified, the frame rate parameter is adjusted to the third frame rate.

[0011] In one implementation of this application, determining whether a target moving object is rapidly approaching the target vehicle based on multiple frames of exterior scene images includes: Based on multiple frames of exterior scene images, detect and track all moving objects in the exterior scene of the target vehicle that are moving toward the target vehicle; For each moving object among all moving objects, if the moving speed of the moving object exceeds the second threshold, or if the size growth rate of the moving object's image in the multi-frame exterior scene image exceeds the third threshold, then the moving object is identified as the target moving object.

[0012] In one implementation of this application, the perception data for the target dimension is sensor data of the target vehicle; adjusting the frame rate parameter based on the perception data for the target dimension includes: Based on sensor data, determine whether the target vehicle is experiencing an emergency. If an emergency is detected in the target vehicle, the frame rate parameter is adjusted to the fourth frame rate.

[0013] A second aspect of this application provides a device for adjusting the frame rate parameters of an in-vehicle camera, comprising: The perception data acquisition module is used to acquire perception data of the target vehicle from multiple dimensions; among which, the perception data of different dimensions correspond to different priorities. The frame rate parameter adjustment module is used to adjust the frame rate parameters of the vehicle's onboard camera based on multiple dimensions of perception data and the priorities of each dimension of perception data.

[0014] A third aspect of this application provides an in-vehicle controller, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the in-vehicle camera frame rate parameter adjustment method provided in the first aspect of this application.

[0015] A fourth aspect of this application provides a computer program product that, when run on an in-vehicle controller, causes the in-vehicle controller to execute the in-vehicle camera frame rate parameter adjustment method provided in the first aspect of this application.

[0016] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle camera frame rate parameter adjustment method provided in the first aspect of this application.

[0017] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method for adjusting the frame rate parameter of an in-vehicle camera provided in an embodiment of this application; Figure 2 This is a schematic diagram of a system technical architecture used in the vehicle camera frame rate parameter adjustment method provided in this application embodiment; Figure 3 This is a schematic diagram of a vehicle speed frame rate mapping relationship provided in an embodiment of this application; Figure 4 This is a schematic diagram of an operation process for adjusting the frame rate parameters of an in-vehicle camera based on a driver's facial image, provided in an embodiment of this application. Figure 5This is a schematic diagram of an operation process provided in this application embodiment to adjust the frame rate parameters of an in-vehicle camera by detecting a rapidly approaching target; Figure 6 This is a schematic diagram of an operation process for adjusting the frame rate parameters of an in-vehicle camera based on sensor data, provided in an embodiment of this application. Figure 7 This is a structural framework diagram of a vehicle-mounted camera frame rate parameter adjustment device provided in an embodiment of this application; Figure 8 This is a schematic diagram of an in-vehicle controller provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0020] Currently, automotive cameras such as electronic rearview mirror cameras and in-vehicle cameras typically use fixed and high frame rate parameters. This causes vehicle components, including the camera, image processing chip, and display screen, to operate at continuously high power consumption, severely depleting the vehicle's battery and reducing its driving range. Furthermore, high frame rate video data is massive, consuming a significant amount of storage space.

[0021] To address the aforementioned technical problems, this application provides a method for adjusting the frame rate parameters of an in-vehicle camera, an in-vehicle controller, and a computer program product. These methods can rationally adjust the frame rate parameters of the in-vehicle camera based on vehicle perception data, avoiding continuous high-power operation of various vehicle components, thereby saving vehicle battery power consumption, increasing vehicle range, and saving data storage space. For more specific technical implementation details of this application's embodiments, please refer to the method embodiments described below.

[0022] It should be understood that the execution subject of the various method embodiments of this application can be various types of vehicle controllers, such as vehicle electronic control units, vehicle terminals, or electronic rearview mirror controllers, etc. The embodiments of this application do not limit the specific type of vehicle controller.

[0023] Please see Figure 1 This application illustrates a method for adjusting the frame rate parameter of an in-vehicle camera according to an embodiment of the present application, including: 101. Acquire perception data of the target vehicle from multiple dimensions; where different dimensions of perception data correspond to different priorities; First, acquire perception data from multiple dimensions of the target vehicle. The target vehicle refers to any vehicle equipped with an onboard camera (such as a camera in an electronic rearview mirror or an in-vehicle camera). The perception data from various dimensions may include, but is not limited to: vehicle speed, driver's facial image, external scene image, accelerometer, gyroscope or airbag sensor data, geographical location, environmental parameters, driver behavior, driving scenario, and vehicle status, etc.

[0024] Each dimension of perceived data is pre-assigned a corresponding priority, with different priorities for different dimensions. This determines which dimension of perceived data takes precedence as the control factor for the frame rate parameter of the vehicle camera. For example, suppose the multiple dimensions of perceived data include vehicle speed, driver facial image, exterior scene image, and sensor data. Considering that sensor data can detect emergency scenarios such as sudden braking or collisions, exterior scene images can detect hazards outside the vehicle, driver facial images can detect whether the driver needs to view the camera feed, and vehicle speed is a normal control factor under stable conditions, we can assign the highest priority (1) to sensor data, priority (2) to exterior scene images, priority (3) to driver facial images, and the lowest priority (4) to vehicle speed. With this setting, when multiple dimensions of perceived data simultaneously trigger frame rate parameter adjustment, the perception data with the highest priority determines how to adjust the frame rate parameter. For example, if sensor data detects that the vehicle has entered an emergency scenario, the frame rate parameter needs to be adjusted to the highest level. However, by detecting vehicle speed, the frame rate parameter needs to be adjusted to a lower value. Since sensor data has higher priority than vehicle speed, the frame rate parameter of the vehicle camera is adjusted to... Instead .

[0025] 102. Adjust the frame rate parameters of the vehicle's onboard camera based on multiple dimensions of perception data and the corresponding priorities of each dimension of perception data.

[0026] After acquiring multi-dimensional perception data of the target vehicle, the system analyzes the perception data of each dimension and its corresponding priority to determine how to adjust the frame rate parameters of the vehicle's onboard cameras. These frame rate parameters can include video capture frame rate, display refresh rate, and video encoding / decoding rate. Specifically, by analyzing the perception data of each dimension, it can be determined whether the frame rate parameters of each dimension need adjustment and by what amount. Then, considering the priority of each dimension of perception data, the control factors for adjusting the frame rate parameters are determined in descending order of priority. For example, assuming there is perception data A (priority 1), perception data B (priority 2), and perception data C (priority 3), and the current frame rate parameter is a normal value... If, based on the perception data A, it is determined that the frame rate parameter needs to be adjusted to... Based on the perceived data B, it was determined that the frame rate parameter needed to be adjusted to... Based on the perceived data C, it was determined that the frame rate parameter needed to be adjusted to... Since the perceived data A has the highest priority, the execution result is to change the frame rate parameter from... Adjust to If, based on perception data A, it is determined that no frame rate adjustment is needed, but based on perception data B, it is determined that the frame rate parameter needs to be adjusted... Based on the perceived data C, it was determined that the frame rate parameter needed to be adjusted to... At this point, only the sensing data B and sensing data C that trigger frame rate adjustment are considered. Since sensing data B has a higher priority, the execution result is to adjust the frame rate parameter from... Adjust to If, based on perception data A, it is determined that no frame rate adjustment is needed, and based on perception data B, it is determined that no frame rate adjustment is needed, but based on perception data C, it is determined that the frame rate parameter needs to be adjusted... At this point, only the perceived data C that triggers the frame rate adjustment is considered, so the execution result is to change the frame rate parameter from... Adjust to And so on.

[0027] In one implementation of this application, the frame rate parameter of the vehicle-mounted camera of the target vehicle is adjusted according to multiple dimensions of perception data and the priorities corresponding to each dimension of perception data, including: (1) If there is at least one dimension of the perception data that satisfies the preset frame rate adjustment condition, then the perception data of the target dimension with the highest priority is determined from the perception data of at least one dimension. (2) Adjust the frame rate parameters based on the perceptual data of the target dimension.

[0028] Data analysis of the perceived data in each dimension can determine whether the perceived data in that dimension meets the preset frame rate adjustment conditions, i.e., whether a frame rate adjustment operation is triggered. For example, for vehicle speed, if the vehicle speed changes or reaches a certain set threshold, it can be determined that the vehicle speed meets the frame rate adjustment conditions; for driver facial images, if the driver's gaze at the camera image changes (e.g., from gaze to non-gaze, or from non-gaze to gaze), it can be determined that the driver facial image meets the frame rate adjustment conditions; for external scene images, if a rapidly approaching moving object is detected based on the external scene image, it can be determined that the external scene image meets the frame rate adjustment conditions; for sensor data, if the sensor data detects that the vehicle is entering an emergency scenario such as sudden braking or a collision, it can be determined that the sensor data meets the frame rate adjustment conditions, and so on. If no perceived data in multiple dimensions meets the frame rate adjustment conditions, then there is no need to adjust the frame rate parameters of the vehicle camera. If at least one dimension of the perceived data satisfies the frame rate adjustment condition, the highest priority target dimension of perceived data is first determined from that dimension, and then the frame rate parameter is adjusted based on the target dimension. For example, suppose the multiple dimensions of perceived data include vehicle speed, driver's facial image, exterior scene image, and sensor data, arranged in ascending order of priority. If only vehicle speed satisfies the frame rate adjustment condition, then the target dimension of perceived data is vehicle speed, and the frame rate parameter is adjusted accordingly. If both the exterior scene image and sensor data satisfy the frame rate adjustment condition, then the target dimension of perceived data is sensor data, and the frame rate parameter is adjusted accordingly. If vehicle speed, driver's facial image, and exterior scene image all satisfy the frame rate adjustment condition, then the target dimension of perceived data is the exterior scene image, and the frame rate parameter is adjusted accordingly, and so on.

[0029] It can be seen that by combining perception data from various dimensions and their corresponding priorities for comprehensive decision-making, adaptive control and adjustment of the frame rate parameters of the vehicle camera based on the actual vehicle conditions can be achieved. This avoids the frame rate parameters from being continuously fixed at a high value while ensuring driving safety, significantly reduces system power consumption, increases the vehicle's driving range, and extends the video data storage time, thereby improving the user experience.

[0030] As an example, Figure 2 This is a schematic diagram of a system technical architecture used in the vehicle camera frame rate parameter adjustment method provided in this application embodiment. Figure 2The system architecture shown includes a multi-dimensional information perception module, a scene recognition and decision-making module, a frame rate control execution module, and a safety monitoring module. The multi-dimensional information perception module includes a vehicle speed perception unit, a driver behavior perception unit, an environmental target perception unit, and an emergency scene perception unit. The vehicle speed perception unit can acquire real-time vehicle speed information through CAN bus, GPS positioning module, and vehicle pulse sensor. The driver behavior perception unit can detect the driver's gaze direction through image recognition technology to assess whether the driver is looking at the camera feed. The environmental target perception unit can detect the presence of rapidly approaching objects in the vehicle's external environment through moving target detection technology. The emergency scene perception unit can detect emergency situations using devices such as acceleration sensors or brake sensors. The scene recognition and decision-making module includes a scene classification unit, a frame rate decision-making unit, and a priority arbitration unit. The system can classify scenes based on the fusion of multi-dimensional perception information. The frame rate decision unit can intelligently decide the target frame rate to be adjusted according to the scene type. The priority arbitration unit is used to handle priority arbitration when multiple conditions are triggered simultaneously (i.e., multiple dimensions of perception data trigger frame rate adjustment operations). The frame rate control execution module includes a camera frame rate control unit, a display refresh rate control unit, and an encoding parameter control unit. The camera frame rate control unit is used to control the video acquisition frame rate of the camera, the display refresh rate control unit is used to control the refresh frequency of the display screen, and the encoding parameter control unit is used to control the bitrate and frame rate of video encoding and decoding. The security monitoring module includes an anomaly detection unit, an emergency response unit, and a log recording unit. The anomaly detection unit is used to monitor abnormal situations in the system, the emergency response unit is used to trigger a rapid response mechanism in emergency situations, and the log recording unit is used to record key operation information and historical data, such as historical changes in frame rate parameters.

[0031] Next, we will describe in turn the specific implementation methods of adjusting frame rate parameters based on real-time vehicle speed, adjusting frame rate parameters based on driver facial images, adjusting frame rate parameters based on external scene images, and adjusting frame rate parameters based on sensor data.

[0032] In one implementation of this application, the target dimension perception data is the real-time speed of the target vehicle; adjusting the frame rate parameter based on the target dimension perception data includes: (1) Determine the optimal frame rate corresponding to the real-time vehicle speed according to the preset vehicle speed frame rate mapping relationship; wherein, the vehicle speed frame rate mapping relationship records the optimal frame rate corresponding to different vehicle speeds respectively; (2) The frame rate parameter is smoothly adjusted from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed.

[0033] If the real-time speed of the target vehicle meets the frame rate adjustment conditions, and no other dimension of perception data with higher priority meets the frame rate adjustment conditions, then the real-time speed is used as the target dimension of perception data. At this point, the operation process of adjusting the frame rate parameters based on the real-time speed begins. Real-time speed can be obtained in various ways, and the optimal data source can be automatically selected based on vehicle configuration and availability. For example, vehicle speed information can be read in real-time through the target vehicle's CAN bus interface; CAN bus speed data has the advantages of high accuracy and high real-time performance. Real-time speed can also be calculated using positioning modules such as GPS; this method is suitable for vehicles without a CAN bus interface or as a backup data source. Alternatively, vehicle speed can be calculated using the pulse signals from wheel speed sensors; this method is suitable for traditional vehicles, and the relationship between pulse frequency and vehicle speed can be expressed by the following formula:

[0034] in, Indicates real-time vehicle speed (km / h). Indicates pulse frequency (Hz). This indicates the circumference of the wheel (m). This indicates the number of pulses per revolution, and 3.6 is the unit conversion factor from m / s to km / h.

[0035] If the target vehicle has multiple real-time speed data sources, each data source can be assigned a corresponding priority. For example, CAN bus data, with its highest accuracy, can be assigned the highest priority, GPS positioning data can be assigned a medium priority, and pulse sensor data can be assigned the lowest priority. When acquiring real-time speed, CAN bus data is used first. If CAN bus data acquisition fails, it switches to GPS positioning data, and if that fails, it switches to pulse sensor data. This demonstrates a fault-tolerant mechanism that automatically switches to a backup data source when the primary data source fails.

[0036] In addition, to prevent vehicle speed signal jitter, a sliding window mean filtering method can be used to smooth the vehicle speed signal. The corresponding processing formula is as follows:

[0037] in, This represents the vehicle speed after smoothing at time t. This refers to the size of the sliding window, typically 5-10. It is the sampling time interval. This represents historical vehicle speed data.

[0038] A vehicle speed-frame rate mapping relationship is pre-built, which records the optimal frame rate corresponding to different vehicle speeds. For example, a certain vehicle speed-frame rate mapping relationship can be represented by the following formula:

[0039] in, (Unit: km / h) indicates real-time vehicle speed. (Unit: fps) indicates the relationship with The corresponding optimal frame rate This is the high-speed frame rate threshold, typically 30fps or 60fps. This is a low frame rate threshold, typically 10fps or 15fps. This is the high-speed judgment threshold, with a typical value of 60 km / h. It is the low-speed detection threshold, with a typical value of 20 km / h.

[0040] The above vehicle speed frame rate mapping relationship shows that when the real-time vehicle speed... Reaching the high-speed judgment threshold The optimal frame rate corresponding to this time is When the real-time vehicle speed Not exceeding the low-speed judgment threshold The optimal frame rate corresponding to this time is And when the real-time vehicle speed In and When the frame rate is between these values, the optimal frame rate is obtained using a linear interpolation formula. This calculation allows for a smooth transition in frame rate, avoiding visual discomfort for users caused by sudden changes in frame rate.

[0041] Based on the aforementioned vehicle speed-frame rate mapping relationship, the optimal frame rate corresponding to the target vehicle's current real-time speed can be calculated. If the difference between the vehicle camera's current frame rate and this optimal frame rate is significant, directly switching the current frame rate to the optimal frame rate will lead to a severe frame rate jump. To avoid this phenomenon, a frame rate transition control mechanism can be introduced to slowly and smoothly adjust the vehicle camera's frame rate parameter from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed. For example, to avoid frame rate jumps, the following smooth transition algorithm can be used:

[0042] in, This represents the target frame rate at time t. This indicates the frame rate at the previous moment. This represents the optimal frame rate calculated at time t using the aforementioned vehicle speed-frame rate mapping relationship. This is the smoothing coefficient, with a value ranging from [0, 1] and a typical value of 0.5-0.8. By introducing the above smoothing transition algorithm, the frame rate parameter of the vehicle camera can be smoothly transitioned from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed, thereby avoiding screen jumps and effectively improving the user's visual experience.

[0043] As an example, Figure 3 This is a schematic diagram illustrating a vehicle speed-frame rate mapping relationship provided in an embodiment of this application. Figure 3 In the context of vehicle speed, when the real-time vehicle speed does not exceed 20 km / h, it is considered a low-speed zone, where the optimal frame rate determined by the vehicle speed-frame rate mapping relationship is a low frame rate of 10 fps. When the real-time vehicle speed reaches 60 km / h, it is considered a high-speed zone, where the optimal frame rate determined by the vehicle speed-frame rate mapping relationship is a high frame rate of 30 fps. When the real-time vehicle speed is between 20 km / h and 60 km / h, it is considered a transition zone, where the optimal frame rate determined by the linear interpolation formula in the vehicle speed-frame rate mapping relationship is a medium frame rate between 10 fps and 30 fps.

[0044] It can be seen that when the vehicle speed is high, the vehicle is more dangerous. At this time, adjusting to a higher frame rate parameter can allow the driver to see a clearer camera image, thereby improving safety. When the vehicle speed is low, the vehicle is less dangerous. At this time, adjusting to a lower frame rate parameter can reduce system power consumption and improve the vehicle's range. Therefore, adaptive control and adjustment of the frame rate parameter based on real-time vehicle speed is achieved.

[0045] The above describes the relevant content regarding adjusting frame rate parameters based on real-time vehicle speed. The following describes the relevant content regarding adjusting frame rate parameters based on driver facial images.

[0046] In one implementation of this application, the target dimension perception data consists of multi-frame facial images of the driver of the target vehicle; the frame rate parameter is adjusted based on the target dimension perception data, including: (1) For each frame of facial image, determine the gaze state corresponding to the frame of facial image based on the driver's gaze direction in the frame of facial image; wherein, the gaze state includes gaze and no gaze, which are used to indicate whether the driver is gazing at the display area of ​​the vehicle camera; (2) If the proportion of frames of facial images with a gaze state in multiple frames of facial images exceeds the first threshold, then the display area of ​​the driver's gaze is determined; otherwise, the display area of ​​the driver's non-gaze state is determined. (3) If it is determined that the driver is looking at the display area, the frame rate parameter is adjusted to the first frame rate; if it is determined that the driver is not looking at the display area, the frame rate parameter is adjusted to the second frame rate, and the first frame rate is higher than the second frame rate.

[0047] If the driver's facial image of the target vehicle meets the frame rate adjustment conditions, and no other perceptual data of higher priority dimensions meets the frame rate adjustment conditions, the driver's facial image is used as the target dimension perceptual data. At this point, the operation process of adjusting the frame rate parameters based on the driver's facial image begins. The main principle of this operation process is to use image recognition technology to detect whether the driver is looking at the display area of ​​the vehicle's camera. Only when the driver is looking at the camera will the frame rate parameters be adjusted to a high frame rate to meet the need for a clear image. When the driver is not looking at the camera, the frame rate parameters can be adjusted to a low frame rate to reduce system power consumption. Specifically, multiple frames of the driver's face can be captured by a camera installed in the cockpit. For each frame, a deep learning-based facial detection algorithm can be used to locate the driver's facial position, and eye-tracking technology can be used to estimate the driver's gaze direction. Based on this gaze direction and the position information of the display area of ​​the vehicle camera, the gaze state corresponding to that frame can be determined. The gaze state includes "gazing" and "not gazing," which respectively indicate whether the driver is looking at the display area of ​​the vehicle camera. For example, if the driver is looking at the display area in facial image A, then the gaze state corresponding to facial image A is "gazing." If the driver is not looking at the display area in facial image B, then the gaze state corresponding to facial image B is "not gazing." If the percentage of frames with a "gazing" state in the acquired multiple facial images exceeds a first threshold (e.g., 50%), it can be considered that the driver is indeed looking at the display area. In this case, the frame rate parameter of the vehicle camera is adjusted to a higher first frame rate. Conversely, if the percentage of frames showing a gaze in the acquired multi-frame facial images does not exceed the first threshold, it can be assumed that the driver is not gazing at the display area. In this case, the frame rate parameter of the vehicle camera is adjusted to a lower second frame rate. .

[0048] In one implementation of this application embodiment, determining the gaze state corresponding to the facial image frame based on the driver's gaze direction in the facial image frame includes: (1) Perform eye region detection on the facial image frame to determine the coordinates of the pupil center point and the corner of the eye; (2) Determine the driver's line of sight vector based on the coordinates of the pupil center point and the corner of the eye; (3) Determine the gaze state corresponding to the facial image in the frame based on the geometric relationship between the gaze vector and the display area.

[0049] When determining the gaze state of a facial image based on the driver's gaze direction, the process first involves performing eye region detection on the image to determine the coordinates of the pupil center point and the corner of the eye. Then, the driver's gaze vector is determined based on the pupil center point and the corner of the eye coordinates. Finally, by analyzing the geometric relationship between this gaze vector and the display area of ​​the vehicle camera, it can be determined whether the driver is gazing at the display area in the facial image, thus determining the gaze state of the image.

[0050] For example, after determining the coordinates of the pupil's center point and the corner of the eye, the following formula can be used to calculate the direction of the gaze:

[0051] in: Indicates the angle of view. Indicates the coordinates of the center point of the pupil. This indicates the coordinates of the inner corner of the eye.

[0052] Assume the projection of the display area onto the driver's field of vision is a rectangular region R, and the driver's line-of-sight vector is... The distance from the end of the line of sight to the display area is Then when Furthermore, when the gaze vector is projected onto the rectangular region R, the gaze state corresponding to the image is determined to be gazed; otherwise, the gaze state corresponding to the image is determined to be non-gazed. The preset distance threshold is typically 0.5 meters.

[0053] To prevent misjudgment of the driver's gaze status, a time-cumulative judgment mechanism can be adopted. This involves statistically analyzing the gaze status of multiple frames of facial images within a set time window. If the gaze status of most facial images is gaze, it is determined that the driver is gazing at the screen display area; otherwise, it is determined that the driver is not gazing at the screen display area.

[0054] Considering that drivers typically only glance at the display area quickly, the time window can be set relatively short, assuming the algorithm detects a frame rate of... (30fps), then in the time window The number of tests within is The driver's gaze state at time t can be represented by the following formula:

[0055] Discretizing the formula yields:

[0056] in, This represents the driver's gaze state at time t. This indicates the detection results at a historical moment (1 for fixation, 0 for no fixation). This indicates the length of the time window, typically 100-150ms. This indicates the detection time interval, with a typical value of 33ms (i.e., 30fps). This represents the number of samples within the time window, i.e. , This is the threshold for judgment; a typical value is... This indicates that if a driver's gaze is detected in more than 50% of the frames within a time window, it is considered a gaze from the driver. It is the time ratio threshold, with a typical value of 0.5.

[0057] If calculated using the above formula = If the driver is looking at the display area, then the frame rate parameter of the vehicle camera is adjusted to a high frame rate. If calculated using the above formula = If this indicates that the driver is not looking at the display area, the frame rate parameter of the vehicle camera should be adjusted to a low frame rate. .

[0058] As an example, Figure 4 This is a schematic diagram illustrating an operation process for adjusting the frame rate parameters of an in-vehicle camera based on a driver's facial image, provided in an embodiment of this application. Figure 4 In this process, facial images of the driver are captured, and facial detection algorithms are used to detect facial position. 68 key points in the eye region are extracted to locate the pupil center and corner of the eye, thereby calculating the gaze angle and determining the driver's gaze vector. The gaze state for each frame of the facial image is determined by whether the gaze vector projects onto the display area. A time window accumulation mechanism is then used: if more than 50% of the frames show a gaze, the driver is considered to be looking at the display area, and the frame rate of the vehicle camera is adjusted to a high frame rate of 30fps; otherwise, the driver is considered not looking at the display area, and the frame rate of the vehicle camera is adjusted to a low frame rate of 10fps. Furthermore, a smooth transition algorithm can be introduced when adjusting the frame rate to avoid abrupt frame rate changes.

[0059] It can be seen that adjusting the frame rate parameter of the vehicle camera to a high frame rate when the driver is looking at the display area can meet the driver's need to see a clear picture, while adjusting the frame rate parameter to a low frame rate when the driver is not looking at the display area can reduce system power consumption and improve vehicle range. Therefore, adaptive control and adjustment of frame rate parameter based on driver's facial image is realized.

[0060] The above describes the relevant content on adjusting frame rate parameters based on driver facial images. The following describes the relevant content on adjusting frame rate parameters based on images of the vehicle exterior scene.

[0061] In one implementation of this application, the perception data for the target dimension consists of multi-frame images of the exterior scene of the target vehicle; the frame rate parameter is adjusted based on the perception data for the target dimension, including: (1) Based on multiple frames of images of the exterior of the vehicle, determine whether there is a moving target object rapidly approaching the target vehicle; (2) If it is determined that there is a moving target, the frame rate parameter is adjusted to the third frame rate.

[0062] If the exterior scene image of the target vehicle meets the frame rate adjustment conditions, and no other dimension of perception data with higher priority meets the frame rate adjustment conditions, the exterior scene image is used as the target dimension of perception data. At this point, the operation process of adjusting the frame rate parameters based on the exterior scene image begins. The main principle of this operation process is to detect rapidly approaching moving objects in the exterior scene image using computer vision technology. When such moving objects are detected, the frame rate parameters of the onboard camera are increased to clearly capture potentially dangerous scenes. Specifically, multiple frames of exterior scene images can be acquired from the rearview camera of the target vehicle. Background subtraction or optical flow methods are used to detect moving objects in these exterior scene images. For example, the basic formula for background subtraction is:

[0063] in, This represents the current frame image of the scene outside the vehicle at time t. This represents the background model at time t. express The difference image, when When the pixel is identified as a foreground pixel, To set a threshold.

[0064] After detecting each moving object using background subtraction or optical flow methods, each moving object is tracked individually, and its displacement and velocity are calculated between consecutive frames. This allows determination of whether each moving object is a target object rapidly approaching the target vehicle. If at least one target object is found rapidly approaching the target vehicle, it indicates a potential hazard. In this case, the smooth transition algorithm is bypassed, and the frame rate parameter of the vehicle camera is directly adjusted to a higher third frame rate. The typical frame rate is 60fps, maintained for a certain time window (e.g., 3-5 seconds) to capture clear images of potential hazards. If there is no rapidly approaching moving object, there is no potential hazard, and there is no need to adjust the frame rate parameter of the vehicle camera. For example, the frame rate parameter can be kept at a lower original value to reduce system power consumption.

[0065] In one implementation of this application, determining whether a target moving object is rapidly approaching the target vehicle based on multiple frames of exterior scene images includes: (1) Based on multiple frames of exterior scene images, detect and track all moving objects in the exterior scene of the target vehicle that are moving toward the target vehicle; (2) For each of all moving objects, if the moving speed of the moving object exceeds the second threshold, or if the size growth rate of the image of the moving object in the multi-frame exterior scene image exceeds the third threshold, then the moving object is identified as the target moving object.

[0066] When identifying moving objects rapidly approaching a target vehicle, the system first detects and tracks all moving objects moving towards the target vehicle from the vehicle's exterior scene based on multiple frames of exterior scene images. It then calculates the displacement and velocity of each moving object between consecutive frames. The velocity calculation formula is as follows:

[0067] in, This represents the velocity (pixels per second) of the moving object at time t. This represents the coordinates of the center point of the moving object at time t. This represents the coordinates of the center point of the moving object at the previous moment. Indicates a time interval.

[0068] For a given moving object, its speed or the size increase rate of its image across multiple frames of the vehicle's exterior scene can be used to determine if it is a target object rapidly approaching the vehicle. For example, the formula for calculating the size increase rate is as follows:

[0069] in, Indicates the size growth rate. Let represent the area of ​​the image of the moving object at time t. Indicates the number of frames in the time interval.

[0070] If the speed of a moving object (The velocity threshold, i.e., the second threshold mentioned above), or the growth rate of the size of the moving object. (If the size growth rate threshold, i.e. the third threshold mentioned above, is usually a value between 0.05 and 0.1), then the moving object is determined to be a target moving object that is rapidly approaching the target vehicle; otherwise, the moving object is determined not to be a target moving object.

[0071] As an example, Figure 5 This is a schematic diagram illustrating an operation process provided in this application embodiment for adjusting the frame rate parameters of an onboard camera by detecting a rapidly approaching target. Figure 5 In the process, images of the external scene from the electronic rearview mirror are acquired, and moving objects in the images are detected using the background subtraction method. If a moving object is detected, a target tracking algorithm is used to track each moving object. The moving speed and size growth rate of each moving object are calculated between consecutive frames. If the moving speed or size growth rate exceeds a set threshold, the corresponding moving object is determined to be a target moving object that is rapidly approaching the vehicle. At this time, the frame rate parameter of the vehicle's onboard camera is directly set to a high frame rate of 60fps and maintained for 3-5 seconds.

[0072] It can be seen that when there is a rapidly approaching target, adjusting the frame rate parameter of the vehicle camera to a high frame rate can capture clear images of potential dangers. When there is no rapidly approaching target, keeping the frame rate parameter of the vehicle camera unchanged can reduce system power consumption and improve vehicle range. Therefore, adaptive control and adjustment of frame rate parameter based on images of the external scene is achieved.

[0073] The above describes the relevant content on adjusting frame rate parameters based on images of the vehicle's external scene. Finally, the relevant content on adjusting frame rate parameters based on sensor data is described.

[0074] In one implementation of this application, the perception data for the target dimension is sensor data of the target vehicle; adjusting the frame rate parameter based on the perception data for the target dimension includes: (1) Determine whether the target vehicle has experienced an emergency based on sensor data; (2) If it is determined that an emergency has occurred to the target vehicle, the frame rate parameter is adjusted to the fourth frame rate.

[0075] If the sensor data of the target vehicle meets the frame rate adjustment conditions, and no other dimension of perception data with higher priority meets the frame rate adjustment conditions, the sensor data is used as the target dimension's perception data. At this point, the operation process of adjusting the frame rate parameters based on the sensor data begins. The main principle of this operation process is to detect whether an emergency event (such as sudden braking or a collision) has occurred in the vehicle using various sensors (such as acceleration sensors, airbag sensors, or brake sensors). If an emergency event occurs, the frame rate parameter of the onboard camera is immediately adjusted to the highest fourth frame rate to capture clear footage of the emergency event. Specifically, sudden braking or collision events can be detected using the target vehicle's acceleration sensor. For example, the instantaneous acceleration of the target vehicle can be calculated using the following formula:

[0076] in, Indicates instantaneous acceleration. This represents the vehicle speed at time t. For time intervals. When (Braking acceleration threshold, e.g., 2 m / s²) and At that time, it can be determined that the target vehicle has experienced a sudden braking event.

[0077] When performing collision event detection, the magnitude of the acceleration vector can be calculated based on triaxial accelerometer data using the following formula:

[0078] in, This represents the magnitude of the acceleration vector at time t. , These represent the acceleration values ​​in the three directions at time t. When the collision threshold (e.g., 10 m / s²) is reached, it can be determined that a collision event has occurred with the target vehicle.

[0079] Alternatively, airbag sensors can be used to detect collision events; for example, when an airbag deploys, it can be determined that a collision has occurred with the target vehicle.

[0080] When an emergency event is detected in the target vehicle, an emergency scenario is entered. At this point, the smooth transition algorithm is bypassed, and the frame rate parameter of the vehicle camera is directly adjusted to the highest level (e.g., 60fps or higher). This allows for the capture and recording of high-definition footage of the emergency event, and the recorded video data can be marked as important data to prevent overwriting. If it is determined that no emergency event has occurred in the target vehicle, there is no need to adjust the frame rate parameter of the vehicle camera; for example, the frame rate parameter can be kept at a lower original value to reduce system power consumption.

[0081] As an example, Figure 6 This is a schematic diagram illustrating an operation process for adjusting the frame rate parameters of an onboard camera based on sensor data, provided in an embodiment of this application. Figure 6 In the process, the system acquires sensor data from the vehicle and determines whether the vehicle has experienced an emergency event such as sudden braking or collision based on the sensor data. If an emergency event is determined, the system directly sets the frame rate parameter of the vehicle's onboard camera to a high frame rate of 60fps, records and saves the emergency event video data; otherwise, the original frame rate parameter remains unchanged.

[0082] It can be seen that adjusting the frame rate parameter of the vehicle camera to a high frame rate when an emergency event is detected can capture and record clear images of the emergency event. When no emergency event occurs, keeping the frame rate parameter of the vehicle camera unchanged can reduce system power consumption and improve vehicle range. Therefore, adaptive control and adjustment of frame rate parameter based on sensor data is achieved.

[0083] The above describes the frame rate parameter adjustment strategies for four different scene types, which can be summarized in Table 1 below.

[0084] Table 1

[0085] When the four scenario types shown in Table 1 simultaneously trigger frame rate parameter adjustments, emergency scenarios can be prioritized as the highest priority (1), rapid approach to a target as the second highest priority (2), driver gaze as the next highest priority (3), and speed changes as the lowest priority (4). Alternatively, the following priority arbitration rule can be introduced:

[0086] in, This indicates the frame rate that needs to be set in emergency scenarios. This indicates the frame rate that needs to be set to quickly approach the target scene. This indicates the frame rate that needs to be set for scenes where the driver is focused on. This indicates the frame rate that needs to be set for scenarios involving changes in vehicle speed. This indicates the final frame rate setting. It can be seen that this arbitration rule, by setting the frame rate parameter to the highest among all scene types, can cover the actual needs of all scene types.

[0087] In addition, the embodiments of this application can also set the following security protection mechanisms: (1) control the frame rate parameter of the vehicle camera to never be lower than the lower limit frame rate (e.g., 10fps) at any time to ensure that the basic visual information of the camera image is available; (2) when the vehicle's sensors malfunction and the corresponding perception data is unavailable, the frame rate parameter of the vehicle camera can be automatically restored to the safe default frame rate, such as 30fps; (3) the vehicle records all frame rate adjustment events for subsequent analysis. By setting these security protection mechanisms, the completeness and security of frame rate parameter adjustment can be further improved.

[0088] In the technical solution of this application embodiment, firstly, multiple dimensions of perception data of the target vehicle are acquired, with different dimensions of perception data corresponding to different priorities. Then, based on the multiple dimensions of perception data and their respective priorities, the frame rate parameters of the vehicle's onboard camera are adjusted. This process intelligently perceives multiple dimensions of perception data and pre-assigns different priorities to different dimensions, enabling reasonable adjustment of the onboard camera's frame rate parameters according to the current perception data and their corresponding priorities. For example, assuming the multiple dimensions of perception data include emergency scenarios, driver behavior, and vehicle speed, with emergency scenarios having the highest priority, when the vehicle is not in an emergency scenario, the frame rate parameters of the onboard camera can be adjusted to a moderate value based on driver behavior or vehicle speed. However, when the vehicle enters an emergency scenario, driver behavior and vehicle speed are disregarded, and the frame rate parameters of the onboard camera are directly adjusted to the highest value. As can be seen from the above examples, the technical solution of this application embodiment can reasonably adjust the frame rate parameter according to the current actual situation of the vehicle. In this way, the frame rate parameter will not be fixed at a high value, thus avoiding continuous high power consumption operation of various vehicle components, thereby saving vehicle battery power consumption and improving vehicle range.

[0089] In summary, this application proposes an intelligent frame rate control method based on the fusion of multi-dimensional perception information such as vehicle speed, driver's line of sight, rear target movement, and emergency scenarios. By integrating multi-dimensional perception information, it can make intelligent decisions and dynamically adjust the frame rate parameters of the vehicle camera, thereby achieving the best balance between energy saving and safety.

[0090] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] The above mainly describes a method for adjusting the frame rate parameters of an in-vehicle camera. The following will describe a device for adjusting the frame rate parameters of an in-vehicle camera.

[0092] Please see Figure 7 One embodiment of a vehicle-mounted camera frame rate parameter adjustment device in this application includes: The perception data acquisition module 701 is used to acquire perception data of the target vehicle in multiple dimensions; among which, the perception data of different dimensions correspond to different priorities. The frame rate parameter adjustment module 702 is used to adjust the frame rate parameter of the vehicle camera of the target vehicle based on multiple dimensions of perception data and the priority of each dimension of perception data.

[0093] In one implementation of this application, the frame rate parameter adjustment module includes: The target dimension determination unit is used to determine the highest priority target dimension perception data from the perception data of at least one dimension if the perception data of at least one dimension satisfies the preset frame rate adjustment condition. The frame rate parameter adjustment unit is used to adjust the frame rate parameter based on the perceptual data of the target dimension.

[0094] In one implementation of this application, the perception data for the target dimension is the real-time speed of the target vehicle; the frame rate parameter adjustment unit includes: The optimal frame rate determination subunit is used to determine the optimal frame rate corresponding to the real-time vehicle speed according to the preset vehicle speed frame rate mapping relationship; wherein, the vehicle speed frame rate mapping relationship records the optimal frame rate corresponding to different vehicle speeds respectively; The first frame rate parameter adjustment subunit is used to control the frame rate parameter to be smoothly adjusted from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed.

[0095] In one implementation of this application, the perceived data for the target dimension is a multi-frame facial image of the driver of the target vehicle; the frame rate parameter adjustment unit includes: The image gaze state determination subunit is used to determine the gaze state corresponding to each frame of facial image based on the driver's gaze direction in that frame of facial image; wherein, the gaze state includes gaze and no gaze, which are used to indicate whether the driver is looking at the display area of ​​the vehicle camera; The driver gaze state determination subunit is used to determine the driver gaze display area if the proportion of frames with gaze state in multiple frames of facial images exceeds a first threshold; otherwise, it determines the driver non-gaze display area. The second frame rate parameter adjustment subunit is used to adjust the frame rate parameter to the first frame rate if it is determined that the driver is looking at the display area of ​​the screen; and to adjust the frame rate parameter to the second frame rate if it is determined that the driver is not looking at the display area of ​​the screen, wherein the first frame rate is higher than the second frame rate.

[0096] In one implementation of this application, the image gaze state determination subunit includes: The eye region detection subunit is used to perform eye region detection on the facial image frame and determine the coordinates of the pupil center point and the corner of the eye. The gaze vector determination subunit is used to determine the driver's gaze vector based on the coordinates of the pupil center point and the corner of the eye. The gaze state generation subunit is used to determine the gaze state corresponding to the facial image in the frame based on the geometric relationship between the gaze vector and the display area.

[0097] In one implementation of this application, the perception data for the target dimension is a multi-frame image of the exterior scene of the target vehicle; the frame rate parameter adjustment unit includes: The fast approach target determination subunit is used to determine whether there is a moving target object rapidly approaching the target vehicle based on multiple frames of exterior scene images; The third frame rate parameter adjustment subunit is used to adjust the frame rate parameter to the third frame rate if it is determined that there is a moving target object.

[0098] In one implementation of this application embodiment, the rapid approach target determination subunit includes: The moving object tracking subunit is used to detect and track all moving objects moving toward the target vehicle in the external scene of the target vehicle based on multiple frames of external scene images. The moving object determination subunit is used to determine each moving object as a target moving object if its moving speed exceeds a second threshold or the size growth rate of its image in multiple frames of vehicle exterior scene images exceeds a third threshold.

[0099] In one implementation of this application, the perception data for the target dimension is sensor data of the target vehicle; the frame rate parameter adjustment unit includes: The emergency event determination subunit is used to determine whether an emergency event has occurred on the target vehicle based on sensor data. The fourth frame rate parameter adjustment subunit is used to adjust the frame rate parameter to the fourth frame rate if it is determined that an emergency has occurred in the target vehicle.

[0100] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle camera frame rate parameter adjustment method as described in any of the above embodiments.

[0101] This application also provides a computer program product that, when run on an in-vehicle controller, causes the in-vehicle controller to execute the in-vehicle camera frame rate parameter adjustment method as described in any of the above embodiments.

[0102] Figure 8 This is a schematic diagram of an embodiment of the vehicle controller provided in this application. Figure 8 As shown, the vehicle controller 8 in this embodiment includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the embodiments of the various vehicle camera frame rate parameter adjustment methods described above, for example... Figure 1 Steps 101 to 102 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of modules 701 to 702 are shown.

[0103] The computer program 82 can be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 82 in the vehicle controller 8.

[0104] The processor 80 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0105] The memory 81 can be an internal storage unit of the vehicle controller 8, such as a hard drive or memory of the vehicle controller 8. The memory 81 can also be an external storage device of the vehicle controller 8, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the vehicle controller 8. Furthermore, the memory 81 can include both internal storage units and external storage devices of the vehicle controller 8. The memory 81 is used to store the computer program and other programs and data required by the vehicle controller. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0108] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0114] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for adjusting a frame rate parameter of a car-mounted camera, characterized in that, include: Acquire perception data of the target vehicle from multiple dimensions; among which, the perception data of different dimensions correspond to different priorities; Based on the perception data from the multiple dimensions and the priority corresponding to each of the multiple dimensions of perception data, the frame rate parameters of the vehicle-mounted camera of the target vehicle are adjusted.

2. The method of claim 1, wherein, The step of adjusting the frame rate parameters of the vehicle-mounted camera of the target vehicle based on the multiple dimensions of perception data and the priorities corresponding to each of the multiple dimensions of perception data includes: If at least one dimension of the perception data among the multiple dimensions satisfies the preset frame rate adjustment condition, then the perception data of the highest priority target dimension is determined from the perception data of the at least one dimension. The frame rate parameter is adjusted based on the perceived data of the target dimension.

3. The method of claim 2, wherein, The perception data for the target dimension is the real-time speed of the target vehicle; adjusting the frame rate parameter based on the perception data for the target dimension includes: Based on a preset vehicle speed-frame rate mapping relationship, the optimal frame rate corresponding to the real-time vehicle speed is determined; wherein, the vehicle speed-frame rate mapping relationship records the optimal frame rate corresponding to different vehicle speeds respectively; The frame rate parameter is smoothly adjusted from the current frame rate to the optimal frame rate corresponding to the real-time vehicle speed.

4. The method of claim 2, wherein, The target dimension perception data consists of multi-frame facial images of the driver of the target vehicle; adjusting the frame rate parameter based on the target dimension perception data includes: For each frame of the facial image, the gaze state corresponding to that frame of the facial image is determined based on the driver's gaze direction in that frame of the facial image; wherein, the gaze state includes gaze and no gaze, which are used to indicate whether the driver is gazing at the display area of ​​the vehicle camera; If the percentage of frames in the multi-frame facial images that correspond to a gazed state exceeds a first threshold, then it is determined that the driver is gazing at the screen display area; otherwise, it is determined that the driver is not gazing at the screen display area. If it is determined that the driver is looking at the screen display area, the frame rate parameter is adjusted to a first frame rate; if it is determined that the driver is not looking at the screen display area, the frame rate parameter is adjusted to a second frame rate, wherein the first frame rate is higher than the second frame rate.

5. The method of claim 4, wherein, Determining the gaze state corresponding to the facial image frame based on the driver's gaze direction in the facial image frame includes: Perform eye region detection on the facial image frame to determine the coordinates of the pupil center point and the corner of the eye; The driver's gaze vector is determined based on the coordinates of the pupil center point and the coordinates of the corner of the eye. The gaze state corresponding to the facial image in that frame is determined based on the geometric relationship between the gaze vector and the display area.

6. The method of claim 2, wherein, The perception data for the target dimension consists of multi-frame images of the exterior scene of the target vehicle; adjusting the frame rate parameter based on the perception data for the target dimension includes: Based on the multi-frame images of the vehicle exterior, determine whether there is a moving target object rapidly approaching the target vehicle; If the target moving object is determined to exist, the frame rate parameter is adjusted to the third frame rate.

7. The method of claim 6, wherein, The step of determining whether there is a target moving object rapidly approaching the target vehicle based on the multi-frame exterior scene images includes: Based on the multi-frame exterior scene images, detect and track all moving objects in the exterior scene of the target vehicle that are moving toward the target vehicle; For each of the moving objects, if the moving speed of the moving object exceeds a second threshold, or if the size growth rate of the image of the moving object in the multi-frame exterior scene image exceeds a third threshold, then the moving object is identified as the target moving object.

8. The method of claim 2, wherein, The perception data for the target dimension is the sensor data of the target vehicle; adjusting the frame rate parameter based on the perception data for the target dimension includes: Based on the sensor data, determine whether the target vehicle has experienced an emergency. If an emergency is determined to have occurred to the target vehicle, the frame rate parameter is adjusted to the fourth frame rate.

9. An in-vehicle controller comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the vehicle camera frame rate parameter adjustment method as described in any one of claims 1 to 8.

10. A computer program product, characterised in that, When the computer program product is run on the vehicle controller, it causes the vehicle controller to perform the vehicle camera frame rate parameter adjustment method as described in any one of claims 1 to 8.

Images