Camera frame rate switching method for vehicle and related equipment

Abstract

The invention discloses a camera frame rate switching method for a vehicle and related equipment, and the method comprises the steps: collecting the information of the vehicle; deciding whether to switch the frame rate of the preset camera based on the collected information; if switching is needed, bottom layer camera hardware is linked in real time, the frame rate of the preset camera is switched, and the bottom layer camera hardware comprises a deserializer capable of outputting a plurality of trigger sources at the same time, a serializer used for receiving the needed trigger sources and an image sensor used for dynamically modifying the output frame rate.

Description

Technical Field

[0001] This application relates to the field of driver assistance technology, and in particular to a camera frame rate switching method and related equipment for vehicles. Background Technology

[0002] In related technologies, when implementing Level 2 assisted driving functions on embedded platforms with low computing power and limited DDR (Double Data Rate) memory resources, there are significant differences in the frame rate requirements of surround-view camera systems for driving (such as cruise control and following) and parking (such as automatic parking and remote parking). Existing solutions typically use a fixed frame rate configuration, but this approach struggles to address the following conflicting needs: high-speed driving scenarios (such as Level 2 cruise control and urban navigation) require minimizing resource consumption (DDR bandwidth and computational load), while low-speed and function-activated scenarios (such as automatic parking, reversing, manual activation of 360° imaging, and parking space search) require a high frame rate to ensure real-time perception accuracy.

[0003] In autonomous driving platforms with low computing power and scarce DDR resources, traditional solutions suffer from the pain points of continuous resource waste and unavailability of dynamic switching. For example, forcibly maintaining a high frame rate of 30fps or 25fps in driving scenarios unnecessarily occupies excess DDR bandwidth; and traditional frame rate switching requires module hardware reset, which takes more than 1 second (specifically including camera power-on / off + SerDes (Serializer-Deserializer) reconfiguration + Sensor reconfiguration, etc.). Due to the long period of no image output from the camera, it will cause lag in response to sudden obstacles in scenarios such as parking, which poses a significant risk. Summary of the Invention

[0004] This application provides a camera frame rate switching method, electronic device, and storage medium for vehicles, which at least solve one of the above-mentioned technical problems.

[0005] In a first aspect, embodiments of this application provide a camera frame rate switching method for a vehicle, comprising: collecting vehicle information; deciding whether to switch the frame rate of a preset camera based on the collected information; if the decision requires switching, real-time linkage with underlying camera hardware to switch the frame rate of the preset camera, wherein the underlying camera hardware includes a deserializer capable of simultaneously outputting multiple trigger sources, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate.

[0006] Secondly, embodiments of this application provide an electronic device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described camera frame rate switching methods for vehicles of this application.

[0007] Thirdly, embodiments of this application provide a storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-mentioned camera frame rate switching methods for vehicles in this application.

[0008] Fourthly, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described camera frame rate switching methods for vehicles.

[0009] Fifthly, embodiments of this application also provide a mobile platform, including a vehicle body and an electronic device, the electronic device including: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method described in the first aspect.

[0010] The method of this application, based on user operation and the vehicle's perception of the current environment and state, links with the underlying camera hardware driver in real time, thereby enabling dynamic adjustment of the camera frame rate. This allows the vehicle to significantly reduce the original image DDR bandwidth usage, system load usage, and algorithm processing DDR bandwidth usage in driving scenarios. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A comparative diagram showing the pain points of related technologies and the breakthroughs of this application; Figure 2 A flowchart illustrating a camera frame rate switching method for a vehicle, provided as an embodiment of this application; Figure 3 The overall system architecture diagram is shown as a specific example of a camera frame rate switching method for a vehicle provided in an embodiment of this application. Figure 4 A flowchart illustrating a common sensor dynamic frame rate modification process, which is a specific example of a camera frame rate switching method for a vehicle provided in an embodiment of this application. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Please refer to Figure 1 It shows a schematic diagram comparing the relevant technical pain points with the breakthroughs of this application.

[0015] like Figure 1 As shown, the inventors believe that the relevant technical solutions have at least the following drawbacks: First, there is the resource waste inherent in fixed frame rate architectures. Because the industry commonly uses fixed frame rate solutions (such as a uniform 30fps for parking / driving), this inevitably leads to resource overload and system frequency throttling risks in high-speed driving scenarios on low-computing-power platforms. For example, during cruise control, the surround-view camera continuously runs at a high frame rate, unnecessarily occupying over 40% of DDR bandwidth; the persistently high computational load triggers platform temperature control and frequency throttling, affecting the real-time performance of the perception algorithm.

[0016] Secondly, the real-time performance of the second-level initialization switching fails (assuming an attempt is made to use a common camera initialization process for dynamic switching). If frame rate switching is implemented based on the existing camera control framework, the entire switching process will take approximately 1 second due to reliance on the complete camera hardware initialization process, creating a fundamental bottleneck.

[0017] Finally, the functionality is not feasible. A 1-second delay far exceeds the tolerance threshold of the scenario. For example, in a parking scenario, a 1-second delay can reduce the obstacle response distance by about 3 meters (at a vehicle speed of 5km / h); in a driving scenario, it cannot support the L2+ system to switch to a high frame rate in the shortest possible time to deal with emergencies.

[0018] In related technologies, configuring a camera requires a series of steps, including powering down and re-energizing the entire module, configuring the vehicle's electronic control unit's deserializer, resetting the sensor, configuring the module's serializer, and configuring the sensor again, taking anywhere from hundreds of milliseconds to several seconds. This application's solution minimizes modifications. Once the camera has generated an image, it avoids the aforementioned complete re-initialization process. Only necessary registers (including 1-2 registers for the serializer's mapping trigger signal and 2-3 synchronization frame rate-related registers on the sensor side) need to be modified, reducing the effective time to single-digit register configuration latency, thus keeping it within 10ms.

[0019] To achieve optimal performance under resource constraints, this application proposes a scene-granular dynamic camera frame rate switching mechanism. By integrating user operation signals (gear / function buttons), autonomous driving status (vehicle speed, function module activation flags) and environmental perception results, the mechanism links the camera hardware layer (SerDes, Sensor driver) in real time to complete seamless frame rate switching within 10ms.

[0020] To optimize system resource utilization (especially DDR bandwidth and computational load), this application proposes a dynamic camera frame rate switching scheme. This scheme, based on user operations (such as activating the parking function) and the autonomous driving system's perception of the current environment and state (such as whether the vehicle is driving or parked), dynamically adjusts the frame rate of the surround-view camera in real time, in conjunction with the underlying camera hardware drivers (including SerDes, Sensor, etc.). SerDes is a high-speed serial communication technology used to convert parallel data into serial data (serializer) for transmission and to restore the serial data to parallel data (deserializer) at the receiving end. It is widely used in inter-chip communication, fiber optic networks, video transmission, and other fields.

[0021] For example, in low-frame-rate scenarios such as driving: when the system determines that it is currently in a driving or other scenario where high-frame-rate surround view is not required, it can dynamically reduce the surround view frame rate to a preset low frame rate. This switching process can be controlled within 10ms, with no perceptible interference to system operation and diagnosis, and can significantly reduce DDR bandwidth usage and system computing load.

[0022] For high-frame-rate scenarios such as parking: When the system determines that it is in a scenario requiring high-frame-rate surround view, such as parking, it can dynamically increase the surround view frame rate to a preset high frame rate. This switching process takes less than 10ms and has no perceptible interference with system operation and diagnostics, effectively meeting the real-time and accuracy performance requirements of the parking system.

[0023] Please refer to Figure 2This document illustrates a flowchart of a camera frame rate switching method for a vehicle according to an embodiment of this application. The executing entity of this embodiment can be an electronic device or a camera frame rate switching device for a vehicle disposed within an electronic device. The camera frame rate switching device for a vehicle can be implemented in software or a combination of software and hardware. The camera frame rate switching device for a vehicle can be a processor within an electronic device. For example, the electronic device can be an electronic control unit (ECU) on a vehicle, or other devices connected to the ECU. The method of this embodiment is used for camera frame rate switching in a vehicle.

[0024] like Figure 2 As shown, in step 201, vehicle information is collected; In step 202, a decision is made based on the collected information as to whether the frame rate of the preset camera needs to be switched. In step 203, if a decision requires switching, the underlying camera hardware is linked in real time to switch the frame rate of the preset camera. The underlying camera hardware includes a deserializer capable of simultaneously outputting multiple trigger sources, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate.

[0025] In this embodiment, in step 201, the camera frame rate switching device for the vehicle can collect vehicle information such as user operation signals, vehicle status, and / or environmental perception information. Furthermore, the user operation information may include reverse gear, parking button, 360-degree image button, etc.; the vehicle status may include whether the vehicle speed is less than a threshold, parking space recognition signs, etc.; and the environmental perception information may include obstacle distance, parking space coordinates, etc.

[0026] Then, for step 202, the camera frame rate switching device for the vehicle decides whether to switch the frame rate of the preset camera based on the collected information. For example, when the reverse gear is activated (i.e., the gear is switched to reverse), the automatic parking function is actively or passively triggered (i.e., the parking function is activated), the vehicle speed is less than a preset threshold (i.e., the vehicle is in motion and the speed is lower than a preset threshold), or the user manually turns on the 360-degree camera (i.e., the user is viewing the 360-degree camera), the frame rate of the preset camera needs to be switched to a high frame rate; when the vehicle speed is greater than the preset threshold and there is no need for surround view, the frame rate of the preset camera needs to be switched to a low frame rate.

[0027] Finally, for step 203, if a switch is required, the underlying camera hardware is activated in real time to switch the frame rate of the preset camera. The underlying camera hardware includes a deserializer capable of simultaneously outputting multiple trigger sources, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate. In a specific example, if the decision requires switching to a higher frequency, the serializer selects to receive the high-frequency deserialized signal and dynamically modifies the image sensor's output frame rate based on the trigger signal corresponding to the decision. If the decision requires switching to a lower frequency, the serializer selects to receive the low-frequency deserialized signal. The deserializer includes both high-frequency and low-frequency deserialization functions; in practice, this can be implemented using one or more deserializers. For a vehicle's surround-view system, one deserializer and multiple serializers (e.g., 1 deserializer and 4 serializers) can be configured. One deserializer can receive multiple trigger sources, which are essentially level signals connected to different GPIO pins on the deserializer.

[0028] The method in this embodiment switches the frame rate of the preset camera in real time by linking the underlying camera hardware based on the collected vehicle information, thereby realizing dynamic adjustment of the camera frame rate. This allows the system to significantly reduce the original image DDR bandwidth usage, system load usage, and algorithm processing DDR bandwidth usage in driving scenarios.

[0029] In some optional embodiments, the real-time linkage with the underlying camera hardware to switch the frame rate of the preset camera includes: the camera frame rate switching device for the vehicle responds to a frame rate switching command by modifying the trigger source mapped by the serializer. For example, the deserializer includes high-frequency deserialization and low-frequency deserialization functions. If a switch to a high frequency is required, the serializer selects to receive the high-frequency deserialization signal. If a switch to a low frequency is required, the serializer selects to receive the low-frequency deserialization signal.

[0030] Then, the camera frame rate switching device for the vehicle modifies the registers in the image sensor related to the synchronization frame rate to switch the preset camera frame rate. Taking the ISX031 fisheye camera as an example, VMAX (VerticalMaximum) and VMAX_OFFSET can be modified simultaneously. VMAX is the maximum vertical exposure time (unit: lines), used to control the sensor's frame rate. Adjusting VMAX changes the sensor's frame rate. It defines the minimum time interval between the start of one frame and the start of the next. If VMAX is set too small, it may result in insufficient exposure time, affecting image quality; if it is too large, it will reduce the frame rate. VMAX_OFFSET is the offset of VMAX, used to fine-tune the timing of the vertical synchronization signal. In some cases (such as multi-camera synchronization or special trigger modes), it is necessary to adjust VMAX_OFFSET to ensure timing alignment. VMAX_OFFSET is usually used in conjunction with VMAX to optimize the sensor's exposure and readout timing.

[0031] The method in this embodiment switches the frame rate of a preset camera by modifying the trigger source mapped by the serializer and the register in the image sensor used for synchronizing the frame rate, thereby enabling dynamic adjustment of the frame rate of the surround-view camera.

[0032] In some optional embodiments, modifying the registers in the image sensor used for synchronizing frame rate includes: the camera frame rate switching device for the vehicle detecting the sensor type of the image sensor; and then the camera frame rate switching device for the vehicle calling the corresponding registers for configuration modification based on the sensor type. In further optional embodiments, the sensor type may include ISX031, OVX3J, and AR0233. AR0233 (CMOS image sensor) is a 1 / 2.6-inch, 2.3-megapixel (1920×1200) global shutter CMOS sensor, widely used in industrial vision, drones, and autonomous driving (such as surround view systems). OVX3J (OmniVision image sensor) is a high-performance global shutter CMOS image sensor, mainly used in industrial vision, robot navigation, AR / VR, and high-speed motion capture.

[0033] In a specific example, if the sensor type is an ISX031 fisheye camera, both VMAX and VMAX_OFFSET can be modified for configuration changes. Furthermore, the same modification method can be used for the 3M fisheye camera ISX019, and the 8M front-view / side-view cameras IMX728 and IMX424. If the sensor type is OVX3J, the VTS (Vertical Total Size) related registers can be modified for configuration changes. Similarly, the same modification method can be used for the 3M fisheye camera OX03D4C and the 8M front-view / side-view camera OX08D10. VTS is the total number of vertical rows, including effective pixel rows and blank rows (such as vertical blanking areas). VTS can affect the sensor's frame rate and exposure time; adjusting VTS can optimize sensor timing, especially in high frame rate or long exposure applications. Typically, VTS = VMAX + number of vertical blanking rows. VMAX must be ≤ VTS, otherwise timing errors will occur. If the sensor type is AR0233, the frame length register can be called to modify it. It should be noted that, in addition to the sensors mentioned in the examples above, other types of sensors can also be used in practice.

[0034] The method in this embodiment achieves frame rate switching by modifying the configuration of the corresponding register based on the sensor type. Sensor detection is essentially instantaneous, reflecting a software platform strategy and supporting multiple camera types.

[0035] In some optional embodiments, before the real-time linkage with the underlying camera hardware, the method further includes setting the image sensor to an external trigger mode. In a specific example, the external trigger mode can be used regardless of whether switching to a low frame rate or a high frame rate. It should be noted that the TRIG_MODE register does not need to be modified during dynamic frequency reduction; the main modifications are made to the sensor's frame rate-related registers. TRIG_MODE is the camera operating mode of each sensor manufacturer, which is generally subdivided into active image capture and triggered image capture. Active image capture is characterized by the sensor continuously acquiring images at a fixed frame rate (FPS) without requiring an external trigger signal. Triggered image capture is characterized by the sensor requiring an external signal (hardware or software trigger) to acquire single or multiple frames of images.

[0036] In some optional embodiments, the vehicle information includes at least one of the following: vehicle status information, environmental perception information, and user operation signals. This allows a decision to switch camera frequencies based on the aforementioned vehicle information.

[0037] In some optional embodiments, the decision on whether to switch the frame rate of the preset camera based on the collected information includes: determining whether the current scenario requires high frame rate surround view based on vehicle status information, environmental perception information, and / or user operation signals; if the current scenario requires high frame rate surround view, such as when reverse gear is activated, automatic parking is triggered, vehicle speed is less than a preset threshold, or the user manually activates 360-degree imaging, the frame rate of the preset camera can be switched to a high frame rate; if the current scenario does not require high frame rate surround view, such as when vehicle speed is greater than a preset threshold and there is no surround view requirement, the frame rate of the preset camera can be switched to a low frame rate.

[0038] The method in this embodiment can determine whether the current scene requires high frame rate surround view based on vehicle status information, environmental perception information and / or user operation signals, thereby enabling dynamic adjustment of the surround view camera frame rate. This allows the system to significantly reduce the original image DDR bandwidth usage, system load usage, and algorithm processing DDR bandwidth usage in scenes where high frame rate surround view is not required.

[0039] In some optional embodiments, the scenarios requiring high frame rate surround view include activating parking functions, viewing 360-degree images, and other scenarios that trigger the surround view monitoring system. Specifically, the frame rate increase is triggered when parking, when the user actively requests to view 360-degree images on the vehicle's infotainment system, when driving at low speeds (e.g., 30 km / s), or when nearby vehicles are close together. The AVM (Around View Monitor) is a driver assistance function that uses multi-camera fusion and image processing technology to display a 360° bird's-eye view in real time on the vehicle's central control screen. It helps drivers eliminate blind spots, especially improving safety during low-speed parking, driving on narrow roads, or in complex road conditions.

[0040] Further reference Figure 3 The diagram illustrates the overall system architecture of a specific example of a camera frame rate switching method for a vehicle provided in an embodiment of this application.

[0041] like Figure 3 As shown, this application addresses the conflicting frame rate requirements in driving and parking scenarios for autonomous driving platforms with low computing power and limited DDR resources. By employing a technology that dynamically switches camera frame rate output without requiring a reset, seamless frame rate switching within 10ms is achieved. This allows the system to automatically reduce the frame rate in high-speed driving scenarios, saving system DDR bandwidth; the bandwidth of raw image data is reduced by 66.7% (675.6MB / s → 225.2MB / s); and the algorithm processing bandwidth is reduced by 66.7% (8,445.0MB / s → 2,815.0MB / s). Based on actual test data from a certain platform, the resolution of the surround view image is 1920×1536. The system automatically increases the frame rate in parking / low-speed scenarios to ensure real-time perception.

[0042] In a specific example, firstly, user operation signals (such as reverse gear, parking button, 360-degree camera button, etc.), vehicle status (such as whether the vehicle speed is less than a threshold, parking space recognition signs), and environmental perception information (such as obstacle distance, parking space coordinates) are collected. A scene decision engine is then used to make scene decisions based on these three items. This scene decision engine can be the software decision logic of the domain controller. When the above operation signals are received, it combines the current actual vehicle status to determine whether to trigger frequency reduction or increase. Then, based on the scene decision result (such as high-speed cruise, navigation, low speed, or function activation), it decides whether to use a low frame rate command or a high frame rate command. Afterwards, according to the corresponding commands, the following are implemented: SerDes driver trigger source setting, image sensor frame rate setting, and surround-view camera output frame rate switching. Finally, the camera images are output through a low-computing-power SOC platform (low frame rate mode can release DDR bandwidth, while high frame rate mode can ensure perception accuracy and meet functional timeliness).

[0043] Please refer to Figure 4It shows a flowchart of a common sensor dynamic frame rate modification process, which is a specific example of a camera frame rate switching method for a vehicle provided in an embodiment of this application. like Figure 4 As shown, Step 1: Scene-driven frame rate switching decision. When the system determines that it is currently in a driving or other scenario that does not require a high frame rate surround view, it can dynamically reduce the surround view frame rate to a preset low frame rate (10Hz). In other scenarios, it can be increased to 20 / 30Hz at any time. For example, if the driving speed is greater than 30 km / h and the customer has not used parking or turned on 360-degree imaging, which require a high frame rate surround view, the system will default to a low frame rate scenario. In other words, the default is a low frame rate scenario, and the system will switch to a high frame rate when it recognizes that it needs to switch. The surround view frame rate usually refers to the video frame rate (FPS, Frames Per Second) of a 360-degree surround view system (AVM) or panoramic imaging system, that is, the number of image frames processed and output by the system per second. The frame rate directly affects the smoothness and real-time performance of the surround view image. In addition to surround view, it can also be used for cameras such as panoramic view.

[0044] Step 2: Specific implementation of dynamic frame rate switching at the hardware level.

[0045] Taking the commonly used ISX031 fisheye camera as an example, both VMAX and VMAX_OFFSET can be modified simultaneously; for the OVX3J sensor, the VTS-related registers can be modified to achieve the same result; and for the AR0233, the frame length register can be called to modify it. The ultimate goal is to match the sensor's output frame rate with the new trigger signal in the shortest possible time.

[0046] Table 1 As shown in Table 1, GPIO_SEL mainly selects two GPIOs as inputs to the trigger source at the deserializer end and passes them through to the serializer. The serializer will then select which of these two GPIOs to use as its trigger signal to output to the camera as needed. Furthermore, multiple frame rate switching can be set. For example, if three frame rate switchings are required, three trigger sources can be set. If other numbers of frame rate switchings are required, only the number of trigger sources needs to be modified. This application does not impose any restrictions on this.

[0047] TEIG_CFG refers to a fixed GPIO pin on the module's serializer that connects to the sensor. It outputs a fixed trigger frequency to the sensor for periodic graph output. This frequency is 10Hz, 30Hz, or other frequencies passed through from two or more GPIOs of the deserializer. The serializer can launch the selection of a specific trigger source based on the GPIO ID of the deserializer, thus quickly modifying the frequency of the trigger source received by the sensor on the module side. DES pass-through refers to a SERDES technology, such as Maxim Integrated's SERDES GMSL2 protocol or TI's FPD Link protocol. These technologies support the transmission of an input signal from a specific GPIO on the deserializer side of the domain controller to a specific GPIO on the serializer of the module.

[0048] In other embodiments, this application also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the camera frame rate switching method for a vehicle in any of the above method embodiments. As one implementation, the non-volatile computer storage medium of this application stores computer-executable instructions, which are configured as follows: Collect vehicle information; The decision on whether to switch the frame rate of the preset camera is based on the information collected. If a switch is required, the underlying camera hardware is linked in real time to switch the frame rate of the preset camera. The underlying camera hardware includes a deserializer that can output multiple trigger sources simultaneously, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate.

[0049] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the camera frame rate switching device for the vehicle, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely configured relative to a processor, which can be connected to the camera frame rate switching device for the vehicle via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0050] This application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform any of the above-described camera frame rate switching methods for vehicles.

[0051] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 5 As shown, the device includes one or more processors 510 and a memory 520. Figure 5 Taking a processor 510 as an example, the device for the camera frame rate switching method for a vehicle may further include an input device 530 and an output device 540. The processor 510, memory 520, input device 530, and output device 540 can be connected via a bus or other means. Figure 5 Taking a bus connection as an example, the memory 520 is the aforementioned non-volatile computer-readable storage medium. The processor 510 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 520, thereby implementing the camera frame rate switching method for vehicles described in the above method embodiment. The input device 530 can receive input digital or character information and generate key signal inputs related to user settings and function control of the camera frame rate switching device for vehicles. The output device 540 may include a display screen or other display device.

[0052] This application also provides a mobile platform, which may include: a vehicle body, a power system, and electronic devices as described in the above embodiments. The power system is installed on the vehicle body and provides power; the principle and implementation of the electronic devices are consistent with those described in the above embodiments, and will not be repeated here. The electronic devices may be controllers or other computing devices installed on the mobile platform. Optionally, the mobile platform may include at least one of the following: a vehicle, a mobile robot, or an unmanned vehicle.

[0053] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0054] In one embodiment, the above-described electronic device is applied to a camera frame rate switching device for a vehicle, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: Collect vehicle information; The decision on whether to switch the frame rate of the preset camera is based on the information collected. If a switch is required, the underlying camera hardware is linked in real time to switch the frame rate of the preset camera. The underlying camera hardware includes a deserializer that can output multiple trigger sources simultaneously, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate.

[0055] The electronic devices described in this application exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0056] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0057] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (e.g., iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0058] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0059] (5) Other electronic devices with data interaction functions.

[0060] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0061] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for switching camera frame rates for vehicles, comprising: Collect vehicle information; The decision on whether to switch the frame rate of the preset camera is based on the information collected. If a switch is required, the underlying camera hardware is linked in real time to switch the frame rate of the preset camera. The underlying camera hardware includes a deserializer that can output multiple trigger sources simultaneously, a serializer for receiving the required trigger sources, and an image sensor for dynamically modifying the output frame rate.

2. The method according to claim 1, characterized in that, The real-time linkage with the underlying camera hardware to switch the frame rate of the preset camera includes: In response to a frame rate switching command, the trigger source mapped to the serializer is modified; Modify the registers in the image sensor used for synchronizing frame rate to switch the frame rate of the preset camera.

3. The method according to claim 2, characterized in that, The modification of the registers in the image sensor used for synchronizing frame rate-related parameters includes: Detect the sensor type of the image sensor; The configuration is modified by calling the corresponding register based on the sensor type.

4. The method according to claim 3, characterized in that, The sensor type includes at least one of the following: ISX031, IMX019, IMX728, IMX424, OVX3J, OX03D4C, OX08D10, and AR0233.

5. The method according to claim 1, characterized in that, Prior to the real-time linkage with the underlying camera hardware, the method further includes: Set the image sensor to external trigger mode.

6. The method according to claim 1, characterized in that, The vehicle information includes at least one of the following: vehicle status information, environmental perception information, and user operation signals.

7. The method according to claim 5, characterized in that, The decision on whether to switch the frame rate of the preset camera based on the collected information includes: The system determines whether the current scene requires high frame rate surround view based on the vehicle status information, the environmental perception information, and / or the user's operation signal. If you are in a scene that requires high frame rate panoramic view, switch the preset camera frame rate to high frame rate; If you are in a scene where a high frame rate surround view is not required, switch the preset camera frame rate to a low frame rate.

8. The method according to claim 7, characterized in that, The scenarios requiring high frame rate surround view include at least one of the following: the gear shifting to reverse, the vehicle being in motion and the speed being below a preset threshold, the parking function being activated, the user viewing 360-degree images, and the panoramic monitoring system being triggered.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

10. A mobile platform, characterized in that, The device includes a vehicle body and electronic equipment, the electronic equipment comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-8.

Images