A method and system for displaying a reverse image, a vehicle, a medium, a program product

Abstract

The application discloses a display method, system, vehicle, medium and program product of a reversing image, and the display method comprises the following steps: in the kernel starting stage, a kernel virtual drive and a fast reversing thread are created in the kernel layer, the kernel virtual drive is used for registering a virtual device, and the fast reversing thread is used for taking over a reversing hardware device to display the reversing image; after the operating system of an application layer is started, a starting completion instruction is sent to the virtual device; the fast reversing thread releases the reversing hardware device and sends event information to the operating system in response to the starting completion instruction received by the virtual device; and the operating system takes over the reversing hardware device to display the reversing image in response to the event information. The above scheme can improve the display speed of the reversing image.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, system, vehicle, medium, and program product for displaying a reversing image. Background Technology

[0002] As vehicles become increasingly intelligent, in-vehicle infotainment (IVI) systems are becoming more complex, and operating systems (such as Android and Linux) are growing in size. Modern cars are generally equipped with reversing cameras, which capture images through a camera mounted at the rear of the vehicle, transmit them via a serializer / deserializer (SerDes) link, and finally display them on the central control screen in the cockpit to help the driver observe blind spots and improve driving safety.

[0003] In a typical car startup process, the reversing camera usually runs as an application (App) of the operating system. After the system is powered on, it needs to go through multiple stages, including bootloader, kernel startup, system service initialization, display framework loading, and application startup, before a complete video acquisition and display chain can be established. Summary of the Invention

[0004] This application provides a method, system, vehicle, medium, and program product for displaying a reversing image, which can improve the display speed of the reversing image.

[0005] The first aspect of this application provides a method for displaying a reversing image, comprising: during the kernel startup phase, creating a kernel virtual driver and a fast reversing thread at the kernel layer, wherein the kernel virtual driver is used to register a virtual device, and the fast reversing thread is used to take over the reversing hardware device for displaying the reversing image; after the operating system at the application layer completes startup, it sends a startup completion command to the virtual device; in response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system; in response to the event information, the operating system takes over the reversing hardware device for displaying the reversing image.

[0006] In some embodiments, during the kernel startup phase, a kernel virtual driver and a fast reverse thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reverse thread is used to take over the reverse hardware device for displaying the reverse image. This includes: during the kernel startup phase, creating a kernel virtual driver at the kernel layer; registering virtual devices based on the kernel virtual driver; obtaining global configuration information using the virtual devices; and, in response to the global configuration information containing a fast reverse function identifier, creating a fast reverse thread at the kernel layer. The fast reverse thread is used to take over the reverse hardware device for displaying the reverse image.

[0007] In some embodiments, after creating a fast reversing thread at the kernel layer, the method further includes: the fast reversing thread initializing a reversing hardware device, which includes at least one of a GPIO interface, a SerDes link, a camera sensor, a display, and a camera controller; and the fast reversing thread displaying a reversing image based on the SerDes link, the camera sensor, the display, and the camera controller in response to receiving a reversing signal through the GPIO interface.

[0008] In some embodiments, the fast reversing thread includes an on state, a off state, and an exit state. The fast reversing thread, in response to receiving a reversing signal via the GPIO interface, displays a reversing image based on the SerDes link, camera sensor, display, and camera controller. This includes: when the fast reversing thread is in the off state, it switches to the on state upon receiving a reversing signal to start the reversing image video stream; when the fast reversing thread is in the on state, it switches to the off state upon not receiving a reversing signal or receiving an exit command to close the reversing image video stream; when the fast reversing thread is in the off state, it releases the reversing hardware device and sends event information to the application layer upon receiving an exit command to switch to the exit state.

[0009] In some embodiments, starting the reversing image video stream includes: the fast reversing thread controlling the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link; sending the reversing image video stream to the kernel display module so that the kernel display module renders the reversing image video stream and sends the rendered reversing image video stream to the display via the SerDes link; or the fast reversing thread responding to the virtual device receiving a start-up completion instruction, releasing the reversing hardware device, and sending event information to the operating system includes: the fast reversing thread in the closed state responding to the virtual device receiving a start-up completion instruction, releasing the reversing hardware device, and sending event information to the operating system to switch to the exit state.

[0010] In some embodiments, the kernel display module renders the reversing image video stream, including: in the initial stage, the fast reversing thread controls the kernel display module to render the reversing image video stream based on a first frame rate; after the initial stage, the fast reversing thread controls the kernel display module to render the reversing image video stream based on a second frame rate; wherein the first frame rate is less than the second frame rate; or after the fast reversing thread initializes the reversing hardware device, it further includes: allocating a shared memory region; the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, including: the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, and stores the reversing image video stream in the shared memory region; sending the reversing image video stream to the kernel display module, including: sending the reversing image video stream from the shared memory region to the kernel display module.

[0011] In some embodiments, the fast reverse thread, in response to the virtual device receiving a startup completion instruction, releases the reverse hardware device and sends event information to the operating system, including: the virtual device setting a startup completion flag in response to receiving the startup completion instruction; and the fast reverse thread releasing the reverse hardware device and sending event information to the operating system in response to the startup completion flag.

[0012] In some embodiments, the kernel startup phase is the late_initcall phase; or the operating system responds to event information and takes over the reversing hardware device to display the reversing image, including: the operating system responds to event information and determines that there is a kernel-level abnormality, and delays taking over the reversing hardware device.

[0013] A second aspect of this application provides a vehicle infotainment system, comprising: a kernel module, used during the kernel startup phase to create a kernel virtual driver and a fast reversing thread at the kernel layer; the kernel virtual driver is used to register a virtual device, and the fast reversing thread is used to take over the reversing hardware device for displaying a reversing image; and an application module, whose operating system, after startup, sends a startup completion command to the virtual device; wherein, in response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system; the operating system, in response to the event information, takes over the reversing hardware device for displaying a reversing image.

[0014] A third aspect of this application provides a vehicle comprising: a camera sensor; a display; a memory; and a processor connected to the camera sensor, the display, and the memory, wherein the memory stores program data and the processor executes the program data to implement the reversing image display method described above.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the reversing image display method described above.

[0016] The fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the reversing image display method described above.

[0017] The above scheme, during the kernel startup phase, creates a kernel virtual driver and a fast reversing thread at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device for displaying the reversing image. After the operating system at the application layer completes its startup, it sends a startup completion command to the virtual device. In response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system. In response to the event information, the operating system takes over the reversing hardware device for displaying the reversing image. Through this approach, on the one hand, during the kernel startup phase (before the operating system completes its startup), the kernel enables rapid display of the reversing image, allowing it to be displayed quickly as soon as the vehicle's infotainment system is powered on. On the other hand, the handshake mechanism between the kernel layer and the application layer (the interaction of the startup completion command and event information) avoids cross-system hardware resource conflicts and achieves seamless switching of the reversing image.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0020] Figure 1 This is a schematic flowchart of an embodiment of the reversing image display method provided in this application; Figure 2 This is a flowchart illustrating step S11 in one embodiment; Figure 3 This is a system architecture diagram of the vehicle infotainment system in one embodiment; Figure 4 This is a schematic diagram of the first process of implementing reversing image display based on kernel in one embodiment; Figure 5 This is a schematic diagram of the second process for implementing reversing image display based on the kernel in one embodiment; Figure 6 yes Figure 5 A diagram showing the switching between the corresponding on, off, and off states; Figure 7 This is one embodiment Figure 1 Corresponding interactive diagram; Figure 8 This is a schematic diagram of the framework of an embodiment of the vehicle infotainment system provided in this application; Figure 9 This is a schematic diagram of the framework of an embodiment of the vehicle provided in this application; Figure 10 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0021] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0022] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0023] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0024] In a typical car startup process, the reversing camera usually runs as an application (App) within the operating system. After the system powers on, it needs to go through multiple stages, including bootloader, kernel startup, system service initialization, display framework loading, and application startup, before a complete video capture and display chain can be established. The above method has the following problems: 1. Conflict between system startup time and security regulations.

[0025] Startup delay: Modern operating systems (such as Android) typically take 15 to 30 seconds or even longer to fully boot up. However, according to ISO safety standards or national traffic regulations (such as those of the NHTSA), the reversing camera must appear within a very short time (usually within 2 seconds) after the vehicle is put into reverse.

[0026] Display lag: Traditional reversing cameras rely on upper-level user-space frameworks (such as Android Camera Service and SurfaceFlinger). If we wait for these services to become ready, the reversing image will be delayed, which will not meet the safety requirement of "immediate reversing after loading".

[0027] 2. Low efficiency in software and hardware collaboration.

[0028] High resource consumption: During the system startup phase, CPU and memory resources are under extreme strain. If a user-mode program processes a high-resolution video stream, it will further slow down the overall system startup speed and may even cause screen stuttering.

[0029] The SerDes link is initialized too late: Traditional drivers often perform complex configurations of the Sensor and Deser late in the kernel loading process or even in user space. This serial execution mode wastes valuable startup time.

[0030] 3. Lack of a flexible kernel and system-level coordination mechanism.

[0031] The exit logic is opaque: Traditional fast reversing camera systems mostly operate in a "blind run" mode, meaning the kernel runs for a fixed period after startup. Due to the lack of real-time interaction instructions with the Android system layer (such as the Camera HAL), the kernel cannot know when Android has truly completed its takeover preparation. This results in either the kernel exiting too early (leading to a black screen) or exiting too late (leading to errors due to competing with Android for sensor control).

[0032] Missing state synchronization: The underlying hardware link status (such as whether the deserializer is locked or whether the camera is offline) cannot be fed back to the system framework that has not yet fully started in real time, which makes it very easy for the display link to collapse or for unrecoverable logical errors to occur at the edge of system handover.

[0033] In related embodiments, in a multi-core SoC architecture, an independent real-time core (such as the Cortex-M or Cortex-R series) runs an RTOS (such as FreeRTOS, QNX, etc.) specifically for acquiring and displaying reversing images. This approach places strict requirements on the hardware architecture, increasing the complexity of system design. Furthermore, the logic for switching display screens between the RTOS core and the Android main core (Cortex-A) (which typically requires hardware layer overlay) is extremely complex, easily leading to hardware resource contention and incurring very high development costs.

[0034] In related embodiments, during the Linux kernel startup phase, a simple driver directly initializes the I2C and MIPI links, starts the sensor, and delivers the image to the display framebuffer. This driver is typically designed to "force a fixed duration of operation after startup" (e.g., automatically exit after 10 seconds). This approach lacks interaction and coordination. Because the kernel driver cannot perceive the Android system's loading progress, it often results in a black screen due to the kernel driver exiting before Android is ready; or Android may be ready, but the kernel driver is still occupying hardware resources, causing the Android Camera service initialization to fail.

[0035] In related embodiments, the Android system's service list is streamlined as much as possible, and the startup priority of the Camera HAL and display service (SurfaceFlinger) is increased. Display latency is shortened by "forcibly starting" user-space services earlier. However, this approach is still limited by Android's massive system architecture (such as Zygote process creation and Binder communication initialization), making it difficult to consistently achieve a display time of less than 2 seconds. Furthermore, forcibly running a high-resolution video stream during the high-load phase of system startup can easily cause CPU fluctuations during startup, leading to system crashes or severe frame drops.

[0036] In some embodiments, the initialization parameters for the SerDes (deserializer / serializer) and the Sensor are hard-coded in the kernel bootloader or driver header file. This approach results in extremely poor driver extensibility. Once the version of the vehicle hardware (such as camera model or link channel number) changes, the entire kernel image must be recompiled.

[0037] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the reversing image display method provided in this application. The reversing image display method includes: Step S11: During the kernel startup phase, a kernel virtual driver and a fast reverse thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reverse thread is used to take over the reverse hardware device to display the reverse image.

[0038] Understandably, after the vehicle infotainment system is powered on, it first goes through the kernel boot stage, then the operating system, and finally the user interface is entered after the operating system boots up. Users can then control the vehicle infotainment system based on the user interface.

[0039] Taking Linux kernel and Android system as examples, in the in-vehicle system based on Linux and Android, the kernel boot phase is the core link in the entire system from pressing the power button to finally entering the desktop interactive interface. It is responsible for preparing the underlying hardware and then launching the Android operating system, the "big housekeeper".

[0040] During Linux kernel startup, thousands of drivers and subsystems cannot all be initialized at once; they must be initialized sequentially in a strict order (this mechanism is collectively called initcall). The kernel assigns them different priority levels (from 0 to 7). For example, core_initcall (level 1): the earliest, responsible for the most fundamental tasks such as scheduler and memory management. arch_initcall (level 3): responsible for low-level settings related to specific hardware architectures. device_initcall / module_init (level 6): most common hardware drivers (such as displays, CAN controllers, etc.) are activated at this stage. late_initcall (level 7): initialized only after most of the above devices and drivers have been initialized.

[0041] Optionally, the reversing camera display in step S11 can be implemented in the late_initcall stage. Specifically, many in-vehicle functions require the basic environment to be fully set up before they can run. For example, an advanced audio processing service or a cross-hardware collaboration function may depend on both an audio driver and specific bus communication. To ensure everything runs smoothly, the initialization of such drivers is placed in the late_initcall stage, guaranteeing that all underlying hardware and core subsystems they depend on are ready.

[0042] In one embodiment, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating step S11 in one embodiment. Step S11 may specifically include: S111: During the kernel startup phase, a kernel virtual driver is created at the kernel layer.

[0043] Combination Figure 3 , Figure 3This is a system architecture diagram of a vehicle infotainment system in one embodiment. The system architecture of this embodiment consists of three parts: a hardware layer, a kernel layer, and a user space layer (including an application layer, a framework layer, and user space interfaces). The user space interface includes a virtual character device interface and a sysfs state interface. The virtual character device is used to receive instructions from the application layer, and the sysfs interface is used for state synchronization and Uevent monitoring. The kernel layer includes a GPIO driver, a kernel virtual driver, a fast reversing thread, a camera controller, and a kernel display module. These modules work together to complete signal detection, device control, video processing, and image rendering. The hardware layer includes GPIO interfaces, camera sensors, a SerDes link, and a display.

[0044] S112: Register virtual devices based on kernel virtual drivers.

[0045] In the context of the Linux kernel, "virtual driver" and "virtual device" refer to components that do not directly control real physical hardware but are simulated by software to provide standard device interfaces and operations. A virtual driver is a kernel module disguised as a device driver; it does not operate on physical hardware itself but registers standard operation interfaces with the kernel. A virtual device is a software abstraction in the system, serving as the stage and object on which the virtual driver functions.

[0046] In one specific embodiment, during the late_initcall phase of the Linux kernel startup, the fastrvc_init function is called to register the platform_driver (kernel virtual driver) and create a virtual character device (e.g., named / dev / fastrvc). This virtual character device serves as a bridge for signaling interaction between user space and kernel space.

[0047] S113: Obtain global configuration information using virtual devices.

[0048] S114: In response to the global configuration information containing the fast reverse function identifier, a fast reverse thread is created in the kernel layer. The fast reverse thread is used to take over the reverse hardware device to display the reverse image.

[0049] Optionally, the virtual device reads various pre-configured parameters from the device tree, including GPIO pin configuration, camera sensor model, video channel number, resolution format, video standard, screen rotation angle, mirroring mode, and color configuration. These parameters determine the specific behavior of the driver and the hardware initialization method.

[0050] Furthermore, the virtual device obtains global configuration information. Optionally, the virtual device obtains the vehicle's global configuration information through the MetaZone partition. Specifically, the system reads the fast reverse function flag (the enable flag for the FastRVC function) and determines whether the vehicle is equipped with the fast reverse function (FastRVC function) based on this flag. If the FastRVC function is detected as disabled, it means that the system does not need the fast reverse function. In this case, the fast reverse driver should proactively yield to avoid hardware resource conflicts. If the FastRVC function is enabled, the subsequent fast reverse initialization process continues.

[0051] In other embodiments, the global configuration information may also include early pre-reading of vehicle physical sensors (such as wheel speed sensors and ultrasonic radar) in kernel mode. This allows for more precise startup decisions. For example, if MetaZone indicates the vehicle is equipped with an ultra-high-resolution display, the kernel virtual driver can automatically select a "half-resolution display" mode to reduce memory bandwidth pressure; or, based on ambient light sensor data, it can pre-adjust the camera sensor's exposure parameters (Gain / Exposure) at the kernel level to ensure the reversing image is clearly visible under any lighting conditions.

[0052] In an optional embodiment, after step S114, the fast reversing thread initializes the reversing hardware device, which includes at least one of the aforementioned GPIO interface, SerDes link, camera sensor, display, and camera controller; in response to receiving a reversing signal through the GPIO interface, the fast reversing thread displays the reversing image based on the SerDes link, camera sensor, display, and camera controller.

[0053] In an optional embodiment, dynamic detection of the Lock / Unlock signal is added during SerDes link initialization. If the SerDes link deserializer is detected to be unlocked (possibly due to camera damage or broken wiring harness), the kernel virtual driver immediately stops the initialization process and directly switches to displaying the stored "fault prompt image." This simplifies the exception handling logic. Fault diagnosis and user prompts are completed at the kernel level without waiting for Android system errors, avoiding kernel panic caused by link instability and improving the overall system safety of the vehicle.

[0054] Further reading Figure 4 , Figure 4This is a schematic diagram of the first process of implementing reversing image display based on the kernel in one embodiment. During the kernel startup phase, a kernel virtual driver is created at the kernel layer, and virtual devices are registered based on the kernel virtual driver. Then, using the global configuration information obtained from the virtual device, it is determined whether the fast reversing function flag is enabled. If the fast reversing function flag is not enabled, it means that the vehicle does not support the function of displaying reversing images in the kernel, and the process exits; if the fast reversing function flag is enabled, it means that the vehicle supports the function of displaying reversing images in the kernel, and a fast reversing thread is created. The fast reversing thread initializes hardware devices such as GPIO interfaces, SerDes links, camera sensors, displays, and camera controllers, and then enters the reversing image display process (main loop).

[0055] In one embodiment, the fast reversing thread initializes the reversing hardware, which may specifically include: Hardware Initialization: During hardware initialization, the driver first attempts to detect the camera sensor. If the detection fails, it indicates that the camera sensor is missing or has an abnormal connection; in this case, the rapid reverse function should safely exit. If the detection is successful, the driver sequentially completes the power configuration, register initialization, and operating mode configuration of the camera sensor. The camera sensor selects the correct resolution, frame rate, and video format based on the device tree parameters, and then initializes the MIPI receiver data path. After hardware initialization is complete, the driver enters the main loop processing phase.

[0056] In addition to the hardware initialization mentioned above, it may also include: GPIO Initialization: The system initializes the interrupt module of the GPIO driver. After the GPIO driver is configured, the GPIO interface can respond to the reversing signal from the vehicle's electrical system. When the driver engages reverse gear, the reversing light circuit is turned on, and the GPIO interface level changes. The driver identifies this change through edge detection.

[0057] Kernel display module initialization: Initializes a lightweight kernel display module in kernel mode. This module is specifically optimized for fast startup scenarios and consumes very few resources. After initialization, the module first displays the startup logo screen, ensuring that users see visual feedback from the moment the system powers on, thus improving the user experience.

[0058] Further reading Figure 5 , Figure 5 This is a schematic diagram of the second process for implementing reversing image display based on the kernel in one embodiment. Figure 5 That is Figure 4 The process after entering the reversing camera display process.

[0059] In this embodiment, we define three states for the fast reversing thread: on, off, and off. The on state indicates that the reversing video stream is enabled. Specifically, the fast reversing thread controls the camera controller to acquire the reversing video stream from the camera sensor via the SerDes link; it sends the reversing video stream to the kernel display module, enabling the kernel display module to render the reversing video stream and then send the rendered reversing video stream to the display via the SerDes link. The off state indicates that the reversing video stream is disabled (also known as the standby state). In one embodiment, combined with... Figure 3 The reversing signal is transmitted from the hardware layer through the GPIO driver and kernel virtual driver / virtual device to the fast reversing thread; video data is transmitted from the camera sensor through the SerDes link and camera controller to the kernel display module and finally output to the monitor. The exit state means stopping the fast reversing thread.

[0060] In other embodiments, after initializing the hardware, a shared memory region can be allocated. In this case, the aforementioned "enabled" state indicates that the reversing image video stream is enabled. Specifically, the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor via the SerDes link and stores the reversing image video stream in the shared memory region; the reversing image video stream is then sent from the shared memory region to the kernel display module. This method reduces the copying of data from the camera controller to the kernel display module, significantly reducing memory bandwidth usage and CPU load, allowing the system to run high-resolution (e.g., 1080P) reversing images with extremely low resource consumption.

[0061] The rapid reversing thread switches between three states based on the reversing signal, the start-up completion command, and the exit command. The reversing signal, obtained via the GPIO interface, indicates that the vehicle has shifted into reverse gear and a reversing operation is required. The start-up completion command is generated by the application layer's operating system after startup and is combined with... Figure 3 The startup completion command is sent to the kernel virtual driver through the virtual character device interface; the exit command is a system-generated command to force exit the reverse mode.

[0062] In one scenario, if the rapid reversing thread is in a disabled state but receives a reversing signal without receiving a startup completion command, it switches to an enabled state to start the reversing image video stream. This scenario occurs when the application layer operating system has not yet finished booting and reversing image display is required.

[0063] In another scenario, when the rapid reversing thread is enabled, it switches to the disabled state to close the reversing image video stream if it receives neither a start-up completion command nor a reversing signal, or if it receives an exit command but not a start-up completion command. This scenario includes two cases: one is when the application layer operating system has not yet finished booting but is already displaying the reversing image; if the reversing operation stops, the reversing image video stream is closed, and the thread enters the disabled state. The other is when the application layer operating system has not yet finished booting but is already displaying the reversing image; if an exit command is received, the reversing image video stream is closed. It is worth noting that when the thread is enabled, if an exit command is received, it does not immediately switch to the exit state but first switches to the disabled state. This is a safety mechanism to avoid accidental exits due to misoperation.

[0064] Specifically, the main loop continuously monitors the status of the reversing signal (e.g., a reversing signal of 1 indicates that the reversing signal has been received, and a reversing signal of 0 indicates that the reversing signal has not been received), and dynamically controls the opening and closing of the video stream based on the system's operating status and Android's readiness. When the driver engages reverse gear and the Android system is not yet ready, the driver immediately starts the video stream. At this time, the camera controller's reverse channel is opened, and the camera begins to output video data. The data is transmitted through the SerDes link and enters the camera controller. After simple format processing, it is directly rendered by the kernel display module and output to the display. The entire process runs entirely in kernel mode, without waiting for any user space services, thus enabling display to be completed in a very short time, meeting the response time limits required by safety regulations. The main loop implements signal debouncing processing through a continuous counting mechanism. When the reversing signal is valid, the counter is reset to zero; when the reversing signal disappears, the counter increments. Only when the counter reaches a preset threshold is the reversing state considered truly over. When the reversing signal disappears and the continuous detection count reaches the threshold, or when the system receives an exit command, the driver closes the video stream.

[0065] In another scenario, when the fast reverse thread is in the off state, it responds to the exit command by releasing the reverse hardware and sending event information to the application layer to switch to the exit state.

[0066] In another scenario, when the fast reverse thread is in a closed state, it responds to the start-up completion command by releasing the reverse hardware and sending event information to the application layer to switch to the exit state.

[0067] In other embodiments, other constraints can be added to the switching from the closed state to the exit state. For example, in the closed state, after receiving a start-up completion command or an exit command, it is necessary to wait for a set time threshold. If no reversing signal is received within the set time threshold, the reversing hardware device is released and an event information is sent to the application layer to switch to the exit state.

[0068] Optionally, in the above embodiments, when the kernel display module renders the reversing image video stream, its frame rate can be dynamically controlled, i.e., an adaptive frame rate adjustment can be added to the kernel display module. Specifically, a buffer monitoring mechanism can be added. When the CPU load is too high in the early stages of system startup, causing display link congestion, the kernel virtual driver actively reduces the acquisition frequency (e.g., from 30fps to 15fps) or adopts a "frame skipping strategy." This simplifies the implementation process and ensures smoothness. This ensures that even during the extremely busy boot moment, the reversing image remains smooth, while reserving more CPU time slices for system startup, achieving a win-win situation.

[0069] Further reading Figure 6 , Figure 6 yes Figure 5 The diagram illustrates the switching between the on, off, and exit states. The off state represents the initial and standby states of the fast reverse thread. In this state, the video stream is not running, and the system continuously monitors the reverse signal and the start-up completion command (Android ready state). The system can transition from the off state to the on state. When the reverse signal is valid, the system starts the video stream and enters normal operation mode. The off state can also transition to the exit state. When a start-up completion command or exit command is received, and a threshold of no continuous reverse signal detection is reached, the system exits. The on state indicates that the video stream is running. When the reverse signal disappears or an exit command is received, the system stops the video stream and returns to the standby state. Note that the on state cannot directly transition to the exit state; it must first return to the off state. This is a mandatory safety constraint in the state machine design. The exit state is the system's final state. After completing all resource cleanup and sending a notification event, the thread terminates.

[0070] In step S11 above, during the Linux kernel startup phase, the kernel's fast reverse thread pre-drives the GPIO interface, camera sensor, SerDes link, and camera controller CCIC, and works in conjunction with the kernel display module to directly render and output the image. This bypasses the massive user-space service stack of the Android system (such as Zygote, Camera Service, and SurfaceFlinger). It eliminates the overhead of process creation, Binder communication, and permission verification during the high-load phase of system startup. The video stream is directly closed-loop at the kernel level (capture → memory → display), greatly shortening the data path and thus breaking through the time bottleneck of traditional Android solutions at the physical level. It achieves an extremely fast response of "immediate connection and output," with a response time consistently within 2 seconds.

[0071] Step S12: After the application layer operating system has started, it sends a startup completion command to the virtual device.

[0072] In one embodiment, combined with Figure 3 The startup completion instruction is passed from the Android framework through the virtual character device interface and the kernel virtual driver to the fast reverse thread.

[0073] Step S13: The fast reverse thread responds to the virtual device receiving the start-up completion command, releases the reverse hardware device, and sends event information to the operating system.

[0074] Optionally, step S13 may include: the virtual device setting a startup completion flag in response to receiving the startup completion command; and the fast reverse thread releasing the reverse hardware device and sending event information to the operating system in response to the startup completion flag.

[0075] Specifically, a virtual device may include a status register. The virtual device can configure the value of this register to indicate whether a start-up completion command has been received. For example, in response to receiving a start-up completion command, the virtual device configures the value of the status register to 1. When the fast reverse thread detects that the value of the status register is 1, it releases the reverse hardware device and sends event information to the operating system.

[0076] In one embodiment, combined with Figure 3 Event information is passed from the fast reverse thread back to the Android framework via the sysfs state interface.

[0077] Step S14: The operating system responds to the event information and takes over the reversing hardware to display the reversing image.

[0078] In other embodiments, step S14 can be replaced by: the operating system responding to the event information and determining that the kernel layer is abnormal, delaying the takeover of the reverse hardware device. In this embodiment, a "two-way handshake" security heartbeat detection is added. After receiving the Android_ready signal, the kernel not only sends a one-time Uevent, but can also continuously update a heartbeat counter or status bit through a character virtual device. The user layer can determine whether the kernel layer is abnormal based on the heartbeat counter or status bit. This prevents the Android system from prematurely taking over the hardware due to misjudging the kernel state. If the Android side detects an abnormal kernel state, it can delay the startup of the Camera HAL to avoid system crashes and improve the fault tolerance of the handover process.

[0079] Step S11 mainly involves the specific process of displaying the reversing image at the kernel layer, while steps S12 to S14 involve the interaction between the kernel layer and the user layer and the switching of the reversing image.

[0080] Further reading Figure 7 , Figure 7 This is one embodiment Figure 1 The corresponding interaction diagram. The entire interaction process is divided into five key time points.

[0081] T1, the kernel startup phase, involves the hardware camera sensor capturing video, which is then rendered and displayed directly by the kernel's fast reverse thread through the kernel display module. This process does not require waiting for the Android system, and the response speed is determined by the kernel initialization speed.

[0082] T2, the operating system has finished booting. The camera service writes the boot completion instruction (Android_ready) to the virtual device (kernel virtual driver). The kernel virtual driver sets the boot completion flag of the fast reverse thread (Android_ready=true).

[0083] T3, Exit Process: The fast reversing thread responds to the virtual device's startup completion flag, releasing the reversing hardware device, specifically by sequentially stopping video capture, releasing controller resources, and cleaning up various modules.

[0084] T4, the fast reverse thread event information (Uevent) confirms that the kernel has released hardware resources.

[0085] T5 marks the official start of the Android Camera Link, which initializes standard camera services and takes over the display, completing a smooth transition from kernel-based fast display to native Android display.

[0086] Step S12 above, by creating a kernel virtual driver, enables interaction with the user layer, allowing the user-mode system to provide feedback on the operating system's loading status by writing boot completion instructions. This establishes an "active feedback instruction chain" from user mode to kernel mode, breaking the unidirectional logic of the kernel driver's "blind run." By monitoring the android_ready flag, the kernel achieves "on-demand release" of hardware control, ensuring that the kernel executes the Deinit process only at the precise moment when the Android HAL is ready to take over, avoiding black screens due to premature release or EBUSY hardware preemption errors due to late release.

[0087] Step S13 above is based on a strict state machine and an asynchronous Uevent exit notification mechanism. The driver maintains a STATE_CLOSE (closed state), STATE_OPEN (open state), and STATE_EXIT (exit state) state machine internally, and sends a broadcast Uevent (event information) after resource release, implementing an asynchronous closed-loop notification process of "clean up first, then notify". The state machine ensures that the driver can orderly reclaim the power resources of the GPIO interface and camera sensor under any abnormal trigger (such as a forced exit instruction); while the asynchronous Uevent mechanism acts as a "handover signal," enabling the Android HAL to know in real time and accurately that the kernel has completely relinquished bus control, thereby triggering subsequent link reconstruction and ensuring the continuity of image display.

[0088] The reversing image display method provided in this embodiment includes: during the kernel startup phase, a kernel virtual driver and a fast reversing thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device for displaying the reversing image. After the operating system at the application layer completes its startup, it sends a startup completion command to the virtual device. In response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system. In response to the event information, the operating system takes over the reversing hardware device for displaying the reversing image. Through this method, on the one hand, during the kernel startup phase (before the operating system finishes startup), the kernel is used to achieve fast display of the reversing image, enabling rapid display of the reversing image as soon as the vehicle system is powered on. On the other hand, through the handshake mechanism between the kernel layer and the application layer (interaction of startup completion command and event information), cross-system hardware resource conflicts are avoided, achieving seamless switching of the reversing image screen.

[0089] The beneficial effects of this embodiment can specifically include: 1. Breakthrough response speed (millisecond-level ultra-fast image output), enabling the reversing image to be displayed within 2 seconds after the system is powered on.

[0090] By establishing an independent video capture and rendering pipeline during the kernel startup phase (such as late_initcall), the complex process creation, Binder communication, permission checks, and display compositing framework (SurfaceFlinger) in Android user space are completely bypassed. Video data is directly looped within the kernel layer, greatly shortening the path from sensor capture to screen display.

[0091] 2. High system handover stability (seamless and smooth switching), which solves common problems such as black screen, screen freeze or hardware deadlock during the switching between kernel mode driver and user mode system.

[0092] A closed-loop handshake mechanism based on character virtual devices (kernel virtual drivers) was introduced. The kernel's fast reverse thread implements controlled handover of hardware control by monitoring the Android_ready instructions written by Android in real time. This "two-way confirmation" mechanism ensures that the kernel will only exit after the Android framework is fully ready, guaranteeing the continuity of image display.

[0093] 3. Excellent anti-interference and robust performance (signal debouncing logic) effectively filters out false triggering or frequent start-stop phenomena caused by complex electromagnetic noise in the vehicle environment.

[0094] The kernel thread employs a time-slice-based (e.g., msleep(25)) periodic monitoring and continuous status confirmation counter. This hardware signal debouncing algorithm, implemented at the driver level, can accurately identify actual reverse gear operations, avoiding invalid resource occupation caused by false triggering during high-load startup.

[0095] 4. Dynamic adaptability and low maintenance costs: A single software architecture can automatically adapt to various hardware configurations (such as RVC or AVM), eliminating the need to develop multiple firmware versions for different vehicle models.

[0096] It integrates a MetaZone-based non-volatile memory read mechanism. Before initialization, the driver dynamically identifies the current vehicle's function enable flags (such as METAZONE_CAM_AVM_EN) and adjusts the startup strategy accordingly. This "configuration-driven" design greatly simplifies subsequent mass production maintenance.

[0097] 5. Excellent system resource coordination efficiency, while providing extremely fast response, does not interfere with the Android system's startup process, avoiding system frame drops or slow startup.

[0098] The kernel driver uses a lightweight kernel display module (such as the fastrvc_flinger renderer), which has extremely low CPU and memory bandwidth usage. Because all the heavy video processing logic is offloaded to the kernel, the main CPU core can allocate more computing power to system-level services during the critical boot period, achieving parallel optimization of "fast-start display" and "system startup".

[0099] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0100] See Figure 8 , Figure 8 This is a schematic diagram of the framework of an embodiment of the vehicle infotainment system provided in this application. The vehicle infotainment system 800 includes a kernel module 81 and an application module 82.

[0101] In the kernel module 81, during the kernel startup phase, a kernel virtual driver and a fast reversing thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device for displaying the reversing image. After the operating system of the application module 82 starts up, it sends a startup completion command to the virtual device. In response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system. In response to the event information, the operating system takes over the reversing hardware device for displaying the reversing image.

[0102] In an optional embodiment, the kernel startup phase is the late_initcall phase.

[0103] Optionally, in some embodiments, kernel module 81 is specifically used to: create a kernel virtual driver in the kernel layer during the kernel startup phase; register a virtual device based on the kernel virtual driver; obtain global configuration information using the virtual device; and, in response to the global configuration information containing a fast reverse function identifier, create a fast reverse thread in the kernel layer, the fast reverse thread being used to take over the reverse hardware device for displaying the reverse image.

[0104] Optionally, in some embodiments, after the kernel module 81 creates the fast reverse thread, the fast reverse thread initializes the reverse hardware device, which includes at least one of a GPIO interface, a SerDes link, a camera sensor, a display, and a camera controller; the fast reverse thread displays the reverse image based on the SerDes link, the camera sensor, the display, and the camera controller in response to receiving a reverse signal through the GPIO interface.

[0105] Optionally, in some embodiments, the fast reverse thread includes an on state, a off state, and an off state.

[0106] Specifically, the fast reversing thread responds to receiving a reversing signal via the GPIO interface. When the fast reversing thread is in the off state, it switches to the on state upon receiving a reversing signal to start the reversing video stream. When the fast reversing thread is in the on state, it switches to the off state upon not receiving a reversing signal or receiving an exit command to stop the reversing video stream. When the fast reversing thread is in the off state, it releases the reversing hardware device and sends event information to the application layer upon receiving an exit command to switch to the exit state. When the fast reversing thread is in the off state, it releases the reversing hardware device and sends event information to the operating system upon receiving a startup completion command from the virtual device to switch to the exit state.

[0107] Specifically, the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link; it sends the reversing image video stream to the kernel display module so that the kernel display module can render the reversing image video stream, and then sends the rendered reversing image video stream to the display via the SerDes link.

[0108] Optionally, in some embodiments, during the initial stage, the fast reversing thread controls the kernel display module to render the reversing image video stream based on a first frame rate; after the initial stage, the fast reversing thread controls the kernel display module to render the reversing image video stream based on a second frame rate; wherein the first frame rate is less than the second frame rate.

[0109] Optionally, in some embodiments, kernel module 81 is further configured to allocate a shared memory region; the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, including: the fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, and stores the reversing image video stream in the shared memory region; kernel module 81 is further configured to send the reversing image video stream from the shared memory region to the kernel display module.

[0110] Optionally, in some embodiments, the virtual device sets a startup completion flag in response to receiving a startup completion instruction; the fast reverse thread releases the reverse hardware device and sends event information to the operating system in response to the startup completion flag.

[0111] Optionally, in some embodiments, the application module 82 is used to: in response to event information and upon determining a kernel-level exception, delay takeover of the reversing hardware device.

[0112] See Figure 9 , Figure 9This is a schematic diagram of a vehicle embodiment provided in this application. The vehicle 900 includes a processor 91 and a camera sensor 92, a display 93, and a memory 94 connected to the processor 91.

[0113] In one embodiment, the camera sensor 92 may include a camera installed at the rear of the vehicle for capturing images of the area behind the vehicle. In other embodiments, the camera sensor 92 may also include multiple cameras installed around the vehicle for capturing multi-directional images to display a panoramic reversing image.

[0114] The display 93 is generally located in the cockpit and typically includes a touch screen for user operation.

[0115] The memory 94 is used to store program data, and the processor 91 is used to execute the program data to implement the following method for displaying the reversing image: During the kernel startup phase, a kernel virtual driver and a fast reversing thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device to display the reversing image. After the operating system at the application layer completes startup, it sends a startup completion command to the virtual device. In response to the virtual device receiving the startup completion command, the fast reversing thread releases the reversing hardware device and sends event information to the operating system. In response to the event information, the operating system takes over the reversing hardware device to display the reversing image.

[0116] Specifically, processor 91 controls itself, as well as camera sensor 92, display 93, and memory 94, to implement the steps of any of the above-described embodiments of the reversing image display method. Processor 91 can also be referred to as a CPU (Central Processing Unit). Processor 91 may be an integrated circuit chip with signal processing capabilities. Processor 91 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 91 can be implemented using integrated circuit chips.

[0117] Please see Figure 10 , Figure 10This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 1000 stores a computer program 101 that can be executed by a processor. The computer program 101 is used to implement the steps of any of the above-described embodiments of the reversing image display method.

[0118] In some embodiments, this application also provides a computer program product, which, when executed by a processor, is used to implement the steps of the above-described reversing image display method embodiments.

[0119] Understandably, a computer program product can be a computer program product contained on a tangible computer-readable medium, which includes program code for performing the above-described method for displaying a reversing image. In some embodiments, the computer program product can be downloaded and installed from a network, and can also be copied, transferred, and installed between different computer hardware. Its wireless transmission method can include the Internet, Bluetooth, WIFI, etc., and its wired transmission method can include USB, Lightning, Type-C, etc.

[0120] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0121] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for displaying a reversing image, characterized in that, include: During the kernel startup phase, a kernel virtual driver and a fast reversing thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device to display the reversing image. After the application layer operating system completes its startup, it sends a startup completion command to the virtual device. The fast reversing thread responds to the virtual device receiving the start-up completion command by releasing the reversing hardware device and sending event information to the operating system; In response to the event information, the operating system takes over the reversing hardware device to display the reversing image.

2. The method for displaying a reversing image according to claim 1, characterized in that, During the kernel startup phase, a kernel virtual driver and a fast reverse thread are created at the kernel layer. The kernel virtual driver is used to register virtual devices, and the fast reverse thread is used to take over the reverse hardware device for displaying the reverse image, including: During the kernel boot phase, a kernel virtual driver is created at the kernel layer. Register virtual devices based on the kernel virtual driver; Use the virtual device to obtain global configuration information; In response to the global configuration information containing a fast reverse function identifier, a fast reverse thread is created in the kernel layer. The fast reverse thread is used to take over the reverse hardware device to display the reverse image.

3. The method for displaying a reversing image according to claim 2, characterized in that, After creating the fast reverse thread at the kernel layer, the process also includes: The fast reversing thread initializes the reversing hardware device, which includes at least one of a GPIO interface, a SerDes link, a camera sensor, a display, and a camera controller. The fast reversing thread responds to receiving a reversing signal through the GPIO interface and displays the reversing image based on the SerDes link, the camera sensor, the display, and the camera controller.

4. The method for displaying a reversing image according to claim 3, characterized in that, The fast reversing thread includes an on state, a off state, and an off state. In response to receiving a reversing signal through the GPIO interface, the fast reversing thread displays a reversing image based on the SerDes link, the camera sensor, the display, and the camera controller, including: When the fast reversing thread is in the off state, it switches to the on state in response to receiving a reversing signal to start the reversing image video stream; When the fast reversing thread is enabled, it switches to the disabled state in response to not receiving a reversing signal or receiving an exit command, thereby shutting down the reversing image video stream. When the fast reversing thread is in the closed state, in response to receiving an exit command, it releases the reversing hardware device and sends event information to the application layer to switch to the exit state.

5. The method for displaying a reversing image according to claim 4, characterized in that, The activation of the reversing camera video stream includes: The rapid reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link; Send the reversing image video stream to the kernel display module, so that the kernel display module renders the reversing image video stream and sends the rendered reversing image video stream to the display via the SerDes link; or The fast reversing thread, in response to the virtual device receiving the start-up completion command, releases the reversing hardware device and sends event information to the operating system, including: When the fast reversing thread is in the off state, in response to the virtual device receiving the start-up completion command, it releases the reversing hardware device and sends event information to the operating system to switch to the exit state.

6. The method for displaying a reversing image according to claim 5, characterized in that, The kernel display module renders the reversing image video stream, including: In the initial stage, the fast reversing thread controls the kernel display module to render the reversing image video stream based on the first frame rate; After the initial phase, the fast reversing thread controls the kernel display module to render the reversing image video stream based on a second frame rate; wherein the first frame rate is less than the second frame rate; or After the fast reversing thread initializes the reversing hardware device, it also includes: Allocate a shared memory region; The rapid reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, including: The fast reversing thread controls the camera controller to acquire the reversing image video stream from the camera sensor based on the SerDes link, and stores the reversing image video stream in the shared memory area; Sending the reversing camera video stream to the kernel display module includes: The reversing image video stream is sent from the shared memory area to the kernel display module.

7. The method for displaying a reversing image according to claim 1, characterized in that, The fast reversing thread, in response to the virtual device receiving the start-up completion command, releases the reversing hardware device and sends event information to the operating system, including: The virtual device sets a startup completion flag in response to receiving the startup completion command; The fast reversing thread responds to the start-up completion flag by releasing the reversing hardware device and sending event information to the operating system.

8. The method for displaying a reversing image according to claim 1, characterized in that, The kernel startup phase is the late_initcall phase; or In response to the event information, the operating system takes over the reversing hardware to display the reversing image, including: The operating system responds to the event information and determines that the kernel layer is abnormal, and delays taking over the reversing hardware device.

9. A vehicle infotainment system, characterized in that, include: The kernel module is used to create a kernel virtual driver and a fast reversing thread at the kernel layer during the kernel startup phase. The kernel virtual driver is used to register virtual devices, and the fast reversing thread is used to take over the reversing hardware device to display the reversing image. After the application module's operating system has started, it sends a startup completion command to the virtual device. The fast reversing thread responds to the virtual device receiving the start-up completion command by releasing the reversing hardware device and sending event information to the operating system; In response to the event information, the operating system takes over the reversing hardware device to display the reversing image.

10. A vehicle, characterized in that, include: Camera sensor; monitor; Memory; A processor is connected to the camera sensor, the display, and the memory, the memory being used to store program data, and the processor being used to execute the program data to implement the reversing image display method as described in any one of claims 1-8.

11. A computer-readable storage medium / computer program product, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for displaying a reversing image as described in any one of claims 1-8; The computer program product includes a computer program that, when executed by a processor, implements the method for displaying a reversing image as described in any one of claims 1-8.

Images