An image acquisition method and apparatus

Abstract

The application provides an image acquisition method and device. The method comprises the following steps: preloading a plurality of configuration parameters supported by an external image sensor and a peripheral interface channel into a cache area. An image configuration switching instruction transmitted by an application layer is received through a user API interface or an image configuration switching instruction is automatically generated according to external factors, so as to control the external image sensor and the peripheral interface channel to switch to a target image configuration based on the image configuration switching instruction, acquire image data through the configured external image sensor, and receive the image data through the peripheral interface channel. The time stamp of the image data is monitored frame by frame in real time, so as to perform timing compensation processing on the image data. The processed image data is subjected to configuration verification and timing verification based on the target image configuration and a preset timing threshold, abnormal data frames are filtered, and finally, the image data is obtained.

Description

Technical Field

[0001] This application belongs to the field of vehicle-mounted visual image acquisition and processing technology, specifically relating to an image acquisition method and device. Background Technology

[0002] In the field of high-level intelligent autonomous driving technology, the visual image acquisition system is a core foundational module for vehicle environmental perception, target recognition, and decision-making. The continuity, stability, format compatibility, and dynamic resolution adaptation capabilities of image acquisition directly determine the accuracy of autonomous driving perception algorithms and driving safety. As autonomous driving application scenarios become increasingly complex, the need for real-time switching between near-range and far-range vision during driving is becoming more urgent. The existing single-channel image acquisition solution based on the Orin (NVIDIA DRIVE Orin) platform has a rigid architecture, which exposes many technical defects under the harsh conditions of autonomous driving, severely limiting the application of the Orin platform in the field of high-end intelligent driving visual acquisition.

[0003] Existing single-channel image acquisition solutions do not support seamless dynamic resolution switching. When switching between different acquisition resolutions, the image sensor device must be restarted or the Orin platform acquisition driver must be reloaded, resulting in an interruption of at least 500ms in the image acquisition link. This interruption can directly cause problems such as visual stuttering, black screens, and flickering.

[0004] The existing Orin platform acquisition driver only supports single image format output of YUV or RAW, and cannot achieve dual-format compatible acquisition. In order to adapt to the format requirements of different perception algorithms, the system must rely on additional hardware and software devices to complete the format conversion. A single conversion will introduce an acquisition delay of at least 100μs, which will destroy the real-time performance of visual acquisition. At the same time, the conversion process is prone to data distortion such as missing image pixels and color deviation, which directly reduces the recognition and judgment accuracy of subsequent perception algorithms.

[0005] During resolution or image format switching, existing solutions cannot achieve synchronous updates of sensor acquisition parameters and Orin platform VI acquisition channel parameters. The parameter update timing is deviated, causing misalignment of line and field synchronization signals and frame timing disorder, which in turn leads to frame loss, image offset, and image misalignment, significantly reducing the stability of autonomous driving visual imaging and failing to meet the requirements of high-reliability imaging conditions.

[0006] In summary, existing single-channel image acquisition solutions based on the Orin platform suffer from three major technical problems: hardware interruption during multi-resolution switching, lack of dual-format compatibility, and misaligned switching timing. These problems prevent the Orin platform from meeting the core requirements of dynamic multi-resolution switching, YUV / RAW dual-format compatibility, and flicker-free continuous image output. Consequently, these solutions are ill-suited for demanding scenarios requiring high precision and stability, such as near-far vision switching in autonomous driving. These issues have become key technical bottlenecks hindering the large-scale deployment of the Orin platform in the field of high-level intelligent driving vision acquisition. Summary of the Invention

[0007] To address the aforementioned technical problems, this application proposes an image acquisition method and apparatus that is delay-free, flicker-free, and compatible with multiple resolutions and image formats. It effectively solves the technical problems existing in current single-channel image acquisition based on the Orin platform, such as hardware interruption during multi-resolution switching, lack of multi-format compatibility, misaligned switching timing, poor adaptability of buffer resources, and ineffective filtering of abnormal frames during switching.

[0008] Specifically, this application proposes an image acquisition method, comprising: preloading multiple configuration parameters supported by a peripheral interface channel and an external image sensor into a buffer; receiving an image configuration switching instruction transmitted from the application layer via a user API interface or automatically generating an image configuration switching instruction based on external factors, so as to control the external image sensor and the peripheral interface channel to switch to a target image configuration based on the image configuration switching instruction; acquiring image data through the configured external image sensor, and receiving the image data through the peripheral interface channel; monitoring the timestamp of the image data frame by frame in real time, so as to perform timing compensation processing on the image data based on a preset global reference clock; and performing configuration verification and timing verification on the processed image data based on the target image configuration and a preset timing threshold, filtering abnormal data frames to obtain the final image data.

[0009] In the above technical solution, image configuration switching commands are received through a user API interface or automatically generated based on external factors, thereby controlling the external image sensor to switch to the target image configuration. This avoids interruptions caused by device restarts or system reloading, preventing issues such as visual lag, black screens, and flickering. By monitoring the timestamps of image data frame by frame in real time and performing timing compensation processing based on a preset global reference clock, dynamic correction of time fluctuations caused by configuration switching is achieved, effectively preventing data frame timing errors. By performing configuration and timing verification on the processed image data and filtering abnormal data frames, the stability and correctness of the output image data frames are ensured, preventing the application layer from receiving erroneous data and ensuring the reliability of image acquisition.

[0010] As one implementation, before preloading the various configuration parameters supported by the peripheral interface channel and the external image sensor into the driver cache, the method further includes: obtaining the device parameters of the external image sensor; performing identity verification on the external image sensor based on the device parameters; refusing access to the external image sensor when the identity verification fails; and accessing the external image sensor and performing initial configuration on the external image sensor and the peripheral interface channel when the identity verification passes.

[0011] By reading the device parameters of the external image sensor for identity verification, the system can reject external image sensors that do not meet the access conditions, thus preventing driver crashes or acquisition anomalies caused by incorrect hardware connections or model numbers, and ensuring that the data acquisition source is trustworthy.

[0012] Furthermore, the step of controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching instruction includes: retrieving target configuration parameters from the buffer based on the image configuration switching instruction and writing them into the register of the external image sensor; the target configuration parameters include target resolution parameters and target image format parameters; controlling the resolution of the external image sensor to switch to the target resolution based on the target resolution parameters; and controlling the image format of the external image sensor to switch to the target image format based on the target image format parameters.

[0013] By controlling the resolution of the external image sensor to the target resolution using the target resolution parameter, the switching process does not require restarting the external image sensor device or reloading the platform acquisition driver, avoiding interruptions in the image acquisition chain and preventing visual stuttering, black screens, and flickering issues. Furthermore, by controlling the image format of the external image sensor to the target image format using the target image format parameter, it can be compatible with multiple image formats and adapt to the format requirements of different sensing algorithms. This eliminates the need for additional hardware or software to perform format conversion, avoiding data distortion such as missing pixels and color deviations caused by format conversion.

[0014] Furthermore, the step of controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching command also includes: controlling the peripheral interface channel to synchronously update the configuration based on the target configuration parameters.

[0015] By controlling the peripheral interface channel to synchronously update the configuration with the external image sensor, faults such as misalignment of line and field synchronization signals, disordered data frame timing, and loss of data frames caused by deviations in parameter update timing are avoided. This effectively improves the stability of visual imaging for autonomous driving and meets the requirements of high-reliability imaging conditions.

[0016] Furthermore, the real-time frame-by-frame monitoring of the timestamp of the image data to perform timing compensation processing on the image data based on a preset global reference clock includes: real-time frame-by-frame monitoring of the timestamp of the image data; and performing timing compensation processing on the image data based on a preset global reference time and the timestamp using a preset timing compensation algorithm.

[0017] By monitoring timestamps frame by frame in real time, the system achieves high-precision perception of the actual arrival time of each frame of image data. Once it is found that the timestamp of a certain frame deviates from the preset global reference time, a preset timing compensation algorithm is used for timing compensation processing. Through timing compensation processing, the timestamp intervals of each frame in the image data are made uniform and conform to the expected frame rate timestamps, making the image data smooth and continuous in the time dimension.

[0018] Furthermore, after performing time-series compensation processing on the image data, the method further includes: obtaining the amount of single-frame image data of the image data based on the target resolution and target image format; allocating a cache pool for the image data based on the amount of single-frame image data; and reclaiming the cache pool of historical resolutions in real time.

[0019] When resolution and image format change, the amount of data in a single frame changes. If a fixed-size buffer pool allocated for the old configuration is still used, the new image frames with high data volumes cannot be fully written, leading to data truncation, frame loss, or even system crashes due to memory overflow, causing software failures more serious than a black screen. This system accurately calculates the memory specifications required for the new single frame image after a configuration change. This allows for the pre-preparation of a buffer pool of the exact right size for the new image data, fundamentally eliminating the risk of memory overflow or insufficient allocation. It maintains high throughput efficiency throughout the acquisition pipeline, ensuring continuous and stable transmission of high-resolution, high-frame-rate data streams without any processing lag.

[0020] Furthermore, based on the target image configuration and the preset timing threshold, configuration verification and timing verification are performed on the processed image data, including: verifying whether the image resolution and image format of the timing-compensated image data are consistent with the target image configuration; if so, the configuration verification passes; otherwise, the configuration verification fails; and verifying whether the deviation between the preset global reference time and the timestamp of the timing-compensated image data is less than or equal to the preset timing threshold; if so, the timing verification passes; otherwise, the timing verification fails.

[0021] Cross-validation of each frame of image data is performed through two dimensions: configuration verification and timing verification. Configuration verification ensures that the resolution and format of the image content are consistent with the target and can capture residual frames that are still in the old format due to switching delays. Timing verification ensures that the time attributes of the frame are within a reasonable fluctuation range of the preset global reference time and can capture abnormal frames with distorted or completely disordered timestamps.

[0022] Furthermore, the filtering of abnormal data frames includes: when either the configuration verification or the timing verification fails, determining that the current data frame of the image data is an abnormal data frame and discarding the abnormal data frame; otherwise, determining that the current data frame of the image data is a valid data frame and retaining the valid data frame.

[0023] If there is only a verification step without an explicit discarding action, abnormal frames may still be misread, stored, or transmitted by an improper software path. Discarding abnormal data frames removes unqualified data, ensuring that abnormal data frames are completely prevented from flowing into any subsequent processing stage.

[0024] Furthermore, after acquiring the final image data, the method further includes: writing the final image data into a local memory, and transmitting the final image data to the application layer at a preset output frequency.

[0025] By writing the final image data into the local memory and transmitting the final image data to the application layer at a preset output frequency, the real-time outflow and high-speed local storage of the final image data are realized, ensuring that the data transmission is smooth and free of data distortion.

[0026] Based on the same inventive concept, this application also proposes an image acquisition device, comprising: at least one dual-mode external image sensor; a processor, which integrates an image signal processor; an interface circuit connecting the dual-mode external image sensor and the processor; and a memory coupled to the processor for storing computer program code, the computer program code including computer instructions; the processor reads the computer instructions from the memory to execute the image acquisition method.

[0027] Compared with the prior art, this application has at least the following beneficial effects: This application effectively solves the technical problems existing in current single-channel image acquisition based on the Orin platform, such as hardware interruption during multi-resolution switching, lack of multi-format compatibility, misaligned switching timing, poor adaptability of cache resources, and ineffective filtering of abnormal frames during switching. It receives image configuration switching commands through a user API interface or automatically generates image configuration switching commands based on external factors, thereby controlling the switching of external image sensors and peripheral interface channels to the target image configuration. This avoids interruptions caused by device restarts or system reloading, preventing visual stuttering, black screens, and flickering. By monitoring the timestamps of image data in real time frame by frame for timing compensation, it dynamically corrects time fluctuations caused by configuration switching, effectively preventing data frame timing errors. Through configuration and timing verification of the processed image data, it filters abnormal data frames, ensuring the stability and correctness of the output image data frames, preventing the application layer from receiving erroneous data, and ensuring the reliability of image acquisition. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the image acquisition method in an embodiment of this application.

[0029] Figure 2 This is a schematic diagram illustrating the connection between an external image sensor and the Orin platform, as shown in an embodiment of this application.

[0030] Figure 3 This is a schematic diagram illustrating the allocation of an elastic cache pool according to an embodiment of this application.

[0031] Figure 4 This is a detailed flowchart illustrating image acquisition as shown in an embodiment of this application.

[0032] Figure 5 This is a schematic diagram of an image acquisition device shown in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. Example 1:

[0035] Please refer to Figure 1 The image acquisition method mainly includes steps S100 to S500.

[0036] Step S100 includes: preloading various configuration parameters supported by the peripheral interface channel and the external image sensor into the buffer. The peripheral interface channel includes, but is not limited to, the CSI-2 interface and the VI acquisition interface. Modifications are made to the Orin platform kernel driver layer, the device tree file is pre-edited, and the VI (Video Input) acquisition channel is configured. This device tree file may include parameters such as the device ID of the supported external image sensors. Please refer to... Figure 2 The external image sensor can be a dual-mode sensor that supports both RAW and YUV image formats. It can be connected to the external image sensor through the peripheral interface channel. The interface can adopt a 50-ohm impedance matching design and integrate ESD±8kV electromagnetic protection to improve the connection stability in complex vehicle environments.

[0037] Preferably, before preloading the various configuration parameters supported by the peripheral interface channel and the external image sensor into the driver cache, the method further includes: obtaining the device parameters of the external image sensor; performing identity verification on the external image sensor based on the device parameters; refusing access to the external image sensor when the identity verification fails; and accessing the external image sensor and performing initial configuration on the external image sensor and the peripheral interface channel when the identity verification passes.

[0038] The device parameters include, but are not limited to, device ID, factory default parameters, supported resolution parameters, and image format parameters. After successful identity verification, the external image sensor and the peripheral interface channel can be initialized and configured to 8MP / RAW12 specifications.

[0039] In the specific implementation process, the Orin platform kernel driver layer sends a read instruction to the register of the external image sensor through the I2C bus to read its 16-bit device ID, such as 0x0820. The device ID is compared with a whitelist array that is pre-hardcoded in the driver or device tree. If the device ID is in the whitelist array, the external image sensor identity verification passes; otherwise, the external image sensor identity verification fails.

[0040] Step S200 includes: receiving an image configuration switching instruction transmitted from the application layer through a user API interface or automatically generating an image configuration switching instruction based on external factors, so as to control the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching instruction.

[0041] The image configuration switching command may include resolution and image format switching commands. The resolution includes, but is not limited to, 1MP, 2MP, 3MP, and 8MP, and the image format includes, but is not limited to, at least two of YUV422, YUV444, RAW10, and RAW12. The image configuration may also include parameters such as pixel clock, line and field synchronization signals, data bit width, and format identifier.

[0042] These external factors include, but are not limited to, ambient brightness and target detection distance. For example, an external image sensor may have a built-in photodiode continuously measuring ambient brightness and updating the Lux value of its internal register every 1ms. If the external image sensor is in YUV format, an automatic image configuration switching command is triggered when the Lux value is less than 10; if the external image sensor is in RAW format, an automatic image configuration switching command is triggered when the Lux value is greater than 50. Another example is that the external image sensor may have a built-in DSP that performs real-time analysis of the configured Region of Interest (ROI) to obtain the estimated distance to targets within the ROI. At a current resolution of 2MP, an automatic image configuration switching command is triggered only when the estimated distance is greater than 150m, controlling the external image sensor to switch to another resolution; at a current resolution of 8MP, an automatic image configuration switching command is triggered only when the estimated distance is less than 50m, controlling the external image sensor to switch to another resolution.

[0043] Preferably, controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching instruction includes: retrieving target configuration parameters from the buffer based on the image configuration switching instruction and writing them into the register of the external image sensor; the target configuration parameters include target resolution parameters and target image format parameters; controlling the resolution of the external image sensor to switch to the target resolution based on the target resolution parameters; and controlling the image format of the external image sensor to switch to the target image format based on the target image format parameters.

[0044] The target configuration parameters can be written to the register of the external image sensor via the I2C bus, with a write delay of ≤10μs.

[0045] Preferably, the step of controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching command further includes: controlling the peripheral interface channel to synchronously update the configuration based on the target configuration parameters.

[0046] The peripheral interface channel includes the VI acquisition channel of the Orin platform. By synchronously updating the configuration of the VI acquisition channel of the Orin platform, the external image sensor and the configuration parameters in the Orin platform are aligned without delay, thus avoiding timing misalignment caused by different parameters.

[0047] Step S300 includes: acquiring image data through a configured external image sensor, and the peripheral interface channel receiving the image data.

[0048] Step S400 includes: monitoring the timestamps of the image data frame by frame in real time, and performing timing compensation processing on the image data based on a preset global reference clock.

[0049] Preferably, the real-time frame-by-frame monitoring of the timestamp of the image data to perform timing compensation processing on the image data based on a preset global reference clock includes: real-time frame-by-frame monitoring of the timestamp of the image data; and performing timing compensation processing on the image data based on a preset global reference time and the timestamp using a preset timing compensation algorithm.

[0050] The preset global reference time can be the 19.2MHz PLL hardware clock of the Orin platform, and the preset timing compensation algorithm can be the ISP (Image Signal Processor) hardware timing compensation algorithm. That is, timing compensation is achieved using the Orin platform's built-in hardware multiplier in the ISP, with a compensation delay ≤10ns. When image data of the target resolution / format is transmitted to the Orin platform, timing compensation processing is performed based on this preset global reference time using the ISP hardware timing compensation algorithm to ensure complete timing continuity between the target data frame and the current frame being switched. The timing deviation between the two frames can be set to ≤30ns, thereby eliminating frame skipping, image misalignment, and other problems.

[0051] Preferably, after performing time-series compensation processing on the image data, the method further includes: obtaining the amount of single-frame image data of the image data based on the target resolution and target image format; allocating a cache pool for the image data based on the amount of single-frame image data; and reclaiming the cache pool of historical resolutions in real time.

[0052] The formula for calculating the data size of a single frame image is: Single frame image data size = Total number of pixels * Single pixel bit width / 8. The target image format is RAW10, with a single pixel bit width of 10 bits; the target image format is RAW12, with a single pixel bit width of 12 bits; the target image format is YUV422, with a single pixel bit width of 16 bits; and the target image format is YUV444, with a single pixel bit width of 24 bits. For example, if the target resolution is 1MP and the target image format is YUV422, then the data size of a single frame image is 1MP (1280*720) * 16 bits / 8 = 2MB, which is 2MB. For example, if the target resolution is 8MP and the target image format is RAW12, then the data size of a single frame image is 8MP (3840*2160) * 12 bits / 8 = 12MB, which is 12MB. Please refer to... Figure 3 It can call a pre-configured 4GB tone cache pool, adopt an on-demand allocation and real-time recycling dynamic scheduling strategy, automatically adjust the cache block size according to the target specifications, and achieve high-speed caching of image data acquired after switching by using a cache pool allocation latency of ≤5ns. At the same time, it can reclaim idle cache resources of the previous resolution in real time, achieving a cache utilization rate of ≥90%.

[0053] Furthermore, step S500 includes: performing configuration verification and timing verification on the processed image data based on the target image configuration and a preset timing threshold, filtering out abnormal data frames, and obtaining the final image data.

[0054] Preferably, configuration verification and timing verification are performed on the processed image data based on the target image configuration and a preset timing threshold, including: verifying whether the image resolution and image format of the timing-compensated image data are consistent with the target image configuration; if so, the configuration verification passes; otherwise, the configuration verification fails; and verifying whether the deviation between the preset global reference time and the timestamp of the timing-compensated image data is less than or equal to the preset timing threshold; if so, the timing verification passes; otherwise, the timing verification fails.

[0055] The preset global reference time can be a 19.2MHz PLL hardware clock, the preset timing threshold can be 50ns, and the supported image resolutions can be 1MP, 2MP, 3MP, and 8MP, with image formats including YUV422, YUV444, RAW10, and RAW12. Assuming the target resolution is 8MP and the target image format is RAW12, if the current frame parameters show a resolution of 8MP and an image format of RAW12, then the current frame and target image configuration are consistent, and the configuration verification passes. If the actual deviation between the timestamp and the preset global reference time is 22ns, meaning this actual deviation is less than the preset timing threshold, then the timing verification passes.

[0056] Preferably, the filtering of abnormal data frames includes: when either the configuration verification or the timing verification fails, determining that the current data frame of the image data is an abnormal data frame and discarding the abnormal data frame; otherwise, determining that the current data frame of the image data is a valid data frame and retaining the valid data frame.

[0057] For example, if the resolution and image format in the current frame parameters are inconsistent with the target resolution parameters or target format, the configuration verification fails, the current frame is determined to be an abnormal data frame, and the abnormal data frame is discarded. The data frame discarding delay is ≤20ns to ensure that the abnormal frame rejection rate is 100%.

[0058] Preferably, after acquiring the final image data, the method further includes: writing the final image data into a local memory, and transmitting the final image data to the application layer at a preset output frequency.

[0059] Valid image data frames after anomaly filtering can be transmitted to application layers such as Argus / DriveWorks via the Orin platform's standard V4L2 interface to meet the input requirements of in-vehicle intelligent driving perception algorithms. The preset output frequency can be 30fps±1fps. During data transmission, the output frequency is stabilized at this preset frequency to achieve a smooth, flicker-free, and data distortion-free process.

[0060] In the specific implementation process, please refer to Figure 4 First, configure the dual-channel data interface, which means modifying the Orin platform device tree file to complete the underlying hardware link adaptation of the CSI-2 image receiving channel and the VI hardware acquisition channel. In the kernel driver layer, pre-store multi-format parameter templates with resolutions of 1MP / 2MP / 3MP / 8MP and image formats of RAW10, RAW12, YUV422, and YUV444. The Orin platform driver layer divides an independent memory area to persistently store all template parameters.

[0061] The single-channel YUV / RAW dual-mode vehicle image sensor is connected to the Orin platform. The sensor device ID and factory default parameters are read via the I2C bus to verify whether the sensor device is compatible with all pre-stored resolution and format template parameters. The 8MP / RAW12 template parameters are loaded and synchronously written to the sensor device register and the Orin platform's VI acquisition channel for initialization configuration.

[0062] The system continuously listens for switching commands. Upon receiving a configuration switching command transmitted from the application layer or automatically triggering the generation of a configuration switching command, it quickly reads the target configuration parameters, such as 1MP YUV422, from the driver layer buffer. It updates the sensor device registers via the I2C bus, synchronously updating the Orin platform VI acquisition channel configuration register. The Orin platform's 19.2MHz PLL hardware clock is locked as the sole timing reference for the entire link, and timestamps are added to image frames of both the old and new specifications. When the first frame of the 1MP YUV422 image data is transmitted to the Orin platform, the ISP's built-in hardware multiplier is immediately invoked to execute a timing compensation algorithm. After timing compensation, the data size of a single frame is calculated. Based on the target configuration of 1MP (1280×720) and YUV422 (16-bit per pixel), the data size of a single frame is calculated as 1280×720×16÷8=2MB. A 2MB contiguous buffer block is allocated from the 4GB total elastic buffer pool to store the image data of the new mode. The buffer space occupied before the switch is synchronously reclaimed and marked as free and reusable resources.

[0063] The current frame header identifier field is read; the resolution identifier is 1MP and the format identifier is YUV422, which perfectly matches the target template, and the format verification passes. The timestamp of this frame based on the 19.2MHz PLL is extracted, and the deviation from the reference clock is calculated to be 22ns, which is less than the preset timing threshold of 50ns, and the timing verification passes. The valid image frames that have passed the verification are pushed to application layers such as Argus and DriveWorks through the Orin platform's native V4L2 interface. Example 2:

[0064] Please refer to Figure 5 This application also proposes an image acquisition device, comprising: at least one dual-mode external image sensor; a processor having an integrated image signal processor; an interface circuit connecting the dual-mode external image sensor and the processor; and a memory coupled to the processor for storing computer program code, the computer program code including computer instructions; the processor reading the computer instructions from the memory to execute the image acquisition method as described in Embodiment 1.

[0065] The processor can be any suitable processing device or set of processing devices, such as, but not limited to, a microprocessor, a microcontroller-based platform, an integrated circuit, one or more field-programmable gate arrays (FPGAs) and / or one or more application-specific integrated circuits (ASICs).

[0066] The memory can be volatile memory (e.g., RAM including non-volatile RAM, magnetic RAM, ferroelectric RAM, etc.), non-volatile memory (e.g., disk storage, flash memory, EPROM, EEPROM, memristor-based non-volatile solid-state memory, etc.), immutable memory (e.g., EPROM), read-only memory, and / or high-capacity storage devices (e.g., hard disk drives, solid-state drives, etc.). In some examples, the memory includes multiple types of memory, particularly volatile and non-volatile memory. Memory is a computer-readable medium on which one or more computer programs (such as software for operating the methods of this disclosure) can be embedded. The computer program can embody one or more of the methods or logic described herein. For example, the computer program may reside entirely or at least partially within any one or more of the memory, computer-readable medium, and / or within a processor during execution. The processor and memory can be integrated in an automotive SoC, as exemplified in this embodiment using the Orin platform, or the memory may be independent of the SoC.

[0067] The interface circuit is a collection of physical and logical links between the dual-mode external image sensor and the processor for image data transmission, control command exchange, and status information feedback. The interface circuit can employ an ohmic impedance matching network, an ESD protection array, etc. It can use a serial link based on the GMSL protocol or a conversion link based on the MIPI CSI-2 protocol. In some examples, the registers of the dual-mode external image sensor can also be connected to the processor via an I2C bus to write target resolution parameters and target image format parameters into the registers of the dual-mode external image sensor via the I2C bus.

[0068] In summary, this application receives image configuration switching commands via a user API interface or automatically generates image configuration switching commands based on external factors, thereby controlling the external image sensor to switch to the target image configuration. This avoids interruptions caused by device restarts or system reloading, preventing issues such as visual lag, black screens, and flickering. By monitoring the timestamps of image data in real time frame by frame for timing compensation, it dynamically corrects time fluctuations caused by configuration switching, effectively preventing data frame timing errors. By performing configuration and timing verification on the processed image data, abnormal data frames are filtered out, ensuring the stability and correctness of the output image data frames, preventing the application layer from receiving erroneous data, and ensuring the reliability of image acquisition.

[0069] In the several embodiments provided in this application, it will be understood that each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0070] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. An image acquisition method, characterized in that, include: Preload various configuration parameters supported by peripheral interface channels and external image sensors into the cache area; The system receives image configuration switching instructions transmitted from the application layer via the user API interface or automatically generates image configuration switching instructions based on external factors, and controls the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching instructions. Image data is acquired by a configured external image sensor, and the peripheral interface channel receives the image data. The timestamps of the image data are monitored in real time frame by frame to perform timing compensation processing on the image data based on a preset global reference clock; Furthermore, based on the target image configuration and preset timing threshold, the processed image data is subjected to configuration verification and timing verification to filter abnormal data frames in order to obtain the final image data.

2. The image acquisition method according to claim 1, characterized in that, Before preloading the various configuration parameters supported by the peripheral interface channel and the external image sensor into the driver cache, the method further includes: Obtain the device parameters of the external image sensor; The external image sensor is verified based on the device parameters. If the external image sensor fails the verification, access to the external image sensor is refused. When the external image sensor passes the identity verification, the external image sensor is connected and the external image sensor and peripheral interface channel are initially configured.

3. The image acquisition method according to claim 1, characterized in that, The step of controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching command includes: Based on the image configuration switching instruction, the target configuration parameters are retrieved from the buffer and written to the register of the external image sensor; the target configuration parameters include target resolution parameters and target image format parameters; Based on the target resolution parameter, the resolution of the external image sensor is controlled to switch to the target resolution; Based on the target image format parameters, the image format of the external image sensor is switched to the target image format.

4. The image acquisition method according to claim 3, characterized in that, The method of controlling the external image sensor and the peripheral interface channel to switch to the target image configuration based on the image configuration switching command further includes: Based on the target configuration parameters, the peripheral interface channel is controlled to synchronously update its configuration.

5. The image acquisition method according to claim 3, characterized in that, The real-time frame-by-frame monitoring of the timestamps of the image data, and the timing compensation processing of the image data based on a preset global reference clock, includes: Real-time frame-by-frame monitoring of the timestamps of the image data; Based on a preset global reference time and the timestamp, a preset timing compensation algorithm is used to perform timing compensation processing on the image data.

6. The image acquisition method according to claim 5, characterized in that, After performing time-series compensation processing on the image data, the method further includes: The amount of single-frame image data is obtained based on the target resolution and target image format. The image data is allocated to a cache pool based on the amount of image data in a single frame, and the cache pool of historical resolutions is reclaimed in real time.

7. The image acquisition method according to claim 1, characterized in that, Based on the target image configuration and preset temporal threshold, configuration verification and temporal verification are performed on the processed image data, including: Verify whether the image resolution and image format of the image data after time-compensated processing are consistent with the target image configuration. If they are, the configuration verification passes; otherwise, the configuration verification fails. If the deviation between the preset global reference time and the timestamp of the image data after time-compensated processing is less than or equal to the preset time threshold, the time-test passes; otherwise, the time-test fails.

8. The image acquisition method according to claim 7, characterized in that, The filtering of abnormal data frames includes: If either the configuration verification or the timing verification fails, the current data frame of the image data is determined to be an abnormal data frame and is discarded; otherwise, the current data frame of the image data is determined to be a valid data frame and is retained.

9. The image acquisition method according to claim 1, characterized in that, After obtaining the final image data, the process also includes: The final image data is written to local memory and transmitted to the application layer at a preset output frequency.

10. An image acquisition device, characterized in that, include: At least one dual-mode external image sensor; The processor integrates an image signal processor. An interface circuit connects the dual-mode external image sensor to the processor; A memory coupled to the processor for storing computer program code, the computer program code including computer instructions; The processor reads the computer instructions from the memory to execute the image acquisition method as described in any one of claims 1-9.

Images