Unified control of mobile industry processor interface cameras

Abstract

Example systems, apparatuses, articles of art, and methods for implementing unified control of cameras via a mobile industry processor interface are disclosed. An example apparatus for controlling a camera is used to access command translation data from at least one of the camera's memory or the memory of a host device to be coupled to the camera. The example apparatus is also used to parse the command translation data to map commands to one or more register settings of the camera, the commands specifying camera configurations applicable to various camera types, the one or more register settings being camera-specific and used to configure the camera based on the commands. The example apparatus is also used to send one or more register settings to the camera via the mobile industry processor interface.

Description

Technical Field

[0001] This disclosure generally relates to Mobile Industry Processor Interface (MIPI) cameras, and more specifically, to unified control of MIPI cameras, MIPI camera sensors, MIPI-based camera modules, etc. Background Technology

[0002] Mobile Industry Processor Interface (MIPI) cameras are camera systems that conform to the MIPI Alliance standard, which is used for communication between the camera sensor and the host (e.g., host system, host processor, etc.). MIPI cameras are high-frame-rate cameras capable of producing high-resolution, high-quality images. The host utilizes appropriate settings to configure the MIPI camera module to adapt the camera's behavior to the specific requirements of the application, thereby ensuring optimal image quality, performance, and compatibility with the overall system. This configuration process may involve communication via an interface (e.g., inter-integrated circuit (IIC)). 2 C) Interfaces such as Universal Serial Bus (USB), PCIe (High-Speed ​​Peripheral Component Interconnect) and Ethernet send control commands and settings to the MIPI camera module. Attached Figure Description

[0003] Figure 1 This is a block diagram of a sample environment in which the sample unified MIPI control interface driver operates to configure a MIPI camera sensor.

[0004] Figure 2 This is a block diagram of another example environment in which the example unified MIPI control interface driver operates to configure the MIPI camera sensor.

[0005] Figure 3 This is a block diagram of another example environment in which the example unified MIPI control interface driver operates to configure the MIPI camera sensor.

[0006] Figure 4 This is a block diagram of an example unified MIPI camera driver.

[0007] Figure 5 yes Figures 1 to 3 A block diagram illustrating an example implementation of a unified MIPI control interface driver.

[0008] Figure 6 This is a sample processing flow for the unified MIPI camera driver.

[0009] Figure 7 The example operation of the unified MIPI camera driver in the usage environment is shown.

[0010] Figure 8An example of creating a command-transformed data structure is shown.

[0011] Figure 9 This is a flowchart illustrating example machine-readable instructions and / or example operations that can be executed, instantiated, and / or implemented by an example programmable circuit system to achieve... Figure 5 Unified MIPI camera driver.

[0012] Figure 10 This is a block diagram of an example processing platform, which includes a programmable circuit system configured to execute, instantiate, and / or implement. Figure 9 Example machine-readable instructions and / or example operations to implement Figure 5 The Unified MIPI Control Interface Driver 405.

[0013] Figure 11 yes Figure 10 A block diagram illustrating an example implementation of a programmable circuit system.

[0014] Figure 12 yes Figure 10 A block diagram of another example implementation of a programmable circuit system.

[0015] Figure 13 This is a block diagram of an example software / firmware / instruction distribution platform (e.g., one or more servers) used to distribute software, instructions, and / or firmware (e.g., corresponding to...) Figure 9 Example machine-readable instructions) are distributed to client devices associated with end users and / or consumers (e.g., for licensing, selling and / or using), retailers (e.g., for selling, reselling, licensing and / or re-licensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products that will be distributed to, for example, retailers and / or other end users such as direct purchase customers).

[0016] Generally, the same reference numerals will be used throughout the accompanying drawings and written description to refer to the same or similar parts. The drawings are not necessarily drawn to scale. Rather, the thickness of layers or regions in the drawings may be enlarged. Although the drawings show layers and regions with clear lines and boundaries, some or all of these lines and / or boundaries may be idealized. In reality, boundaries and / or lines may not be observable, may merge with each other, and / or may be irregular. Detailed Implementation

[0017] MIPI camera sensors (also known as MIPI cameras, MIPI camera modules, MIPI sensors, MIPI sensor modules, etc.) communicate with the host computer via a high-speed MIPI interface. Therefore, compared to other camera technologies such as Universal Serial Bus (USB) cameras, MIPI camera sensors offer higher resolution images, faster frame rates, and lower power consumption. However, for a MIPI camera sensor to function properly with the host computer, the host relies on a MIPI camera driver to facilitate communication with and configuration of the MIPI camera sensor. Different MIPI camera sensors manufactured by different sensor vendors can have different capabilities, characteristics, etc. Therefore, MIPI camera sensors from different vendors may include specific MIPI camera drivers developed by the respective sensor vendor, third-party image signal processor (ISP) vendor, third-party system-on-chip (SoC) vendor, etc., to support the specific configuration parameters of their respective MIPI camera sensors. Therefore, enabling the host computer to support new MIPI camera sensors has always involved significant software development and verification work to integrate the MIPI camera driver on the host computer. Such software development and validation work is very time-consuming and can be particularly problematic for host vendors such as image signal processor (ISP) suppliers or original equipment manufacturers (OEMs) looking to integrate MIPI camera sensors into their devices.

[0018] The example implementation disclosed in this document is a unified MIPI camera driver for configuring and controlling any MIPI camera sensor. The unified MIPI camera driver enables host devices to connect to any MIPI camera sensor without developing a new driver each time a different camera sensor type is used. The unified MIPI camera driver uses a command translation data structure to translate generic camera sensor control and / or configuration commands into camera-specific register settings and sends write operations to the MIPI camera sensor for those specific register settings.

[0019] Figure 1 This is a block diagram of example environment 100, in which an example unified MIPI camera driver operates to configure a MIPI camera sensor. Example environment 100 includes an example unified MIPI camera driver 101, which includes an example unified MIPI control interface driver 105 and an example MIPI streaming interface driver 110. Unified MIPI camera driver 101 facilitates communication between example MIPI camera sensor 115 and example image signal processor (ISP) 120. MIPI control interface driver 105 uses example inter-integrated circuit (IIC)... 2C) Controller 125 communicates with MIPI camera sensor 115 via example camera module control interface (CMCI) 130. MIPI streaming interface driver 101 facilitates data streaming from MIPI camera sensor 115 to ISP 120 via example camera module streaming interface (CMSI) 135. Example ISP driver 140 is a software component responsible for engaging with ISP hardware 120. The ISP driver controls and manages the functions of ISP 120, which processes image sensor data to generate final image or video frames of desired quality. ISP 120 uses pixel frame buffer 150 to store pixel data or processed images.

[0020] In the example shown, the Unified MIPI Control Interface driver 105 sends commands to configure one or more parameters of the MIPI camera sensor 115. In some examples, these parameters include resolution, frame rate, exposure, focus, etc. The MIPI Stream Interface driver 110 manages the data stream from the MIPI camera sensor module 115 to the ISP 120.

[0021] Example ISP 120 enhances and processes raw image data from MIPI camera sensor 115. ISP 120 performs tasks such as demosaicing (e.g., color reconstruction), white balance correction, color correction, noise reduction, and / or other image enhancement processing. In some examples, the output of ISP 120 is a processed image ready for encoding or display.

[0022] Example I 2 The C controller 125 is a hardware component that manages communication between the Unified MIPI Control Interface driver 105 and the MIPI camera sensor module 115 via CMCI 130 (e.g., I...). 2 Communication is performed via the CMCI input / output (I / O) bus. In some examples, the CMCI 130 can be an I / O bus. 3 C-bus or any other control interface. 2 The C controller 125 converts control commands and / or configuration data from the unified MIPI control interface driver 105 into I... 2 The appropriate signal on the C bus. Connect to the CMCI130 (e.g., I...). 2 Example MIPI camera sensor 115 (using the C-bus) receives control and / or configuration commands, processes these commands, and adjusts the configuration of the MIPI camera sensor 115 accordingly. The MIPI camera sensor 115 may have internal registers that store one or more settings. CMCI 130 is used to read from or write to these registers. In some examples, I... 2The C controller 125 receives status information or configuration information from the MIPI camera sensor 115.

[0023] In example environment 100, example camera module streaming interface (CMSI) 135 transmits image data from MIPI camera sensor 115 to ISP 120. In some examples, CMSI 135 is a MIPI interface, a low-voltage differential signaling (LVDS) interface, or any other streaming interface.

[0024] Example ISP driver 140 configures ISP 120 to manage image data streams and control various ISP functions. ISP driver 140 translates commands and / or requests from the operating system (OS) and / or one or more applications into low-level operations performed by ISP 120.

[0025] The example operating system (OS) camera frame adapter 145 serves as the interface between the MIPI camera sensor 115 and the OS. The OS camera frame adapter 145 provides a standardized set of application programming interfaces (APIs) and / or services for applications and drivers to interact with the MIPI camera sensor 115. The OS camera frame adapter 145 abstracts away the underlying hardware details and provides a unified interface for camera-related operations. For example, in a Windows® OS implementation, the OS camera frame adapter 145 could be a Device Media Foundation Transformer (DMFT). As another example, in a Google Chrome OS™ implementation, the OS camera frame adapter 145 could be a Camera Hardware Abstraction Layer (HAL). The camera HAL provides a standardized interface between the underlying camera hardware and higher-level software components.

[0026] Figure 2This is a block diagram of another example environment 200, in which the example unified MIPI camera driver operates to configure a MIPI camera sensor module. Example environment 200 illustrates the use of the MIPI camera sensor module in a distributed environment (e.g., an automotive or robotic environment) where the MIPI camera sensor module is not directly connected to a system-on-chip (SoC) ISP. In example environment 200, the example unified MIPI camera driver 201 includes an example unified MIPI control interface driver 205 and an example MIPI stream interface driver 210. The unified MIPI control interface driver 205 configures and controls the example MIPI camera sensor module 215. The MIPI stream interface driver 210 manages the data stream from the MIPI camera sensor module 215. The unified MIPI camera driver 201 is implemented on an example host platform 220. In example environment 200, multiple example MIPI camera sensor modules 215 are remotely connected to the host platform 220 using an example serializer 225 and an example deserializer 230. The example camera serial interface (CSI) 235 connects the MIPI camera sensor module 215 to the host platform 220 via a serializer 225 and a deserializer 230. The CSI 235 carries image data, control signals, and other information between the MIPI camera sensor module 215 and the host platform 220. The host platform 220 includes an example connector 240, an example ISP 245, an example Fast Peripheral Component Interconnect (PCIe) interface 250, an example host SoC 255, and an example ISP driver 260.

[0027] Example serializer 225 converts parallel data generated by CSI 235 into serialized data. The serialized data is transmitted via communication link 265, which can be a wireless network (e.g., Wi-Fi or any other wireless connection), a wired network (e.g., an Ethernet connection or any other wired connection), or any other communication medium. Example deserializer 230 receives the serialized data and deserializes it back into parallel form. Example parallel communication link 265 (e.g., CSI) is connected to deserializer 230 to carry image data to host platform 220.

[0028] Example ISP 245 processes and enhances image data received from MIPI camera sensor module 215. ISP 245 is connected to host SoC 255 via a high-speed serial communication interface such as example PCIe interface 250. PCIe interface 250 provides high-speed data transfer between ISP 245 and host SoC 255. In some examples, ISP 245 is connected to unified MIPI camera driver 201 and ISP driver 260 via connector 240. ISP driver 260 manages the functionality on ISP 245.

[0029] Figure 3 This is a block diagram of another example environment in which the example unified MIPI camera driver operates to configure a MIPI camera sensor module. Example environment 300 illustrates a MIPI camera sensor module connected via a Universal Serial Bus (USB) interface. In example environment 300, the example unified MIPI camera driver 301 includes an example unified MIPI control interface driver 305 and an example MIPI stream interface driver 310. The unified MIPI camera driver 301 is implemented on an example host SoC 315. The host SoC 315 also includes example ISP software 320 and ISP hardware 325. The ISP software 320 is connected to the unified MIPI control interface driver 305 via an example CMCI 330. The ISP hardware 325 is connected to the MIPI stream interface driver 310 via an example CMCI 335. The host SoC 315 includes an example USB connector 340 connected to a USB cable 345, which is connected to an example MIPI to USB converter 350. The MIPI to USB converter 350 is connected via an example MIPI PICSI 355 and an example I 2 The C-bus 360 communicates with the example MIPI camera sensor 365.

[0030] In example environment 300, the MIPI camera sensor module 365 sends image or video data to the MIPI to USB converter 350 via the MIPI CSI 355. The MIPI to USB converter 350 converts the serial data from the MIPI camera sensor module 365 into a USB output. This USB output is sent to the host SoC 315 via a USB cable 345. Converting serial data to USB format overcomes the length limitations associated with MIPI cables (e.g., the length of the MIPI cable is limited to less than or equal to 30 centimeters (cm)) and enables the transmission of image and / or video data over longer distances. Data received at the USB port 340 on the host SoC 315 is processed by the MIPI streaming interface driver 310 and streamed via the CMSI 335 to the ISP hardware 325 for further processing. In some examples, the CMSI 335 streams USB Video Class (UVC) data to the ISP hardware 325.

[0031] Example ISP software 320 configures one or more settings on the MIPI camera sensor 365. For example, ISP software 320 can configure the MIPI camera sensor 365 by setting parameters such as exposure time, white balance, focus, aperture, gain, and other image acquisition settings. ISP software 320 uses I... 2The CMCI 330 sends commands and / or control settings in C-signal format to the MIPI stream interface driver 310. The Unified MIPI Control Interface driver 305 converts configuration and / or control commands into USB format for transmission to the MIPI camera sensor module 365 via USB cable 345. The MIPI to USB converter 350 converts commands into MIPI format for transmission via I-C signal. 2 The C-bus 360 sends commands to the MIPI camera sensor module 365.

[0032] Figure 4 This is a block diagram of an example unified MIPI camera driver 400. The unified MIPI camera driver 400 includes an example unified MIPI control interface driver 405 and an example MIPI stream interface driver 410. The unified MIPI camera driver 400 can be used to implement... Figure 1 Unified MIPI camera driver 101 Figure 2 Unified MIPI camera driver 201 and / or Figure 3 One or more of the Unified MIPI Camera Driver 301. The Unified MIPI Control Interface Driver 405 manages the configuration and control of the MIPI camera sensor module. The MIPI Stream Interface Driver 410 manages the data stream from the MIPI camera sensor module to the host device. This data stream includes pixel frame data.

[0033] Figure 5 It is used to control / configure the MIPI camera sensor. Figures 1 to 4 A block diagram of an example implementation of the Unified MIPI Control Interface Driver 405. Figure 5 The unified MIPI control interface driver 405 can be instantiated by a programmable circuit system such as a central processing unit (CPU) that executes the first instruction (e.g., creating an instance of it, making it exist for any duration, making it materialized, implementing it, etc.). Alternatively or alternatively, Figure 5 The unified MIPI control interface driver 405 can be instantiated (e.g., created, made to exist for any duration, materialized, implemented, etc.) by: (i) an application-specific integrated circuit (ASIC); and / or (ii) a field-programmable gate array (FPGA) constructed and / or configured to perform the operation corresponding to the first instruction in response to the execution of the second instruction. It should be understood that... Figure 5 Some or all of the circuit system can therefore be instantiated at the same time or at different times. For example, Figure 5 Some or all of the circuit system can be instantiated in one or more threads that execute concurrently on the hardware and / or in one or more threads that execute serially on the hardware. Furthermore, in some examples, Figure 5Some or all of the circuitry in the system can be implemented by a microprocessor circuitry that executes instructions and / or an FPGA circuitry that performs operations, to implement one or more virtual machines and / or containers.

[0034] The example Unified MIPI Control Interface Driver 405 includes an example Conversion Data Loader Circuitry 505, an example Unified Command Interface Circuitry 510, an example Conversion Data Parser Circuitry 515, and an example Device-Specific Configuration Circuitry 520. The Unified MIPI Control Interface Driver 405 operates to access one or both of the following: an example first command conversion data structure 525 from a MIPI camera sensor; and / or an example second command conversion data structure 530 from a host device. In some examples, a MIPI camera sensor can refer to any camera, sensor, imaging device, module, etc., that supports MIPI communication. In some examples, a host device can be a smartphone, laptop computer, tablet computer, embedded system (e.g., those embedded systems used for automotive cameras, drones, or Internet of Things (IoT) devices), monitoring system, etc. The Unified MIPI Control Interface Driver 405 receives control and / or configuration commands from the host device via a software interface or application programming interface (API) 535. The unified MIPI control interface driver 405 maps commands to specific registers and sends settings to the MIPI camera sensor via the example mobile industry processor interface 540 (e.g., MIPI CSI).

[0035] Example Unified MIPI Control Interface Driver 405 is included in, or otherwise implemented or executed by, a host device (e.g., host devices 120, 220, and 315) to control various types of MIPI camera sensors from different camera sensor vendors. Different types of MIPI camera sensors may have different control commands uniquely defined by the camera sensor vendor. Unified MIPI Control Interface Driver 405 controls different types of MIPI camera sensors by using command translation data structures 525 and / or 530 to convert general (or unified) camera sensor control commands, parameters, etc., into specific control commands, parameters, etc., specified by each camera sensor vendor for its respective MIPI camera sensor.

[0036] In the illustrated example, after the unified MIPI control interface driver 405 is activated by the host device (e.g., after activation or loading by the host device's OS), the conversion data loader circuitry 505 of the MIPI control interface driver 405 accesses one or more of the following: command conversion data structure 525 from the memory / storage device of the MIPI camera sensor (e.g., MIPI camera sensors 115, 215, and 365) included in, coupled to, or otherwise associated with the host device, via hardware interface 545; and / or command conversion data structure 530 from the host device's memory / storage device via interface 550. Command conversion data structure 525 and / or command conversion data structure 530 map general camera sensor control commands, parameters, etc., to specific control commands, parameters, etc., used by a specific MIPI camera sensor. This enables plug-and-play functionality for MIPI camera sensors without requiring the integration of different camera control drivers with the host device each time a different camera type or MIPI camera sensor module is used with the host device.

[0037] In the illustrated example, the conversion data loader circuitry 505 of the Unified MIPI Control Interface driver 405 accesses (e.g., reads) one or both of the following: a command conversion data structure 525 from a MIPI camera sensor included in, coupled to, or otherwise associated with the host device; and / or a command conversion data structure 530 from the host device's own memory / storage device. For example, the command conversion data structure 525 may be stored (e.g., written, downloaded, etc.) to the memory / storage device of the MIPI camera sensor by the sensor vendor / manufacturer, and the command conversion data structure 530 may be automatically stored (e.g., written, downloaded, etc.) to the host device's memory / storage device by the device vendor / manufacturer or device user when the MIPI camera sensor, etc., is detected.

[0038] In some examples, for instance, after the conversion data loader circuitry 505 detects that the MIPI camera sensor has been coupled to the host device, the conversion data loader circuitry 505 performs an initial attempt to access the command conversion data structure 525 from the MIPI camera sensor. In some such examples, if the initial attempt to access the command conversion data structure 525 from the MIPI camera sensor fails, the conversion data loader circuitry 505 performs a subsequent attempt to access the command conversion data structure 530 from the host device's memory / storage device. In some examples, the command conversion data structures 525 and / or 530 provide information about: specific memory addresses (e.g., register addresses) of registers in the MIPI camera sensor; the purpose / usage of specific registers in the MIPI camera sensor; and specific register settings, such as specific values ​​that can be written to the corresponding registers to configure / control different parameters / settings of the MIPI camera sensor. In some examples, command translation data structures 525 and / or 530 also map a generic MIPI camera sensor command (also known as a unified MIPI camera sensor command) to one or more of the specific registers and register settings of the MIPI camera sensor. This generic MIPI camera sensor command can be universal (or uniformly applicable) to multiple sensor types / models, etc. For example, the generic camera sensor command could be a "Set White Balance" command, which instructs the MIPI camera sensor to adjust the white balance to ensure that colors in the captured image are rendered naturally under different lighting conditions. White balance adjustment compensates for color temperature variations such as tungsten lighting (indoor) or daylight (outdoor) to maintain accurate color reproduction. When the host device sends the "Set White Balance" command to the MIPI camera sensor, the MIPI camera driver translates this generic command into a write operation to one or more specific registers of the white balance control register. This register controls the white balance setting of the camera sensor. This register may have an example address (0x0203) and bit fields for the gain applied to the red and blue channels of the image. This bit field may be a 4-bit field representing the gain applied to the red and blue channels of the image. The values ​​can range from 0 to 15, with higher values ​​corresponding to increased red or blue gain. The camera sensor driver calculates the appropriate red and blue gain values ​​based on the desired white balance settings provided by the application (e.g., daylight, tungsten lamp). The camera sensor driver writes these gain values ​​to the corresponding bit fields in a register, thereby configuring the camera sensor to apply the specified white balance adjustment during image capture.

[0039] A similar mapping exists for other camera settings such as exposure time. The generic camera sensor command "Set Exposure Time" instructs the camera sensor to adjust the exposure time, which determines how long the sensor's pixels are exposed to light during image capture. Longer exposure times produce brighter images but may introduce motion blur, while shorter exposure times reduce motion blur but may produce darker images. When the host device sends the "Set Exposure Time" command to the camera sensor, the camera sensor driver interprets this generic command and translates it into a specific register write operation targeting the exposure control register. The exposure control register may have an example address (0x0102) and an example bit field for the exposure time value. This bit field may be an 8-bit field representing the desired exposure time setting. The exposure time value can range from 0 to 255, where each unit represents a specific exposure duration. For example, setting the value to 128 can represent an exposure time of one-tenth of a second. The camera sensor driver writes the desired exposure time value (e.g., 128) into the appropriate bit field within the register, thereby configuring the camera sensor to use the specified exposure time during image capture. Similar mappings exist for other camera settings such as gain, frame rate, and resolution.

[0040] In some examples, the Unified MIPI Control Interface driver 405 includes a unit for accessing command conversion data. For example, the unit for accessing command conversion data may be implemented by a conversion data loader circuit system 505. In some examples, the conversion data loader circuit system 505 may be implemented by, for example... Figure 10 Example programmable circuit systems such as programmable circuit system 1012 are instantiated. For example, conversion data loader circuit system 505 can be instantiated by... Figure 11 The example microprocessor 1100 executes machine-executable instructions (e.g., at least by...). Figure 9 The machine-executable instructions implemented in boxes 915 and 920 are used for instantiation. In some examples, the conversion data loader circuit system 505 can be instantiated by a hardware logic circuit system, which can be an ASIC, XPU, or... Figure 12The FPGA circuitry 1200 is implemented, and the ASIC, XPU, or FPGA circuitry 1200 is configured and / or constructed to perform operations corresponding to machine-readable instructions. Alternatively or additionally, the conversion data loader circuitry 505 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the conversion data loader circuitry 505 can be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured and / or constructed to execute some or all of the machine-readable instructions, and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, but other configurations are equally applicable.

[0041] Example Unified Command Interface Circuit System 510 receives generic commands from an external source (e.g., the host device's ISP or software application) to configure and / or control the MIPI camera sensor. Unified Command Interface Circuit System 510 receives generic commands via a software interface or API 535.

[0042] In some examples, the Unified MIPI Control Interface driver 405 includes a unit for obtaining commands. For example, the unit for obtaining commands may be implemented by the Unified Command Interface Circuit System 510. In some examples, the Unified Command Interface Circuit System 510 may be implemented by, for example... Figure 10 Example programmable circuit systems such as programmable circuit system 1012 are instantiated. For example, unified command interface circuit system 510 can be instantiated by... Figure 11 The example microprocessor 1100 executes machine-executable instructions (e.g., at least by...). Figure 9 The unified command interface circuit system 510 is instantiated by the machine-executable instructions implemented in box 930. In some examples, the unified command interface circuit system 510 can be instantiated by a hardware logic circuit system, which can be an ASIC, XPU, or... Figure 12The FPGA circuit system 1200 is implemented, and the ASIC, XPU, or FPGA circuit system 1200 is configured and / or constructed to perform operations corresponding to machine-readable instructions. Alternatively or additionally, the unified command interface circuit system 510 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the unified command interface circuit system 510 can be implemented by at least one or more hardware circuits (e.g., processor circuit systems, discrete and / or integrated analog and / or digital circuit systems, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured and / or constructed to perform some or all of the machine-readable instructions, and / or perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, but other configurations are equally applicable.

[0043] Example conversion data parser circuitry 515 parses command conversion data structures to map general commands to one or more specific commands (e.g., specific register settings) of a MIPI camera sensor. The general command specifies a camera device configuration that can be used across multiple different MIPI camera sensor types, models, etc. Conversion data parser circuitry 515 obtains the general command from a host device (e.g., a smartphone, laptop, or embedded system) via unified command interface circuitry 510. The received command is parsed to extract relevant information. This may involve interpreting the command code, parameters, and arguments to determine the desired action or configuration to be taken. Conversion data parser circuitry 515 locates the general command in one or more loaded data structures and retrieves one or more specific register settings of the camera sensor mapped to the general command in that data structure(s). Conversion data parser circuitry 515 sorts and / or formats the one or more specific register settings(s) and places them in a buffer or data structure in preparation for writing or transmitting commands to the MIPI camera sensor. The buffer or data structure may contain fields for each register address and its corresponding value, as well as additional information required to control and / or configure the camera sensor. The buffer or data structure includes a mapping between general commands (e.g., setting exposure, setting white balance, etc.) and the corresponding register addresses and values ​​required to achieve the desired effect and / or configuration on the camera sensor. The conversion data resolver circuitry 515 populates the desired configuration settings of the camera sensor into this data structure to write these settings to the camera sensor. Various registers and the settings within these registers define various parameters such as exposure, gain, white balance, and other image processing parameters. Once the settings are formatted into the data structure, the conversion data resolver circuitry 515 can organize the settings into a buffer for efficient transmission to the camera sensor. The buffer can be configured according to the communication protocol used for interfacing with the sensor (e.g., MIPI, CSI, I...). 2 Constructed in a way that matches (e.g., C).

[0044] In some examples, the Unified MIPI Control Interface driver 405 includes a unit for parsing command conversion data. For example, the unit for parsing command conversion data can be implemented by a conversion data parser circuit system 515. In some examples, the conversion data parser circuit system 515 can be implemented by, for example... Figure 10 Example programmable circuit systems such as programmable circuit system 1012 are instantiated. For example, conversion data parser circuit system 515 can be instantiated by... Figure 11 The example microprocessor 1100 executes machine-executable instructions (e.g., at least by...). Figure 9The machine-executable instructions implemented in box 925 are used to instantiate the data converter circuitry system 515. In some examples, the data converter circuitry system 515 can be instantiated by a hardware logic circuitry system, which can be an ASIC, XPU, or... Figure 12 The FPGA circuit system 1200 is implemented, and the ASIC, XPU, or FPGA circuit system 1200 is configured and / or constructed to perform operations corresponding to machine-readable instructions. Alternatively or additionally, the conversion data parser circuit system 515 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the conversion data parser circuit system 515 can be implemented by at least one or more hardware circuits (e.g., processor circuit systems, discrete and / or integrated analog and / or digital circuit systems, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured and / or constructed to perform some or all of the machine-readable instructions, and / or perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, but other configurations are equally applicable.

[0045] Example device-specific configuration circuitry 520 accesses one or more specific register settings determined and formatted or sorted by example conversion data parser circuitry 515, and causes these specific register settings to be written to or sent to the MIPI camera sensor (e.g., in an appropriate sequence, at an appropriate time, etc.). Device-specific configuration circuitry 520 obtains a buffer or data structure formatted by conversion data parser circuitry 515 and sends the buffer containing the formatted register settings to the MIPI camera sensor using an appropriate mobile industry processor interface 540. This involves transmitting the buffer via a physical connection to the camera sensor, which may be such as MIPI CSI, I... 2 C bus, I 3A serial bus such as the C-bus or any other control interface. Device-specific configuration circuitry 520 sends multiple commands. These commands include corresponding sets of one or more register settings, including a first set for configuring one or more register settings for a camera sensor. The corresponding commands specify different camera or sensor configurations applicable to multiple different camera or image sensor types. The commands include an ordered sequence of register settings transmitted via the mobile industry processor interface 540. The commands may relate to a specific bit pattern or ordered sequence specific to the MIPI camera sensor. This sequence may involve writing to multiple registers in a specific order. The sent commands include one or more register settings, which may include a camera register address and a register value for setting the camera sensor register identified by that register address. In some examples, the one or more register settings include an ordered sequence of register settings. The ordered sequence of register settings includes an ordered sequence of register addresses and corresponding register values ​​for setting one or more camera registers identified by those register addresses.

[0046] After the settings are sent to the camera sensor, the device-specific configuration circuitry 520 can verify that the camera sensor has correctly received and applied the settings. This verification process ensures that the camera sensor is configured as expected and that there are no communication errors related to the settings.

[0047] In some examples, the Unified MIPI Control Interface driver 405 includes a unit for configuring the camera based on one or more register settings in a command. For example, the configuration unit may be implemented by a device-specific configuration circuitry 520. In some examples, the device-specific configuration circuitry 520 may be implemented by, for example... Figure 10 Example programmable circuit systems such as programmable circuit system 1012 are instantiated. For example, device-specific configuration circuit system 520 can be instantiated by... Figure 11 The example microprocessor 1100 executes machine-executable instructions (e.g., at least by...). Figure 9 The device-specific configuration circuitry 520 can be instantiated using machine-executable instructions implemented in box 930. In some examples, the device-specific configuration circuitry 520 can be instantiated by a hardware logic circuitry system, which can be an ASIC, XPU, or... Figure 12The FPGA circuit system 1200 is implemented, and the ASIC, XPU, or FPGA circuit system 1200 is configured and / or constructed to perform operations corresponding to machine-readable instructions. Alternatively or additionally, the device-specific configuration circuit system 520 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the device-specific configuration circuit system 520 can be implemented by at least one or more hardware circuits (e.g., processor circuit systems, discrete and / or integrated analog and / or digital circuit systems, FPGAs, ASICs, XPUs, comparators, operational amplifiers, logic circuits, etc.) configured and / or constructed to perform some or all of the machine-readable instructions, and / or perform some or all of the operations corresponding to the machine-readable instructions without executing software or firmware, but other configurations are equally applicable.

[0048] The first command conversion data structure 525 can be any type of data structure, such as a data file, data record, array, linked list, table, etc., which can store command conversion data in the memory / storage device of the MIPI camera sensor. In some examples, the command conversion data structure 525 can be stored in the electrically erasable programmable read-only memory (EEPROM) of the MIPI camera sensor, thereby allowing the camera supplier / manufacturer to write, revise, update, etc., to the command conversion data structure 525, and automatically load the command conversion data structure 525 by the conversion data loader circuitry 505 of the unified MIPI control interface driver 405 for plug-and-play operation.

[0049] In some examples, a given MIPI camera sensor has a control register with a unique set of register settings defined by the respective camera sensor vendor. A given MIPI camera sensor also implements a camera control algorithm to control various aspects of camera behavior and image quality. For example, such a camera control algorithm is designed to adjust camera parameters and / or settings to optimize image capture under various conditions. The camera control algorithm can be vendor-specific.

[0050] The conversion data parser circuitry 515 translates generic commands used to specify configurations / settings applicable to various MIPI camera sensor types and models into specific parameters defined by the camera sensor vendor to implement those configurations / settings in their specific MIPI camera sensor. In some examples, the generic commands may specify static and / or dynamic parameters. Static parameters are settings or characteristics of the camera sensor that remain constant or change very little during normal operation. Static parameters may include parameters such as resolution, pixel size, sensor type, lens characteristics (e.g., fixed focal length), format, streaming on or off, etc. Dynamic parameters are settings or characteristics that change frequently during operation or are dynamically adjusted to adapt to changing conditions. Dynamic parameters may include parameters such as exposure time, gain, white balance, focus, frame rate, noise reduction, etc. A given MIPI camera sensor may have its own unique configuration parameters (e.g., a set of register definitions) specified by its camera sensor vendor, which differ from the configuration parameters of other MIPI camera sensors from other vendors. The specific parameters available and their adjustment ranges may vary depending on the camera model and manufacturer.

[0051] The conversion data parser circuitry 515 uses a command conversion data structure 525 to convert general camera commands into specific parameters for controlling various settings for different camera sensor types. By using the command conversion data structure 525, the device-specific configuration circuitry 520 can configure various MIPI camera sensors without developing new drivers each time a different camera sensor type is used. For example, general commands such as "set exposure time" and "adjust white balance" can be mapped to specific register settings within the camera sensor. This mapping enables the device-specific configuration circuitry 520 to configure camera sensor parameters based on the command.

[0052] The second command translation data structure 530 from the host device can be any type of data structure, such as a data file, data record, array, linked list, table, etc., which can store the command translation data in the memory / storage device of the host device coupled to the MIPI camera sensor. The second command translation data structure 530 is similar to or identical to the first command translation data structure 525, but is stored in the host device rather than in the MIPI camera sensor. For example, the command translation data structure can be stored in a Binary Input / Output (BIOS) Advanced Configuration and Power Interface (ACPI) Differential System Description Table (DSDT), allowing the command translation data structure to be automatically loaded by the Unified MIPI Control Interface Driver 405 for plug-and-play operation.

[0053] although Figure 5 Implementation shown Figures 1 to 4The example method of the Unified MIPI Control Interface Driver 405, but Figure 5 One or more of the elements, processes, and / or devices shown may be combined, divided, rearranged, omitted, removed, and / or implemented in any other way. Furthermore, the example conversion data loader circuit system 505, the example unified command interface circuit system 520, the example conversion data parser circuit system 515, and the example device-specific configuration circuit system 520, and / or more generally, Figure 5 The example Unified MIPI Control Interface driver 405 can be implemented solely by hardware, or by a combination of hardware and software and / or firmware. Therefore, for example, the example Conversion Data Loader Circuitry 505, the example Unified Command Interface Circuitry 520, the example Conversion Data Parser Circuitry 515, and the example Device-Specific Configuration Circuitry 520, and / or more generally, any of the example Unified MIPI Control Interface driver 405 can be implemented by: programmable circuitry systems, processor circuitry systems, analog circuitry, digital circuitry, logic circuitry, programmable processors, programmable microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), ASICs, programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs) such as FPGAs. Furthermore, Figure 5 Example of a unified MIPI control interface driver 405 may include, except Figure 5 Other than or alternative to the components, processes and / or equipment shown. Figure 5 The elements, processes, and / or devices shown may include one or more of any or all of the elements, processes, and devices shown.

[0054] Figure 9 A flowchart illustrating example machine-readable instructions and / or example operations is shown, wherein the example machine-readable instructions can be executed by a programmable circuit system to implement and / or instantiate... Figure 5 The unified MIPI control interface driver 405, and this example operation can be implemented by a programmable circuit system to achieve and / or instantiate. Figure 5 The Unified MIPI Control Interface driver 405. Machine-readable instructions can be provided to programmable circuit systems (e.g., in conjunction with the following). Figure 10The programmable circuit system 1012 shown in the example processor platform 1000 discussed in this study executes one or more executable programs or one or more portions of one or more executable programs, and / or the machine-readable instructions may be those that are combined with the following Figure 11 and / or Figure 12 The examples discussed are programmable circuit systems (e.g., FPGAs) implementing one or more functions or portions of functions. In some examples, machine-readable instructions enable operations, tasks, etc., to be implemented and / or performed in a real-world manner automatically. As used herein, “automation” means without human intervention.

[0055] The program may be embodied as instructions (e.g., software and / or firmware) stored on one or more non-transitory computer-readable and / or machine-readable storage media, such as cache memory, magnetic storage devices or disks (e.g., floppy disks, hard disk drives (HDDs), etc.), optical storage devices or optical discs (e.g., Blu-ray discs, compact discs (CDs), digital versatile discs (DVDs), etc.), redundant arrays of independent disks (RAID), registers, ROM, solid-state drives (SSDs), SSD memory, non-volatile memory (e.g., electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), volatile memory (e.g., random access memory (RAM) of any type), and / or any other storage device or disk. The instructions on the non-transitory computer-readable and / or machine-readable media may be programmed and / or executed by a programmable circuit system located in one or more hardware devices, but the entire program and / or portions thereof may alternatively be executed and / or instantiated by one or more hardware devices other than the programmable circuit system, and / or embodied in dedicated hardware. Machine-readable instructions can be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., server and client hardware devices). For example, client hardware devices can be implemented by endpoint client hardware devices (e.g., hardware devices associated with human users and / or machine users) or intermediate client hardware device gateways (e.g., radio access networks (RAN)) that facilitate communication between the server and endpoint client hardware devices. Similarly, non-transitory computer-readable storage media can include one or more media. Furthermore, although references... Figure 9The flowchart(s) shown depict example programs, but many other methods for implementing the example unified MIPI control interface driver 405 may be used instead. For example, the execution order of the blocks(s) in the flowchart(s) may be changed, and / or some blocks in the described blocks may be changed, removed, or combined. Additionally or alternatively, any or all of the blocks in the flowchart may be implemented by one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, comparator, operational amplifier, logic circuitry, etc.) configured to perform the corresponding operations without executing software or firmware. The programmable circuitry may be distributed across different network locations and / or located locally on one or more hardware devices (e.g., a single-core processor (e.g., a single-core CPU), a multi-core processor (e.g., a multi-core CPU, XPU, etc.)). For example, a programmable circuit system may be a CPU and / or FPGA located in the same package (e.g., the same integrated circuit (IC) package) or in two or more separate housings, one or more processors in a single machine, multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server racks, and / or any combination thereof.

[0056] The machine-readable instructions described herein may be stored in one or more formats, including compressed formats, encrypted formats, segmented formats, compiled formats, executable formats, and packaged formats. The machine-readable instructions described herein may be stored as data (e.g., computer-readable data, machine-readable data, one or more bits (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), bit streams (e.g., computer-readable bit streams, machine-readable bit streams, etc.)) or data structures (e.g., as part(s) of instructions, code, code representations, etc.), which can be used to create, manufacture, and / or generate machine-executable instructions. For example, machine-readable instructions may be segmented and stored on one or more storage devices, disks, and / or computing devices (e.g., servers) located at the same or different locations within a network or network set (e.g., in the cloud, edge devices, etc.). Machine-readable instructions may require one or more of the following to be installed, modified, adapted, updated, combined, supplemented, configured, decrypted, decompressed, unpacked, distributed, reassigned, compiled, etc., so that they can be directly read, interpreted, and / or executed by computing devices and / or other machines. For example, machine-readable instructions can be stored in multiple parts, each of which is compressed, encrypted, and / or stored on a separate computing device, wherein when these parts are decrypted, decompressed, and / or combined, they form a set of computer-executable and / or machine-executable instructions that implement one or more functions and / or operations, which can together form a program, such as that described herein.

[0057] In another example, machine-readable instructions may be stored in a state that can be read by a programmable circuit system but requires the addition of libraries (e.g., dynamic link libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc., to be executed on a specific computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., storing settings, input data, recording network addresses, etc.) before they can be fully or partially executed. Therefore, as used herein, machine-readable media, computer-readable media, and / or machine-readable media may include instructions and / or (one or more) programs, regardless of their specific format or state.

[0058] The machine-readable instructions described in this article can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, Hypertext Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0059] As mentioned above, Figure 9 Example operations can be implemented using executable instructions (e.g., computer-readable instructions and / or machine-readable instructions) stored on one or more non-transitory computer-readable and / or machine-readable media. As used herein, the terms "non-transitory computer-readable medium," "non-transitory computer-readable storage medium," "non-transitory machine-readable medium," and / or "non-transitory machine-readable storage medium" are explicitly defined to include any type of computer-readable storage device and / or disk, excluding propagation signals and transmission media. Examples of such non-transitory computer-readable media, non-transitory computer-readable storage media, non-transitory machine-readable media, and / or non-transitory machine-readable storage media include optical storage devices, magnetic storage devices, HDDs, flash memory, read-only memory (ROM), CDs, DVDs, caches, any type of RAM, registers, and / or any other storage device or disk that stores information for any duration (e.g., for an extended period, permanently, temporarily, for temporary buffering, and / or for caching information). As used herein, the terms “non-transitory computer-readable storage device” and “non-transitory machine-readable storage device” are defined as including any physical (mechanical, magnetic, and / or electrical) hardware used to retain information for a period of time, excluding propagation signals and transmission media. Examples of non-transitory computer-readable storage devices and / or non-transitory machine-readable storage devices include any type of random access memory, any type of read-only memory, solid-state memory, flash memory, optical discs, magnetic disks, disk drives, and / or redundant arrays of independent disks (RAID) systems. As used herein, the term “device” refers to a physical structure such as mechanical and / or electrical equipment, hardware, and / or circuitry that may or may not be configured by computer-readable instructions, machine-readable instructions, etc., and / or is manufactured to execute computer-readable instructions, machine-readable instructions, etc.

[0060] Figure 6Example processing flow 600 can be implemented by the Unified MIPI Control Interface Driver 405. The Unified MIPI Control Interface Driver 405, in conjunction with command translation data structures 525 or 530, implements a camera sensor framework. This camera sensor framework is OS-independent. The framework can be integrated with various OSs without being coupled to any specific OS. The camera sensor framework is hardware platform-independent and can be used across different hardware platforms without requiring significant modifications. The camera sensor framework provides a standardized interface for interacting with the camera sensor, independent of the underlying hardware architecture. Example processing flow 600 illustrates the operation of the Unified MIPI Control Interface Driver 405 controlling the configuration / setting of an example MIPI camera sensor 610. For example, MIPI camera sensor 610 can correspond to any of the MIPI camera sensors 115, 215, and 365 described above. The example Unified Command Interface Circuitry 510 of the Unified MIPI Control Interface Driver 405 receives information about general commands from an OS service or application on the host device, where the OS service or application performs initial tuning or runtime sensor control on the MIPI camera sensor (box 615). For example, the host device may correspond to any of the host devices 120, 220, and 315 described above. The general command list includes general sensor parameters (e.g., logic parameters) for controlling and / or configuring the MIPI camera sensor (box 620). The conversion data parser circuitry 515 converts the general sensor parameters into device-specific (e.g., camera sensor) register values ​​(box 625). The conversion data parser circuitry 515 uses command conversion data structures 525 or 530 to map general commands to camera sensor-specific registers.

[0061] The conversion data loader circuitry 505 loads the command conversion data structure during OS boot and temporarily stores it as a local command conversion data structure 675. Command conversion data structure 525 can be stored in the non-volatile memory (NVM) of a specific camera sensor, or command conversion data structure 530 can be stored in the host device's BIOS ACPI DSDT table 670 and / or other host-accessible storage devices. The conversion data loader circuitry 505 can read and load command conversion data structure 525 (box 630) from the NVM 525 of the camera sensor, or it can load command conversion data structure 530 (box 635) from the host device's BIOS ACPI DSDT table 670. Command conversion data structure 525 or 530 is loaded as the local command conversion data structure 675. Figure 6In the context of the MIPI camera sensor 610, the conversion data loader circuitry 505 stores the contents of command conversion data structures 525 and 530 in local storage (e.g., local RAM). Local command conversion data structure 675 is a data structure used to map or convert general commands or parameters to specific settings or configurations for hardware components (e.g., camera sensors). Local command conversion data structure 675 may contain mappings between general commands or parameters (e.g., “set exposure time,” “adjust white balance,” etc.) and corresponding register values, commands, or configurations understood by the MIPI camera sensor 610. Command conversion data structures 525 or 530 include mappings of general commands to specific camera sensor registers for different camera sensor types. In some examples, the camera sensor register mapping is for a first type of camera sensor (e.g., IMxxx 645) with example sensor registers and a second type of camera sensor sensor (e.g., OVxxx 650) with second type camera sensor registers.

[0062] The conversion data parser circuitry 515 sorts and / or formats one or more specific register settings and places them in a buffer or data structure in preparation for writing or transmitting commands to the MIPI camera sensor (box 640). This buffer or data structure may contain fields for each register address and its corresponding value, as well as additional information required to control and / or configure the camera sensor. The buffer or data structure includes a mapping between general commands (e.g., setting exposure, setting white balance, etc.) and the corresponding register addresses and values ​​required to achieve the desired effect and / or configuration on the MIPI camera sensor 610.

[0063] Device-specific configuration circuitry 520 enables one or more specific register settings to be provided to (e.g., written to) an input / output (I / O) backend (box 645). These specific register settings are written to or sent to the MIPI camera sensor (e.g., in the appropriate order, at the appropriate time, etc.) via hardware interface 665 (box 680). In some examples, the I / O backend may be MIPI CSI, I... 2 C bus, I 3 C-bus, USB, Ethernet, etc. Device-specific configuration circuitry 520 enables the MIPI camera sensor 610 to be configured based on this command.

[0064] Figure 7Example operation 700 of the unified MIPI camera driver in a usage environment is illustrated. Example operation 700 includes an example unified MIPI control interface driver 405 combined with command translation data structures 525 or 530 to implement a camera sensor framework independent of the upper-level OS 715 or camera sensor 710. For example, the MIPI camera sensor 710 may correspond to any of the MIPI camera sensors 115, 215, 365, and 610 described above. The unified MIPI control interface driver 405 configures and controls different camera sensors 710 in different operating systems (OS) 715. Typically, camera drivers are OS-specific; however, the unified MIPI control interface driver 405 is OS-independent and can be implemented on any OS. The unified MIPI control interface driver 405 facilitates communication between the OS 715 and the camera sensor 710.

[0065] The example OS 715 includes an example Camera Hardware Abstraction Layer (HAL) 745 and an example ACPI 750. The Camera HAL 745 abstracts the low-level details of the camera hardware and provides a standardized interface for upper-level software components, including camera applications and the OS. The Camera HAL 745 decouples the specific details of the camera hardware from the rest of the system, allowing different camera sensors to be seamlessly integrated with the same camera application. The Camera HAL 745 defines a set of application programming interfaces (APIs) that applications can use to interact with the camera sensor 710. These APIs include functions for adjusting camera parameters, managing camera functions, capturing images, and more.

[0066] Example ACPI 750 manages the power states of the entire system 700. ACPI 750 enables OS 715 to put devices such as camera sensor 710 into a low-power state when they are not in use, thus contributing to improved overall energy efficiency. ACPI 750 helps OS 715 identify and configure camera sensor 710 during the boot process. This involves discovering and initializing camera sensor 710, and making it available to the system.

[0067] The Unified MIPI Control Interface driver 405 includes a sample command translation data loader 505, a sample unified command interface 510, a sample command translation data parser 515, and a sample device-specific configurator 520.

[0068] The camera HAL 745 controls the operation of the camera sensor 710 by sending commands such as exposure, white balance, focus, resolution, and frame rate to the Unified Command Interface 510. The Unified MIPI Control Interface driver 405 translates high-level commands from the camera application into low-level operations understood by the camera sensor 710. The Unified Command Interface 510 receives general commands from the camera HAL 745.

[0069] Command translation data loader 505 loads command translation data structures. Command translation data structure 525 can be stored in the camera sensor EEPROM 735, or command translation data structure 530 can be stored in the BIOS ACPI or flash RAM 740. Data structure 525 or 530 is an offline table generated by the camera sensor vendor or ISP vendor, which describes specific parameters of the camera sensor and their register settings. Command translation data structure 525 or 530 is used to translate or map general commands from the camera HAL 745 to specific register settings of the camera sensor, which are presented as a sequence of physical register value settings.

[0070] Example command conversion data parser 515 parses one or more command conversion data structures 525 or 530. Command conversion data parser 515 obtains a general command from unified command interface 510 and looks up the general command in one or more command conversion data structures 525 or 530. Command conversion data parser 515 retrieves one or more camera sensor-specific register settings mapped to the general command in one or more command conversion data structures 525 or 530. Conversion data parser circuitry 515 sorts and / or formats the one or more specific register settings and places them in a buffer or data structure in preparation for writing or transmitting commands to MIPI camera sensor 710. The buffer or data structure may contain fields for each register address and its corresponding value.

[0071] The example device-specific configurator 520 accesses one or more specific register settings formatted or sorted by the example conversion data parser 515, and causes these specific register settings to be written to or transferred to the sensor 730 (e.g., in the appropriate order, at the appropriate time, etc.). These register settings include various register addresses and register values ​​that define various parameters such as exposure, gain, white balance, and other image processing parameters.

[0072] Figure 8An example process for generating / creating command translation data structures 525 and / or 530 is illustrated. In some examples, the system-on-chip (SoC) vendor 805 generates the command translation data structure (box 810). Command translation data structure 530 is integrated into the BIOS or ACPI DSDT 815. Alternatively and / or alternatively, the camera sensor vendor 820 generates the command translation data structure (box 810). The sensor vendor 820 stores the command translation data structure 525 as metadata in the camera NVM 825 (e.g., EEPROM or flash memory). The Unified MIPI Control Interface driver 405 probes the camera sensor to read metadata from the camera sensor's NVM 825 and parses the metadata content. If the metadata includes command translation data structure 525, the Unified MIPI Control Interface driver 405 loads the command translation data structure 525 into its local driver. The Unified MIPI Control Interface driver 405 uses the command translation data structure 525 to map general commands to MIPI camera sensor-specific register settings. The register(s) set configuration and control of various MIPI camera sensors from various camera sensor vendors. If the metadata does not include command translation data structure 525, the unified MIPI control interface driver 405 queries the command translation data structure 530 in the BIOS ACPI DSDT table 815 and loads the command translation data structure 530 into the local RAM of the unified MIPI control interface driver 405.

[0073] Figure 9 This is a flowchart representing example machine-readable instructions and / or example operations 900, which can be executed, instantiated, and / or implemented by a programmable circuit system to configure a MIPI camera sensor. Figure 9Example machine-readable instructions and / or example operations 900 begin at box 905, where the host device activates the unified MIPI camera driver 400. For example, the unified MIPI camera driver 400 may be activated when the host device's OS loads the unified MIPI camera driver 400. At box 910, the unified command interface circuitry 510 obtains general configuration and control commands from an external source (e.g., an application on the host device). These commands specify MIPI camera sensor configurations applicable to different camera sensor types. At box 915, the conversion data loader circuitry 505 attempts to access or load a command conversion data structure 525 from the memory of the MIPI camera sensor. If access / loading of the command conversion data structure from the MIPI camera is unsuccessful (box 915: No), the conversion data loader circuitry 505 accesses or loads a conversion data structure 530 from the host device's memory (box 920), and control is transferred to box 925. If the conversion data loader circuitry 505 successfully loads the command conversion data structure 525 (box 915: Yes) from the memory of the MIPI camera, then control also moves to box 925.

[0074] At block 925, the conversion data parser circuitry 515 converts the data based on the command, mapping the general command obtained at block 910 to one or more specific register settings of the MIPI camera sensor. These register settings are specific to the camera sensor. These specific register settings are used to configure the MIPI camera sensor based on the general command.

[0075] At block 930, device-specific configuration circuitry 520 sends one or more register settings to the camera sensor via mobile industrial processor interface 540. Device-specific configuration circuitry 520 enables the camera sensor to be configured based on commands. Figure 9 The example instructions and / or operations 900 end here.

[0076] Figure 10 This is a block diagram of an example programmable circuit system platform 1000, which is configured to perform and / or instantiate... Figure 9 Example machine-readable instructions and / or example operations to implement Figure 5The unified MIPI control interface driver 405. The programmable circuit system platform 1000 may be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, tablet computer such as iPad™), personal digital assistant (PDA), internet device, DVD player, CD player, digital video recorder, Blu-ray player, game console, personal video recorder, set-top box, head-mounted device (e.g., augmented reality (AR) head-mounted device, virtual reality (VR) head-mounted device, etc.) or other wearable device, vehicle camera, or any other type of computing and / or electronic device.

[0077] The illustrated programmable circuit system platform 1000 includes a programmable circuit system 1012. The illustrated programmable circuit system 1012 is hardware. For example, the programmable circuit system 1012 can be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. The programmable circuit system 1012 can be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, the programmable circuit system 1012 implements an example converted data loader circuit system 505, an example converted data parser circuit system 515, an example unified command interface circuit system 510, and an example device-specific configuration circuit system 520.

[0078] The programmable circuit system 1012 illustrated includes local memory 1013 (e.g., cache, registers, etc.). The programmable circuit system 1012 of the illustrated example communicates with main memories 1014, 1016 via bus 1018, which include volatile memory 1014 and non-volatile memory 1016. Volatile memory 1014 may be implemented using synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of RAM device. Non-volatile memory 1016 may be implemented using flash memory and / or any other desired type of memory device. Access to the main memories 1014, 1016 of the illustrated example is controlled by a memory controller 1017. In some examples, the memory controller 1017 may be implemented using one or more integrated circuits, logic circuits, microcontrollers from any desired family or manufacturer, or any other type of circuit system for managing the flow of data into and out of the main memories 1014, 1016. exist Figure 10 In the example, the second example command transforms the data structure 530, which can be stored in non-volatile memory 1016.

[0079] The programmable circuit system platform 1000 shown in the example also includes an interface circuit system 1020. The interface circuit system 1020 can be implemented in hardware according to any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB) interface, a Bluetooth® interface, a Near Field Communication (NFC) interface, a Peripheral Component Interconnect (PCI) interface, and / or a Fast Peripheral Component Interconnect (PCIe) interface.

[0080] In the illustrated example, one or more input devices 1022 are connected to the interface circuit system 1020. The input devices 1022 allow a user (e.g., a human user, machine user, etc.) to type data and / or commands into the programmable circuit system 1012. The input devices 1022 may be implemented, for example, by an audio sensor, microphone, camera (still image or video), keyboard, buttons, mouse, touchscreen, touchpad, trackball, pointing device, and / or voice recognition system.

[0081] One or more output devices 1024 are also connected to the interface circuitry 1020 of the illustrated example. The output devices 1024 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube (CRT) displays, in-situ switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or speakers. Therefore, the interface circuitry 1020 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics processor circuitry such as a GPU.

[0082] The interface circuit system 1020 shown in the example also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces, to facilitate data exchange with external machines (e.g., any kind of computing device) via network 1026. For example, communication can be conducted via Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, beyond-line-of-sight wireless systems, line-of-sight wireless systems, cellular telephone systems, optical connections, etc. Figure 10 In the example, the interface circuit system 1020 implements the mobile industry processor interfaces 540 and 545.

[0083] The programmable circuit system platform 1000 illustrated also includes one or more mass storage disks or devices 1028 for storing firmware, software, and / or data. Examples of such mass storage disks or devices 1028 include magnetic storage devices (e.g., floppy disks, drives, HDDs, etc.), optical storage devices (e.g., Blu-ray discs, CDs, DVDs, etc.), RAID systems, and / or solid-state storage disks or devices such as flash memory devices and / or SSDs.

[0084] It can be by Figure 9 The machine-readable instructions 1032 implemented by the machine-readable instructions can be stored in a mass storage device 1028, a volatile memory 1014, a non-volatile memory 1016 and / or at least one non-transitory computer-readable storage medium (e.g., a removable CD or DVD).

[0085] Figure 11 yes Figure 10 A block diagram illustrating an example implementation of the programmable circuit system 1012. In this example, Figure 10 The programmable circuit system 1012 is implemented by the microprocessor 1100. For example, the microprocessor 1100 may be a general-purpose microprocessor (e.g., a general-purpose microprocessor circuit system). The microprocessor 1100 executes... Figure 9 The flowchart contains some or all of the machine-readable instructions to effectively translate... Figure 2 The circuit system is instantiated as logic circuitry for performing operations corresponding to these machine-readable instructions. In some such examples, Figure 5 The circuit system is instantiated by combining the hardware circuitry of the microprocessor 1100 with machine-readable instructions. For example, the microprocessor 1100 can be implemented by a multi-core hardware circuitry system such as a CPU, DSP, GPU, XPU, etc. Although the microprocessor 1100 may include any number of example cores 1102 (e.g., one core), the microprocessor 1100 of this example is a multi-core semiconductor device including N cores. The cores 1102 of the microprocessor 1100 can operate independently or can execute machine-readable instructions collaboratively. For example, the machine code corresponding to the firmware, embedded software program, or software program can be executed by one core of the core 1102, or can be executed by multiple cores of the core 1102 at the same time or at different times. In some examples, the machine code corresponding to the firmware, embedded software program, or software program is split into multiple threads and executed in parallel by two or more cores of the core 1102. The software program may correspond to Figure 9 The flowchart represents part or all of the machine-readable instructions and / or operations.

[0086] Core 1102 can communicate via a first example bus 1104. In some examples, the first bus 1104 can be implemented as a communication bus to enable communication associated with one or more cores in core 1102. For example, the first bus 1104 can be provided by an inter-integrated circuit (IIC) bus. 2 C) The first bus 1104 may be implemented using at least one of a bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Alternatively or alternatively, the first bus 1104 may be implemented using any other type of computing bus or electrical bus. The core 1102 may obtain data, instructions, and / or signals from one or more external devices via the example interface circuitry 1106. The core 1102 may output data, instructions, and / or signals to the one or more external devices via the interface circuitry 1106. Although the core 1102 of this example includes an example local memory 1120 (e.g., a Level 1 (L1) cache that may be partitioned into an L1 data cache and an L1 instruction cache), the microprocessor 1100 also includes an example shared memory 1110 (e.g., a Level 2 (L2) cache) that may be shared by the core for high-speed access to data and / or instructions. Data and / or instructions may be transferred (e.g., shared) by writing to and / or reading from the shared memory 1110. The local memory 1120 and shared memory 1110 of each core in core 1102 may include multi-level cache memory and main memory (e.g., Figure 10 It is part of the storage device hierarchy of main memories (1014, 1016). Typically, higher-level memories in this hierarchy exhibit shorter access times and have smaller storage capacities than lower-level memories. Changes in the various levels of the cache hierarchy are managed by cache coherence policies (e.g., reconciliation).

[0087] Each core 1102 may be referred to as a CPU, DSP, GPU, or any other type of hardware circuitry. Each core 1102 includes a control unit circuitry 1114, an arithmetic and logic (AL) circuitry (sometimes called an ALU) 1116, multiple registers 1118, local memory 1120, and a second example bus 1122. Other structures may also be present. For example, each core 1102 may include a vector unit circuitry, a single instruction multiple data (SIMD) unit circuitry, a load / store unit (LSU) circuitry, a branch / jump unit circuitry, a floating-point unit (FPU) circuitry, etc. The control unit circuitry 1114 includes semiconductor-based circuitry configured to control (e.g., coordinate) the movement of data within the corresponding core 1102. The AL circuitry 1116 includes semiconductor-based circuitry configured to perform one or more mathematical and / or logical operations on the data within the corresponding core 1102. Some examples of the AL circuitry 1116 perform integer-based operations. In other examples, the AL circuitry 1116 also performs floating-point operations. In other examples, the AL circuit system 1116 may include a first AL circuit that performs integer-based operations and a second AL circuit that performs floating-point operations. In some examples, the AL circuit system 1116 may be referred to as an arithmetic logic unit (ALU).

[0088] Register 1118 is a semiconductor-based structure for storing data and / or instructions, such as the results of one or more operations performed by the AL circuit system 1116 corresponding to core 1102. For example, register 1118 may include one or more vector registers, one or more SIMD registers, one or more general-purpose registers, one or more flag registers, one or more segment registers, one or more machine-specific registers, one or more instruction pointer registers, one or more control registers, one or more debug registers, one or more memory management registers, one or more machine check registers, etc. Register 1118 may be as follows: Figure 11 The diagram shows an arrangement of memory banks. Alternatively, registers 1118 can be organized in any other arrangement, format, or structure, for example, distributed throughout the core 1102 to reduce access time. The second bus 1122 can be connected via I... 2 It can be implemented using at least one of the following: C bus, SPI bus, PCI bus, or PCIe bus.

[0089] Each core 1102 and / or more generally, the microprocessor 1100 may include additional and / or alternative structures in addition to those shown and described above. For example, one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more converged / common grid stopping points (CMS), one or more shifters (e.g., barrel shifters), and / or other circuitry may be present. The microprocessor 1100 is a semiconductor device comprising a plurality of transistors interconnected to implement the above-described structures in one or more integrated circuits (ICs) contained in one or more packages.

[0090] Microprocessor 1100 may include one or more accelerators (e.g., acceleration circuitry, hardware accelerators, etc.) and / or cooperate with them. In some examples, accelerators are implemented by logic circuitry to perform specific tasks faster and / or more efficiently than a general-purpose processor. Examples of accelerators include, for example, ASICs and FPGAs as described herein. GPUs, DSPs, and / or other programmable devices may also be accelerators. Accelerators may be located on an onboard component of microprocessor 1100, in the same chip package as microprocessor 1100, and / or in one or more separate packages from microprocessor 1100.

[0091] Figure 12 yes Figure 10 A block diagram illustrating another example implementation of the programmable circuit system 1012. In this example, the programmable circuit system 1012 is implemented by an FPGA circuit system 1200. For example, the FPGA circuit system 1200 can be implemented by an FPGA. The FPGA circuit system 1200 can, for example, be used to execute instructions that would otherwise be readable by the corresponding machine. Figure 11 The example microprocessor 1100 performs the operations. However, once configured, the FPGA circuit system 1200 instantiates operations and / or functions in hardware corresponding to machine-readable instructions, and therefore can typically execute these operations / functions faster than a general-purpose microprocessor implementing the corresponding software.

[0092] More specifically, with the above Figure 11 The microprocessor 1100 (which can be programmed to execute) Figure 9 The flowchart represents some or all of the machine-readable instructions, but the interconnection and logic circuitry of a general-purpose device remain fixed once manufactured. Figure 12 The example FPGA circuit system 1200 includes interconnect and logic circuit systems that can be configured, constructed, programmed, and / or interconnected in different ways after manufacturing to instantiate, for example, with Figure 9The flowchart represents some or all of the machine-readable instructions corresponding to the operations / functions. Specifically, the FPGA circuit system 1200 can be viewed as an array of logic gates, interconnects, and switches. Switches can be programmed to change the way logic gates are interconnected, thereby effectively forming one or more dedicated logic circuits unless the FPGA circuit system 1200 is reprogrammed, or until the FPGA circuit system 1200 is reprogrammed. The configured logic circuits enable logic gates to cooperate in different ways to perform different operations on the data received by the input circuit system. These operations can correspond to... Figure 9 The flowchart represents some or all of the instructions (e.g., software and / or firmware). Therefore, the FPGA circuit system 1200 can be configured and / or constructed to effectively connect with... Figure 9 Some or all of the machine-readable instructions in the flowchart are instantiated as dedicated logic circuits to execute the operations / functions corresponding to these software instructions in a dedicated manner similar to that of an ASIC. Therefore, the FPGA circuit system 1200 can execute operations / functions corresponding to these software instructions faster than a general-purpose microprocessor. Figure 9 Some or all of the corresponding operations / functions in the machine-readable instructions.

[0093] exist Figure 12 In some examples, the FPGA circuit system 1200 is configured and / or constructed in response to being programmed (and / or reprogrammed once or multiple times) based on a binary file. In some examples, the binary file can be compiled and / or generated based on instructions in a hardware description language (HDL), such as Lucid, the VHSIC Hardware Description Language (VHDL), or Verilog. For example, a user (e.g., a human user, a machine user, etc.) can write code or programs in HDL corresponding to one or more operations / functions; this code / program can be converted to a low-level language as needed; and this code / program (e.g., code / program in a low-level language) can be converted (e.g., by a compiler, software application, etc.) into a binary file. In some examples, Figure 12 The FPGA circuit system 1200 can access and / or load the binary file to enable Figure 12 The FPGA circuit system 1200 is configured and / or constructed to perform one or more operations / functions. For example, a binary file can be generated by... Figure 12 The FPGA circuit system 1200 is implemented using accessible bit streams (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions to enable... Figure 12 The FPGA circuit system 1200 or one or more of its components are configured and / or constructed.

[0094] In some examples, binary files are compiled, generated, transformed, and / or otherwise output from a unified software platform used for programming the FPGA. For instance, the unified software platform can translate first instructions (e.g., code or program) corresponding to one or more operations / functions using a high-level language (e.g., C, C++, Python, etc.) into second instructions corresponding to one or more operations / functions using HDL. In some such examples, binary files are compiled, generated, and / or otherwise output from the unified software platform based on the second instructions. In some examples, Figure 12 The FPGA circuit system 1200 can access and / or load the binary file to enable Figure 12 The FPGA circuit system 1200 is configured and / or constructed to perform one or more operations / functions. For example, a binary file can be generated by... Figure 12 The FPGA circuit system 1200 is implemented using accessible bit streams (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions to enable... Figure 12 The FPGA circuit system 1200 or one or more of its components are configured and / or constructed.

[0095] Figure 12 The FPGA circuit system 1200 includes an example input / output (I / O) circuit system 1202 to obtain data from and / or output data to an example configuration circuit system 1204 and / or external hardware 1206. For example, the configuration circuit system 1204 may be implemented by an interface circuit system capable of obtaining a binary file, which may be implemented by bitstreams, data, and / or machine-readable instructions to configure the FPGA circuit system 1200 or portions thereof. In some such examples, the configuration circuit system 1204 may obtain the binary file from a user, a machine (e.g., a hardware circuit system (e.g., a programmable circuit system or a dedicated circuit system) that can implement an artificial intelligence / machine learning (AI / ML) model for generating the binary file), and / or any combination thereof. In some examples, the external hardware 1206 may be implemented by external hardware circuitry. For example, the external hardware 1206 may be implemented by… Figure 11 The microprocessor 1100 is implemented.

[0096] The FPGA circuit system 1200 also includes an array of example logic gate systems 1208, multiple example configurable interconnects 1210, and an example memory circuit system 1212. The logic gate systems 1208 and configurable interconnects 1210 can be configured to instantiate one or more operations / functions, which can correspond to... Figure 9At least some of the machine-readable instructions and / or other desired operations. Figure 12 The illustrated logic gate system 1208 is fabricated as multiple blocks or groups. Each block includes a semiconductor-based electrical structure that can be configured into a logic circuit. In some examples, the electrical structure includes logic gates (e.g., AND gates, OR gates, NOR gates, etc.) that provide basic building blocks for the logic circuit. Each logic gate system in 1208 contains electrically controllable switches (e.g., transistors) to enable the configuration of the electrical structure and / or logic gates to form a circuit for performing a desired operation / function. The logic gate system 1208 may include other electrical structures, such as lookup tables (LUTs), registers (e.g., flip-flops or latches), multiplexers, etc.

[0097] The configurable interconnect 1210 shown in the example is a conductive path, trace, via, etc., which may include an electrically controllable switch (e.g., a transistor) whose state can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more logic gate systems in logic gate system 1208, thereby programming the desired logic circuit.

[0098] The storage circuit system 1212 shown in the example is configured to store the results of one or more operations performed by the corresponding logic gates. The storage circuit system 1212 can be implemented using registers, etc. In the example shown, the storage circuit system 1212 is distributed within the logic gate circuit system 1208 for easy access and improved execution speed.

[0099] Figure 12 The example FPGA circuit system 1200 also includes an example dedicated operating circuit system 1214. In this example, the dedicated operating circuit system 1214 includes a dedicated circuit system 1216 that can be invoked to implement common functions, thereby avoiding the need to program these functions in the field. Examples of such dedicated circuit systems 1216 include memory (e.g., DRAM) controller circuit systems, PCIe controller circuit systems, clock circuit systems, transceiver circuit systems, and memory and multiply-accumulate circuit systems. Other types of dedicated circuit systems may also be present. In some examples, the FPGA circuit system 1200 may also include an example general-purpose programmable circuit system 1218, such as an example CPU 1220 and / or an example DSP 1222. Additionally or alternatively, other general-purpose programmable circuit systems 1218 may also be present, such as GPUs, XPUs, etc., that can be programmed to perform other operations.

[0100] although Figure 11 and Figure 12 It shows Figure 10The two example implementations of the programmable circuit system 1012 are provided, but many other implementations are envisioned. For example, the FPGA circuit system may include an onboard CPU, for example, Figure 11 One or more of the example CPUs 1220. Therefore, Figure 10 The programmable circuit system 1012 can also be achieved by at least... Figure 11 Example microprocessor 1100 and Figure 12 The example FPGA circuit system 1200 is combined to implement this. In some such hybrid examples, Figure 11 One or more cores of 1102 can execute Figure 9 The first part of the machine-readable instructions represented by the flowchart is for implementing one or more first operations / functions; Figure 12 The FPGA circuit system 1200 can be configured and / or constructed to implement with Figure 9 The flowchart represents the second part of the machine-readable instructions corresponding to one or more second operations / functions; and / or the ASIC can be configured and / or constructed to implement the same... Figure 9 The flowchart represents the third part of the machine-readable instructions, corresponding to one or more third operations / functions.

[0101] Therefore, it should be understood that Figure 5 Some or all of the circuit system can be instantiated at the same time or at different times. For example, Figure 11 One or more identical and / or different parts of the microprocessor 1100 can be programmed to execute one or more machine-readable instructions at the same and / or different times. In some examples, Figure 12 One or more identical and / or different parts of the FPGA circuit system 1200 may be configured and / or constructed to perform operations / functions corresponding to one or more parts of machine-readable instructions at the same time and / or at different times.

[0102] In some examples, Figure 5 Some or all of the circuitry in the system can be instantiated, for example, in one or more threads that execute concurrently and / or serially. For example, Figure 11 The microprocessor 1100 can execute machine-readable instructions in one or more threads that execute concurrently and / or serially. In some examples, Figure 12 The FPGA circuit system 1200 can be configured and / or constructed to implement operations / functions concurrently and / or serially. Furthermore, in some examples, Figure 5 Some or all of the circuit systems can be in Figure 11 One or more virtual machines and / or containers are implemented and executed on the microprocessor 1100.

[0103] In some examples, Figure 10 The programmable circuit system 1012 can be located in one or more packages. For example, Figure 11 Microprocessor 1100 and / or Figure 12 The FPGA circuitry system 1200 can be housed in one or more packages. In some examples, the XPU can be... Figure 10 The programmable circuit system 1012 is implemented and may be located in one or more packages. For example, the XPU may include a CPU located in one package (e.g., Figure 11 Microprocessor 1100, Figure 12 CPU 1220, etc.), and DSP located in another package (e.g., Figure 12 The DSP 1222), the GPU in another package, and the FPGA in yet another package (e.g., Figure 12 FPGA circuit system 1200).

[0104] Figure 13 A block diagram illustrating an example software distribution platform 1305 is shown, which is used to distribute software such as... Figure 10 Example machine-readable instructions 1032, such as software, are distributed to other hardware devices (e.g., hardware devices owned and / or operated by a third party other than the owner and / or operator of the software distribution platform). Example software distribution platform 1305 can be implemented by any computer server, data facility, cloud service, etc., capable of storing and transferring software to other computing devices. This third party can be a customer of the entity that owns and / or operates software distribution platform 1305. For example, the entity owning and / or operating software distribution platform 1305 could be, for example, a customer of, the entity that owns and / or operates software distribution platform 1305. Figure 10 The example machine-readable instruction 1032 refers to the developer, seller, and / or licensor of the software. This third party can be a consumer, user, retailer, OEM, etc., who purchases and / or obtains a software license for use and / or resale and / or sublicense. In the illustrated example, the software distribution platform 1305 includes one or more servers and one or more storage devices. The storage devices store the machine-readable instruction 1032, which, as described above, can correspond to... Figure 9Example machine-readable instructions. One or more servers of the example software distribution platform 1305 communicate with the example network 1310, which may correspond to the Internet and / or any one or more of the example networks described above. In some examples, the one or more servers respond to a request to transfer software as part of a commercial transaction to a requesting party. Payment for the delivery, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform and / or a third-party payment entity. The server enables the purchaser and / or licensor to download machine-readable instructions 1032 from the software distribution platform 1305. For example, it may correspond to... Figure 9 Software containing example machine-readable instructions can be downloaded to example programmable circuit system platform 1000, which will execute machine-readable instructions 1032 to implement unified MIPI control interface driver 405. In some examples, one or more servers of software distribution platform 1305 periodically provide, transmit, and / or enforce software (e.g., Figure 10 The example machine-readable instruction 1032 is updated to ensure that improvements, patches, updates, etc., are distributed and applied to the software at the end-user device. Although it is referred to as software above, the distributed "software" may alternatively be firmware.

[0105] This document uses “including” and “comprising” (and all their forms and tenses) as open-ended terms. Therefore, whenever a claim uses “including” or “comprising” in any form (e.g., includes, includes, comprising, having, etc.) in the preamble or in any type of claim statement, it should be understood that additional elements, items, etc., may also be present without exceeding the scope of the corresponding claim or statement. As used herein, the phrase “at least” is open-ended, as are the terms “comprising” and “including”, for example, in the preamble of a claim. When the term “and / or” is used, for example, in the form of “A, B, and / or C”, it refers to any combination or subset of A, B, C, such as: (1) only A; (2) only B; (3) only C; (4) A and B; (5) A and C; (6) B and C; or (7) A, B, and C. The phrase “at least one of A and B” as used herein in the context of describing structures, components, items, objects, and / or things is intended to refer to an implementation that includes any of the following: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B. Similarly, the phrase “at least one of A or B” as used herein in the context of describing structures, components, items, objects, and / or things is intended to refer to an implementation that includes any of the following: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B. The phrase “at least one of A and B” as used herein in the context of describing the implementation or execution of processes, instructions, actions, activities, etc., is intended to refer to an implementation that includes any of the following: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B. Similarly, the phrase “at least one of A or B” used in this document in the context of describing the implementation or execution of processes, instructions, actions, activities, etc. is intended to refer to an implementation that includes any of the following: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.

[0106] The singular references used herein (e.g., “a(a)”, “an(an)”, “first”, “second”, etc.) do not exclude plurals. The term “a(a)” or “an(an)” as used herein refers to one or more of the object. The terms “a(a)” (or “an(an)”), “one or more”, and “at least one” are used interchangeably herein. Furthermore, although multiple units, elements, or actions are listed separately, these units, elements, or actions may be implemented, for example, by the same entity or object. Additionally, although individual features may be included in different examples or claims, these features may be combined, and including them in different examples or claims does not imply that the combination of features is infeasible and / or lacks advantage.

[0107] Unless otherwise stated, the term "above" as used herein describes the relationship between two components relative to the Earth. A first component is above a second component if at least a portion of the second component lies between the Earth and the first component. Similarly, as used herein, a first component is "below" the second component when the first component is closer to the Earth than the second component. As stated above, a first component may be above or below a second component, and may be in one or more of the following situations: there are other components between them; there are no other components between them; the first component is in contact with the second component; or the first component and the second component are not in direct contact with each other.

[0108] Notwithstanding the foregoing, when referring to semiconductor devices (e.g., transistors), semiconductor dies containing semiconductor devices, and / or integrated circuit (IC) packages containing semiconductor dies during manufacturing or production, "above" is not relative to the earth, but relative to the underlying substrate on which the relevant components are manufactured, assembled, mounted, supported, or otherwise disposed. Therefore, unless otherwise stated or implied by the context, a first component (e.g., a transistor or other semiconductor device) within a semiconductor die, as used herein, is located "above" a second component when, during manufacturing / production, it is located further away from the substrate (e.g., a semiconductor wafer) on which both components are manufactured or otherwise disposed than a second component within the semiconductor die. Similarly, unless otherwise stated or implied by the context, a first component (e.g., a semiconductor die) within an IC package is located "above" a second component within the IC package when, during manufacturing, it is located further away from the printed circuit board (PCB) on which the IC package will be mounted or attached. It should be understood that semiconductor devices are often used in an orientation different from that during their manufacturing. Therefore, when referring to semiconductor devices (e.g., transistors), semiconductor dies containing semiconductor devices, and / or integrated circuit (IC) packages containing semiconductor dies during use, the definition of “above” in the preceding paragraph (i.e., the term “above” describes the relationship of two components relative to the earth) may apply depending on the context of use.

[0109] As used in this patent, the statement that any component (e.g., layer, film, region, area, or plate) is located on another component in any way (e.g., positioned on, located on, disposed on, or formed on another component) indicates that the mentioned component is in contact with the other component, or that the mentioned component is located above the other component, and that there are one or more intermediate components between them.

[0110] Unless otherwise indicated, the term "connection" as used herein (e.g., attachment, coupling, connection, and bonding) may include intermediate components between the elements involved by the connection term and / or relative movement between these elements. Therefore, a connection term does not necessarily mean that two elements are directly connected to each other and / or have a fixed relationship with each other. As used herein, the statement that any component is "in contact" with another component is defined as meaning that there is no intermediate component between the two components.

[0111] Unless otherwise expressly stated, the use of descriptive terms such as “first,” “second,” “third,” etc., herein does not assign or otherwise indicate any meaning of priority, physical order, arrangement in a list, and / or any kind of sorting, but is merely used as labels and / or arbitrary names to distinguish elements and thus facilitate understanding of the disclosed examples. In some examples, the descriptive term “first” may be used to refer to an element in a particular embodiment, while in the claims, the same element may be referred to using different descriptive terms such as “second” or “third.” In such cases, it should be understood that such descriptive terms are only used to clearly distinguish these elements in the context of discussion (e.g., within the claims), in which, for example, these elements might otherwise have had the same name.

[0112] As used herein, “approximately” and “about” modify objects / numerical values ​​to acknowledge that variations may exist in real-world applications. For example, as those skilled in the art will understand, “approximately” and “about” may modify dimensions that may not be precise due to manufacturing tolerances and / or other imperfections in the real world. For example, unless otherwise specified herein, “approximately” and “about” may indicate that such dimensions may be within tolerances of + / - 10%.

[0113] As used in this article, "nearly real-time" refers to events that occur almost instantaneously, while acknowledging that there may be real-world delays in computation time, transmission, etc. Therefore, unless otherwise specified, "nearly real-time" means real-time plus 1 second.

[0114] The phrase “to communicate” and its variations as used herein cover direct communication and / or indirect communication through one or more intermediate components, and do not require direct physical communication (e.g., wired communication) and / or continuous communication, but also include selective communication at periodic intervals, predetermined intervals, non-periodic intervals and / or at one-off events.

[0115] As used herein, “programmable circuit system” is defined as including: (i) one or more application-specific circuits (e.g., application-specific integrated circuits (ASICs)) configured to perform one or more specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors); and / or (ii) one or more general-purpose semiconductor-based circuits programmable with instructions to perform one or more specific functions and / or operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of programmable circuit systems include: a programmable microprocessor such as a central processing unit (CPU) capable of executing first instructions to implement one or more operations and / or functions; a field-programmable gate array (FPGA) programmable with second instructions to configure and / or construct the FPGA to instantiate one or more operations and / or functions corresponding to the first instructions; a graphics processing unit (GPU) capable of executing first instructions to implement one or more operations and / or functions; a digital signal processor (DSP) capable of executing first instructions to implement one or more operations and / or functions; an XPU; a network processing unit (NPU); one or more microcontrollers capable of executing first instructions to implement one or more operations and / or functions; and / or integrated circuits such as application-specific integrated circuits (ASICs). For example, an XPU can be implemented by a heterogeneous computing system that includes a variety of programmable circuit systems (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more NPUs, one or more DSPs, etc. and / or any combination thereof) and orchestration techniques (e.g., assigning one or more computing tasks to one or more application programming interfaces (APIs) of any one or more of the various types of programmable circuit systems that are suitable for and can be used to perform those computing tasks).

[0116] As used in this article, an integrated circuit / circuit system is defined as one or more semiconductor packages containing one or more circuit elements (e.g., transistors, capacitors, inductors, resistors, current paths, diodes, etc.). For example, an integrated circuit can be implemented as one or more of an ASIC, FPGA, chip, microchip, programmable circuit system, semiconductor substrate coupling multiple circuit elements, system-on-a-chip (SoC), etc.

[0117] As can be understood from the foregoing, example systems, apparatuses, articles, and methods for achieving unified control of MIPI cameras have been disclosed. More specifically, the examples disclosed herein solve the challenge of configuring and controlling any MIPI camera using a unified camera sensor driver for MIPI cameras. Therefore, the disclosed systems, apparatuses, articles, and methods relate to one or more improvements to the operation of machines such as computers or other electronic and / or mechanical equipment.

[0118] Other examples and combinations thereof include the following. Example 1 includes an apparatus for controlling a camera, the apparatus comprising: an interface circuitry system; instructions; and at least one processor circuitry programmed by the instructions to: access command translation data from at least one of the camera's memory or the memory of a host device to be coupled to the camera; parse the command translation data to map commands to one or more register settings of the camera, the commands specifying camera configurations applicable to a variety of different camera types, the one or more register settings being specific to the camera, the one or more register settings being configured to configure the camera based on the commands; and causing the one or more register settings to be sent to the camera via a mobile industry processor interface.

[0119] Example 2 includes the apparatus of Example 1, wherein one or more of the at least one processor circuitry are configured to: attempt to access the command conversion data from the memory of the camera; and, after an attempt to access the command conversion data from the memory of the camera fails, attempt to access the command conversion data from the memory of the host device.

[0120] Example 3 includes the apparatus of either Example 1 or 2, wherein the command is a first command, the one or more register settings are a first set of one or more register settings, and the command translation data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the camera, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify corresponding different possible camera configurations applicable to the plurality of different camera types, and the corresponding set of one or more register settings being used to configure the camera based on the corresponding command.

[0121] Example 4 includes an apparatus of any of Examples 1 to 3, wherein the one or more register settings include: a register address of the camera, and a register value for setting a camera register identified by the register address.

[0122] Example 5 includes an apparatus of any of Examples 1 to 4, wherein the one or more register settings include an ordered sequence of register settings, and one or more processor circuits of the at least one processor circuit are configured to send the ordered sequence of register settings to the camera via the mobile industry processor interface.

[0123] Example 6 includes the apparatus of Example 5, wherein the ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more camera registers identified by the register addresses.

[0124] Example 7 includes an apparatus of any of Examples 1 to 6, wherein one or more of the at least one processor circuitry are configured to: access command conversion data from at least one of the memory of the camera or the memory of the host device after a unified camera driver applicable to the various different camera types is loaded by the host device; and obtain the command via the unified camera driver.

[0125] Example 8 includes at least one non-transitory computer-readable medium, including instructions for causing at least one processor circuitry to perform at least the following operations: accessing a data structure including command conversion data, the data structure being stored in at least one of the memory of an image sensor or the memory of a host device to be coupled to the image sensor; mapping a command to one or more register settings of the image sensor based on the command conversion data, the command being for specifying a sensor configuration applicable to a variety of different image sensor types, the one or more register settings being specific to the image sensor, the one or more register settings being for configuring the image sensor based on the command; and causing the one or more register settings to be sent to the image sensor via a mobile industry processor interface.

[0126] Example 9 includes at least one non-transitory computer-readable medium of Example 8, wherein the instructions are configured to cause one or more processor circuits in the at least one processor circuit to: attempt to access the data structure from the memory of the image sensor; and, after an attempt to access the data structure from the memory of the image sensor fails, attempt to access the data structure from the memory of the host device.

[0127] Example 10 includes at least one non-transitory computer-readable medium of any of Examples 8 or 9, wherein the command is a first command, the one or more register settings are a first set of one or more register settings, and the command conversion data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the image sensor, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify corresponding different possible sensor configurations applicable to the plurality of different image sensor types, and the corresponding set of one or more register settings being used to configure the image sensor based on the corresponding command.

[0128] Example 11 includes at least one non-transitory computer-readable medium of any of Examples 8 to 10, wherein the one or more register settings include: a register address of the image sensor, and a register value for setting a sensor register identified by the register address.

[0129] Example 12 includes at least one non-transitory computer-readable medium of any of Examples 8 to 11, wherein the one or more register settings include an ordered sequence of register settings, and the instructions are for causing one or more processor circuits of the at least one processor circuit to send the ordered sequence of register settings to the image sensor via the mobile industrial processor interface.

[0130] Example 13 includes at least one non-transitory computer-readable medium of Example 12, wherein the ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more sensor registers identified by the register addresses.

[0131] Example 14 includes at least one non-transitory computer-readable medium of any of Examples 8 to 13, wherein the instructions are configured to cause one or more processor circuits of the at least one processor circuit to perform the following operations: load a unified camera driver applicable to the various different image sensor types; access the data structure after the unified camera driver is loaded; and obtain the command via the unified camera driver.

[0132] Example 15 includes a method for controlling a camera device, the method comprising: loading command-transformation data from a data structure, the data structure being stored in at least one of the memory of the camera device or the memory of a host device to be coupled to the camera device; mapping commands to one or more register settings of the camera device based on the command-transformation data by at least one processor circuit programmed by instructions, the commands specifying device configurations applicable to a variety of different camera device types, the one or more register settings being specific to the camera device, the one or more register settings being configured to configure the camera device based on the commands; and sending the one or more register settings to the camera device via a mobile industry processor interface.

[0133] Example 16 includes the method of Example 15, wherein loading the command conversion data from the data structure includes: attempting to access the data structure from the memory of the camera device; and after the attempt to access the data structure from the memory of the camera device fails, attempting to access the data structure from the memory of the host device.

[0134] Example 17 includes a method from either Example 15 or 16, wherein the command is a first command, the one or more register settings are a first set of one or more register settings, and the command translation data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the camera device, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify corresponding different possible device configurations applicable to the plurality of different camera device types, and the corresponding set of one or more register settings being used to configure the camera device based on the corresponding command.

[0135] Example 18 includes a method of any of Examples 15 to 17, wherein the one or more register settings include an ordered sequence of register settings, and the sending includes sending the ordered sequence of register settings to the camera device via the mobile industry processor interface.

[0136] Example 19 includes the method of Example 18, wherein the ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more registers of the camera device, the one or more registers being identified by the register addresses.

[0137] Example 20 includes a method from any of Examples 15 to 19, further comprising: loading a unified camera driver applicable to the various different camera device types, wherein loading the command conversion data from the data structure occurs after loading the unified camera driver; and obtaining the command via the unified camera driver.

[0138] The following claims are incorporated herein by reference. Although certain example systems, apparatuses, articles of manufacture, and methods are disclosed herein, the scope of protection of this patent is not limited thereto. Rather, this patent covers all systems, apparatuses, articles of manufacture, and methods that reasonably fall within the scope of the claims of this patent.

Claims

1. An apparatus for controlling a camera, the apparatus comprising: Interface circuit system; instruction; as well as At least one processor circuit, said at least one processor circuit being programmed by the instructions to perform the following operations: Command translation data that accesses at least one of the camera's memory or the memory of the host device to be coupled to the camera; The command is parsed to convert data to map the command to one or more register settings of the camera. The command is used to specify a camera configuration applicable to a variety of different camera types. The one or more register settings are specific to the camera and are used to configure the camera based on the command. as well as The settings of one or more registers are sent to the camera via the mobile industrial processor interface.

2. The apparatus according to claim 1, wherein, One or more of the at least one processor circuits are used for: Attempting to access the command conversion data from the camera's memory; and After an attempt to access the command conversion data from the camera's memory fails, an attempt is made to access the command conversion data from the host device's memory.

3. The apparatus according to claim 1, wherein, The command is a first command, the one or more register settings are a first set of one or more register settings, and the command conversion data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the camera, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify corresponding different possible camera configurations applicable to the plurality of different camera types, and the corresponding set of one or more register settings being used to configure the camera based on the corresponding command.

4. The apparatus according to claim 1, wherein, The one or more register settings include: the register address of the camera, and a register value for setting the camera register identified by the register address.

5. The apparatus according to claim 1, wherein, The one or more register settings include an ordered sequence of register settings, and one or more processor circuits in the at least one processor circuit are configured to send the ordered sequence of register settings to the camera via the mobile industry processor interface.

6. The apparatus according to claim 5, wherein, The ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more camera registers identified by the register addresses.

7. The apparatus according to claim 1, wherein, One or more of the at least one processor circuits are used for: After a unified camera driver applicable to the various different camera types is loaded by the host device, the command conversion data is accessed from at least one of the camera's memory or the host device's memory; as well as The command is obtained via the unified camera driver.

8. At least one non-transitory computer-readable medium, comprising instructions for causing at least one processor circuitry to perform at least the following operations: Access includes a data structure that includes command-transformed data, the data structure being stored in at least one of the memory of the image sensor or the memory of a host device to be coupled to the image sensor; Based on the command-transformed data, the command is mapped to one or more register settings of the image sensor. The command is used to specify a sensor configuration applicable to a variety of different image sensor types. The one or more register settings are specific to the image sensor and are used to configure the image sensor based on the command. as well as The settings of the one or more registers are sent to the image sensor via the mobile industrial processor interface.

9. The at least one non-transitory computer-readable medium according to claim 8, wherein, The instructions are used to cause one or more of the at least one processor circuit to perform the following operations: Attempting to access the data structure from the memory of the image sensor; and After an attempt to access the data structure from the memory of the image sensor fails, an attempt is made to access the data structure from the memory of the host device.

10. The at least one non-transitory computer-readable medium according to claim 8, wherein, The command is a first command, the one or more register settings are a first set of one or more register settings, and the command conversion data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the image sensor, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify corresponding different possible sensor configurations applicable to the plurality of different image sensor types, and the corresponding set of one or more register settings being used to configure the image sensor based on the corresponding command.

11. The at least one non-transitory computer-readable medium according to claim 8, wherein, The one or more register settings include: the register address of the image sensor, and a register value for setting the sensor register identified by the register address.

12. The at least one non-transitory computer-readable medium according to claim 8, wherein, The one or more register settings include an ordered sequence of register settings, and the instructions are used to cause one or more of the at least one processor circuitry to send the ordered sequence of register settings to the image sensor via the mobile industrial processor interface.

13. The at least one non-transitory computer-readable medium according to claim 12, wherein, The ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more sensor registers identified by the register addresses.

14. The at least one non-transitory computer-readable medium according to claim 8, wherein, The instructions are used to cause one or more of the at least one processor circuit to perform the following operations: Load a unified camera driver applicable to the various different image sensor types; Access the data structure after the unified camera driver is loaded; and The command is obtained via the unified camera driver.

15. A method for controlling a camera device, the method comprising: Load command transformation data from a data structure, which is stored in at least one of the memory of the camera device or the memory of a host device to be coupled to the camera device; At least one processor circuit programmed via instructions translates data based on the command, mapping the command to one or more register settings of the camera device, the command being used to specify a device configuration applicable to a variety of different camera device types, the one or more register settings being specific to the camera device, the one or more register settings being used to configure the camera device based on the command; as well as The settings of one or more registers are sent to the camera device via the mobile industry processor interface.

16. The method of claim 15, wherein, Loading the command transformation data from the data structure includes: Attempting to access the data structure from the memory of the camera device; and After an attempt to access the data structure from the memory of the camera device fails, an attempt is made to access the data structure from the memory of the host device.

17. The method according to claim 15, wherein, The command is a first command, the one or more register settings are a first set of one or more register settings, and the command conversion data is used to map a corresponding command among a plurality of commands to a corresponding set of one or more register settings specific to the camera device, the plurality of commands including the first command, the corresponding set of one or more register settings including the first set of one or more register settings, the corresponding command being used to specify different possible device configurations applicable to the various different camera device types, and the corresponding set of one or more register settings being used to configure the camera device based on the corresponding command.

18. The method according to claim 15, wherein, The one or more register settings include an ordered sequence of register settings, and the transmission includes sending the ordered sequence of register settings to the camera device via the mobile industrial processor interface.

19. The method according to claim 18, wherein, The ordered sequence of register settings includes: an ordered sequence of register addresses, and corresponding register values ​​for setting one or more registers of the camera device, wherein the one or more registers are identified by the register addresses.

20. The method of claim 15, further comprising: Loading a unified camera driver applicable to the various different camera device types, wherein loading the command conversion data from the data structure occurs after loading the unified camera driver; and The command is obtained via the unified camera driver.

Images