Image data injection system and method based on MIPI signal simulation

Abstract

According to the image data injection system and method based on MIPI signal simulation, a data injection hardware platform has the function of simulating a physical image sensor, and the original physical image sensor can collect images and output MIPI signals to a main control chip; the data injection hardware platform converts the RAW data prepared in advance into the MIPI signal consistent with the output format of the physical image sensor so as to replace the effect of the physical Sensor. The invention relates to a hardware testing tool which is used for simulating the output of an image sensor and can be applied to injecting image data into a main control chip platform, and a method using the tool. According to the invention, the dependence on a physical sensor and an external environment in the traditional IPC development can be eliminated, and a stable, repeatable and efficient test environment is provided, so that the research and development efficiency, the test coverage rate and the problem reproduction accuracy of an image system are improved.

Description

Technical Field

[0001] This invention belongs to the field of image system development, debugging and verification technology, and specifically relates to an image data injection system and method based on MIPI signal simulation. Background Technology

[0002] In the traditional IPC development process, engineers rely on physical sensors to provide image data for image quality debugging, ISP (Image Signal Processor) algorithm verification, and system problem troubleshooting.

[0003] This traditional method has many problems:

[0004] Highly dependent on the environment: Debugging results are limited by changes in the physical environment, such as lighting, color temperature, and scene content. Interference from these variables makes it difficult to maintain consistent testing conditions.

[0005] Difficulty in reproducing specific scenarios: When testers or customers report image problems in specific scenarios (such as noise or artifacts under specific lighting conditions), it is difficult to reproduce the physical scenario 100% in the development environment.

[0006] Hardware instability: In the early stages of development, the physical sensor or its hardware driver may be unstable, which brings additional challenges and interference to the development of the ISP algorithm and system integration.

[0007] These factors combined have led to low R&D efficiency, insufficient test coverage, and difficulty in locating and fixing problems. Summary of the Invention

[0008] The purpose of this invention is to provide an image data injection system and method based on MIPI signal simulation.

[0009] To address the above problems, this invention provides an image data injection system based on MIPI signal simulation, comprising:

[0010] The host computer is used to run and control the host computer software and store RAW format image data;

[0011] The host computer software runs on the host computer and is used to load, configure, and send RAW format image data and parameters of the simulated physical image sensor stored in the host computer.

[0012] A high-speed data interface is used to connect the host computer and the data injection hardware platform;

[0013] The data injection hardware platform is used to receive RAW format image data and parameters of the simulated physical image sensor from the host computer. Based on the parameters of the simulated physical image sensor, the RAW format image data is converted into a MIPI signal that is consistent with the output format of the simulated physical image sensor.

[0014] The tested main control chip platform is a development board equipped with an ISP chip;

[0015] The MIPI high-speed connection unit is used to connect the MIPI signal output by the data injection hardware platform to the MIPI receiving interface of the main control chip platform under test.

[0016] Furthermore, in the above system, the parameters simulating the physical image sensor include:

[0017] Frame height, number of channels, frame rate, image width, height, and RAW data format.

[0018] Furthermore, in the above system, the high-speed data interface is USB 3.0.

[0019] Furthermore, in the above system, the host computer is used to identify and control the data injection hardware platform by installing a preset driver.

[0020] Furthermore, in the above system, the default driver is the WinUSB driver installed by Zadig.

[0021] Furthermore, in the above system, the host computer is used to inject data into the hardware platform for identification as a Gadget Zero device.

[0022] Furthermore, in the above system, the data injection hardware platform outputs the MIPI signal to the tested main control chip platform through the onboard MIPI transmit interface.

[0023] Furthermore, in the above system, the MIPI high-speed connection unit includes: an adapter board, a MIPI high-speed cable connected to one end of the data injection hardware platform and the adapter board respectively, and the other end of the adapter board connected to the MIPI receiving interface of the tested main control chip platform.

[0024] According to another aspect of the present invention, an image data injection method based on MIPI signal simulation is also provided, employing the image data injection system based on MIPI signal simulation described in any of the preceding claims, the method comprising:

[0025] The host computer software sends the pre-acquired RAW format image data and the parameters of the simulated physical image sensor to the data injection hardware platform through a high-speed data interface;

[0026] After receiving RAW format image data and parameters of the simulated physical image sensor, the data injection hardware platform converts the RAW format image data into a MIPI signal that is consistent with the output format of the simulated physical image sensor based on the parameters of the simulated physical image sensor. Then, it injects the MIPI signal into the MIPI receiving interface of the main control chip platform under test through the MIPI transmitting interface.

[0027] Furthermore, in the above method, before the host computer software sends the pre-acquired RAW format image data and the parameters of the simulated physical image sensor to the data injection hardware platform via a high-speed data interface, it also includes:

[0028] Start the main control chip platform under test and run the image output application of the main control chip platform under test;

[0029] Start the data injection hardware platform, and ensure that the data injection hardware platform starts later than the tested main control chip platform;

[0030] The host computer software sends the pre-acquired RAW format image data and parameters of the simulated physical image sensor to the data injection hardware platform via a high-speed data interface, including:

[0031] On the host computer, a RAW format image data file is loaded via host computer software;

[0032] On the host computer software, the parameters of the simulated physical image sensor that matches the RAW format image data and the main control chip platform under test are configured.

[0033] Start the data transmission function of the host computer software;

[0034] The host computer sends RAW format image data to the data injection hardware platform through a high-speed data interface.

[0035] Compared to existing technologies, the data injection hardware platform of this invention simulates the function of a physical image sensor. While a physical image sensor typically acquires images and outputs MIPI signals to the main control chip, the data injection hardware platform in this application converts pre-prepared RAW data into MIPI signals with the same output format as a physical image sensor, thus replacing the function of the physical sensor. This invention is for simulating image sensor output and can be applied to hardware testing tools for injecting image data into a main control chip platform, as well as methods for using such tools.

[0036] This invention can eliminate the dependence on physical sensors and external environment in traditional IPC development, and provide a stable, repeatable and efficient testing environment to improve the R&D efficiency, test coverage and accuracy of problem reproduction of image systems (especially ISP algorithms).

[0037] This invention, through the above technical solution, decouples the development and debugging process from the physical sensors, resulting in significant technical effects:

[0038] Create a repeatable test environment: By injecting identical RAW data, interference from environmental factors such as lighting and scene is completely eliminated, ensuring the uniqueness and comparability of test results.

[0039] Achieve accurate regression testing: When ISP parameters or algorithm code are modified, regression testing can be performed immediately using the same RAW data to accurately evaluate the effects of the changes and prevent performance degradation.

[0040] Improve the objectivity of algorithm evaluation: When evaluating different versions of algorithms, using "same source" RAW data input allows for fair and objective quantitative comparison of performance.

[0041] Shorten testing cycles and achieve automation: This system is easily integrated into automated testing frameworks. It can build test libraries containing RAW data from various typical scenarios, enabling 24 / 7 unattended automated image quality regression testing.

[0042] Efficient fault reproduction and analysis: When image problems occur, simply obtain the RAW data for that scene to reproduce the problem 100% in the development environment, greatly facilitating the location and repair of the problem. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of an image data injection system based on MIPI signal simulation according to an embodiment of the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings.

[0045] In a typical configuration of this application, the terminal, the device of the service network, and the trusted party all include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0046] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0047] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0048] like Figure 1 As shown, the present invention provides an image data injection system based on MIPI signal simulation, comprising:

[0049] The host computer (PC) is used to run and control the host computer software and store RAW format image data;

[0050] The host computer software (FHVE Player) runs on the host computer and is used to load, configure, and send RAW format image data and parameters of the simulated physical image sensor stored on the host computer (PC). Here, users can configure the parameters of the simulated physical image sensor in this software, such as frame height, number of channels, frame rate (FPS), image width, height, and RAW data format (such as RAW_10bit or RAW_12bit).

[0051] A high-speed data interface (USB 3.0) is used to connect the host computer and the data injection hardware platform; wherein, the host computer identifies and controls the data injection hardware platform (identifies it as a Gadget Zero device) by installing a preset driver (such as the WinUSB driver installed by Zadig).

[0052] The data injection hardware platform is used to receive RAW format image data and parameters of the simulated physical image sensor from the host computer. Based on the parameters of the simulated physical image sensor, the RAW format image data is converted into a MIPI signal that is consistent with the output format of the simulated physical image sensor. Preferably, the data injection hardware platform outputs the MIPI signal to the main control chip platform under test (test board) through the onboard MIPI transmitter interface.

[0053] The main control chip platform under test (test board) is a development board equipped with an ISP chip;

[0054] A high-speed MIPI connection unit is used to connect the MIPI signal output by the data injection hardware platform to the MIPI receiving interface of the main control chip platform under test. Preferably, the high-speed MIPI connection unit includes: an adapter board (sensor board), a MIPI high-speed cable connected to one end of the data injection hardware platform and the adapter board (sensor board), and the other end of the adapter board (sensor board) connected to the MIPI receiving interface of the main control chip platform under test.

[0055] This invention provides an image data injection system in which a data injection hardware platform replaces the physical image sensor, directly injecting RAW format image data into the image processing platform under test in the form of a MIPI signal. This system completely replaces the function of the physical image sensor. Its core technical solution is to construct a test link consisting of a host computer (PC), the data injection hardware platform, and the main control chip platform under test.

[0056] This invention may include: a host computer storing RAW format image data; data transmission software running on the host computer for loading the RAW format image data and transmitting it according to preset parameters; a data injection hardware platform connected to the host computer via a high-speed data interface (USB 3.0) for receiving the RAW format image data; the data injection hardware platform having a MIPI transmit (mipi_tx) interface and configured to convert the received RAW format image data into a MIPI signal output by an analog image sensor; the image processing platform under test having a MIPI receive (mipi_rx) interface; and the MIPI transmit interface of the data injection hardware platform and the MIPI receive interface of the image processing platform under test being connected via a high-speed MIPI cable.

[0057] According to another aspect of the present invention, an image data injection method based on MIPI signal simulation is also provided, comprising:

[0058] Step S1: The host computer software (FHVE Player) sends the pre-acquired RAW format image data and the parameters of the simulated physical image sensor to the data injection hardware platform via a high-speed data interface (USB 3.0).

[0059] Preferably, before step S1, the method further includes:

[0060] Step S01: Start the main control chip platform under test and run the image output application of the main control chip platform under test;

[0061] Step S02: Start the data injection hardware platform and ensure that the data injection hardware platform starts later than the tested main control chip platform;

[0062] Preferably, step S1 includes:

[0063] Step S11: On the host computer, load a RAW format image data file through the host computer software;

[0064] Step S12: On the host computer software, configure the parameters of the simulated physical image sensor that matches the RAW format image data and the main control chip platform under test.

[0065] Step S13: Start the data transmission function of the host computer software;

[0066] In step S14, the host computer sends the RAW format image data to the data injection hardware platform via a high-speed data interface (USB 3.0).

[0067] In step S2, after receiving the RAW format image data and the parameters of the simulated physical image sensor, the data injection hardware platform converts the RAW format image data into a MIPI signal that is consistent with the output format of the simulated physical image sensor based on the parameters of the simulated physical image sensor. Then, the MIPI signal is injected into the MIPI receiving interface (mipi_rx) of the main control chip platform under test through the MIPI transmitting interface (mipi_tx).

[0068] Specifically, in some embodiments of the present invention:

[0069] 1. Hardware environment setup:

[0070] You can prepare a RAW data injection hardware platform, a MIPI adapter board (sensor board), a test main control chip platform (test board), a USB 3.0 data cable, and a MIPI high-speed cable.

[0071] You can use a MIPI high-speed cable to connect the MIPI adapter board to the transmit interface (mipi_tx) of the data injection hardware platform, and then connect the MIPI adapter board to the MIPI_RX interface of the main control chip board under test.

[0072] 2. Hardware platform firmware flashing (for first use or update):

[0073] Data can be injected into the hardware platform's U-Boot via UARTO login;

[0074] Using TFTP, the kernel and loader.bin files are sequentially burned to the specified addresses in the SPI Flash. For example, the kernel is burned to 0x100000 and the loader.bin file is burned to 0x400000.

[0075] Set the bootcmd startup command and save it (sa).

[0076] 3. System Startup and Connection:

[0077] Key steps: The main control chip board under test must be started first, and the output demo program must be run on the main control chip board under test.

[0078] After powering on the main control chip board under test and the data injection hardware platform, use a USB 3.0 data cable to connect the data injection hardware platform to the host PC.

[0079] Note: It is essential to ensure that the data injection hardware platform starts after the tested main control chip board; otherwise, the MipiRX of the tested main control chip board may not receive data. Also, ensure that the USB is recognized as version 3.0; version 2.0 will cause a timeout.

[0080] 4. Install the host computer driver:

[0081] After the data injection hardware platform starts up and connects to the host PC, and the driver is installed correctly, the recognized "Gadget Zero" device will appear in the host PC's device manager.

[0082] 5. Image data injection operation:

[0083] Launch the FHVE Player software on the host PC.

[0084] Sensor Configuration: Enter the Settings interface and configure the Frame Height (e.g., 2250) and Lane (e.g., 2 Lanes) according to the actual situation of the physical image sensor being simulated.

[0085] Loading RAW image data: Open an acquired RAW data file. The RAW format image data source should be 10-bit or 12-bit closely packed RAW. It is recommended to store the RAW file on a solid-state drive to ensure read speed.

[0086] RAW parameter configuration:

[0087] Frame Format: You must select a format supported by the sensor, such as RAW_10bit or RAW_12bit.

[0088] Width and Height: Fill in according to the actual RAW stream (e.g., 3840x2160).

[0089] FPS: Configure according to the actual frame rate of the demo output from the chip board (e.g., 25).

[0090] To begin the injection: In the FHVE Player main interface, check the "Start Loop Playback" and "Send to ISP" options in the lower right corner.

[0091] At this point, the data injection hardware platform begins to send MIPI signals in a loop.

[0092] 6. Result Verification:

[0093] During system operation, the data injection hardware platform simulates a physical image sensor to continuously feed MIPI signals to the tested main control chip platform.

[0094] The ISP of the chip board under test will process these MIPI signals and encode the output video stream (such as RTSP or UDP stream).

[0095] At this point, you can use FHVE Player or other third-party players to access the IP address of the chip board under test (e.g., 192.168.72.142) and view the real-time video feed processed by the ISP.

[0096] Here, the data injection hardware platform of this invention has the function of simulating a physical image sensor. Originally, a physical image sensor would acquire images and output MIPI signals to the main control chip. However, in this application, the data injection hardware platform converts pre-prepared RAW data into MIPI signals with the same output format as the physical image sensor, thereby replacing the function of the physical sensor. This invention is for simulating image sensor output and can be applied to hardware testing tools for injecting image data into a main control chip platform, as well as methods for using such tools.

[0097] This invention can eliminate the dependence on physical sensors and external environment in traditional IPC development, and provide a stable, repeatable and efficient testing environment to improve the R&D efficiency, test coverage and accuracy of problem reproduction of image systems (especially ISP algorithms).

[0098] This invention, through the above technical solution, decouples the development and debugging process from the physical sensors, resulting in significant technical effects:

[0099] Create a repeatable test environment: By injecting identical RAW data, interference from environmental factors such as lighting and scene is completely eliminated, ensuring the uniqueness and comparability of test results.

[0100] Achieve accurate regression testing: When ISP parameters or algorithm code are modified, regression testing can be performed immediately using the same RAW data to accurately evaluate the effects of the changes and prevent performance degradation.

[0101] Improve the objectivity of algorithm evaluation: When evaluating different versions of algorithms, using "same source" RAW data input allows for fair and objective quantitative comparison of performance.

[0102] Shorten testing cycles and achieve automation: This system is easily integrated into automated testing frameworks. It can build test libraries containing RAW data from various typical scenarios, enabling 24 / 7 unattended automated image quality regression testing.

[0103] Efficient fault reproduction and analysis: When image problems occur, simply obtain the RAW data for that scene to reproduce the problem 100% in the development environment, greatly facilitating the location and repair of the problem.

[0104] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0105] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, a magnetic or optical drive, a floppy disk, or similar devices. Furthermore, some steps or functions of the present invention can be implemented in hardware, for example, as circuitry that works with a processor to perform the various steps or functions.

[0106] Furthermore, a portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. The program instructions invoking the methods of the invention may be stored in a fixed or removable recording medium, and / or transmitted via a data stream in a broadcast or other signal-carrying medium, and / or stored in the working memory of a computer device operating according to the program instructions. Here, an embodiment of the invention includes an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the apparatus is triggered to operate the methods and / or technical solutions based on the foregoing embodiments of the invention.

[0107] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in the apparatus claims may also be implemented by a single unit or device in software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

Claims

1. An image data injection system based on MIPI signal simulation, characterized in that, include: The host computer is used to run and control the host computer software and store RAW format image data; The host computer software runs on the host computer and is used to load, configure, and send RAW format image data and parameters of the simulated physical image sensor stored in the host computer. A high-speed data interface is used to connect the host computer and the data injection hardware platform; The data injection hardware platform is used to receive RAW format image data and parameters of the simulated physical image sensor from the host computer. Based on the parameters of the simulated physical image sensor, the RAW format image data is converted into a MIPI signal that is consistent with the output format of the simulated physical image sensor. The tested main control chip platform is a development board equipped with an ISP chip; The MIPI high-speed connection unit is used to connect the MIPI signal output by the data injection hardware platform to the MIPI receiving interface of the main control chip platform under test.

2. The image data injection system based on MIPI signal simulation as described in claim 1, characterized in that, The parameters of the simulated physical image sensor include: Frame height, number of channels, frame rate, image width, height, and RAW data format.

3. The image data injection system based on MIPI signal simulation as described in claim 1, characterized in that, The high-speed data interface is USB 3.

0.

4. The image data injection system based on MIPI signal simulation as described in claim 1, characterized in that, The host computer is used to identify and control the data injection hardware platform by installing preset drivers.

5. The image data injection system based on MIPI signal simulation as described in claim 4, characterized in that, The default driver is the WinUSB driver installed by Zadig.

6. The image data injection system based on MIPI signal simulation as described in claim 4, characterized in that, The host computer is used to inject data into the hardware platform to identify it as a Gadget Zero device.

7. The image data injection system based on MIPI signal simulation as described in claim 1, characterized in that, The data injection hardware platform outputs the MIPI signal to the tested main control chip platform through the onboard MIPI transmitter interface.

8. The image data injection system based on MIPI signal simulation as described in claim 1, characterized in that, The MIPI high-speed connection unit includes: an adapter board, a MIPI high-speed cable that is connected to one end of the data injection hardware platform and the adapter board respectively, and the other end of the adapter board is connected to the MIPI receiving interface of the main control chip platform under test.

9. An image data injection method based on MIPI signal simulation, characterized in that, The image data injection system based on MIPI signal simulation as described in any one of claims 1 to 8, the method comprising: The host computer software sends the pre-acquired RAW format image data and the parameters of the simulated physical image sensor to the data injection hardware platform through a high-speed data interface; After receiving RAW format image data and parameters of the simulated physical image sensor, the data injection hardware platform converts the RAW format image data into a MIPI signal that is consistent with the output format of the simulated physical image sensor based on the parameters of the simulated physical image sensor. The MIPI signal is then injected into the MIPI receiving interface of the main control chip platform under test through the MIPI transmitting interface.

10. The image data injection method based on MIPI signal simulation as described in claim 9, characterized in that, Before the host computer software sends the pre-acquired RAW format image data and the parameters of the simulated physical image sensor to the data injection hardware platform via a high-speed data interface, it also includes: Start the main control chip platform under test and run the image output application of the main control chip platform under test; Start the data injection hardware platform, and ensure that the data injection hardware platform starts later than the tested main control chip platform; The host computer software sends the pre-acquired RAW format image data and parameters of the simulated physical image sensor to the data injection hardware platform via a high-speed data interface, including: On the host computer, a RAW format image data file is loaded via host computer software; On the host computer software, the parameters of the simulated physical image sensor that match the RAW format image data and the main control chip platform under test are configured. Start the data transmission function of the host computer software; The host computer sends RAW format image data to the data injection hardware platform through a high-speed data interface.

Images