Graphics processor, running method of graphics processor and electronic equipment

Abstract

The embodiment of the invention discloses a graphics processor, an operation method of the graphics processor and electronic equipment, and relates to the field of computers. According to the method, the DMA module is used for directly transmitting data between the SRAM and the configuration register / memory of the ISP, frequent intervention of a CPU is not needed, the configuration process of the ISP is simplified, and the configuration efficiency is improved. The SRAM maps a section of area with the same size as the ISP configuration register / memory, and quick updating and reading of ISP configuration and information are realized through direct reading and writing of the DMA. Compared with a mode of directly using a plurality of registers and memories, the mapping mode has the advantages that the occupation of hardware resources is reduced, so that the area of the whole SRAM is smaller than the total area of a mode of independently using the registers and the memories.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a graphics processor, a method for operating a graphics processor, and an electronic device. Background Art

[0002] In current image processing technology, the configuration of ISP (Image Signal Processing) is implemented through a mechanism called shadow register / MEM. The core of this mechanism is that there is an accurate correspondence between the shadow register / MEM and the configuration / information register and configuration / information MEM inside the ISP. The specific operation process is as follows: Before the ISP is officially started, the software system will use the APB bus to configure the shadow register / MEM in detail. Once the configuration work is successfully completed, the ISP is immediately activated. At this time, the ISP will automatically read the configuration information from the shadow register / MEM and load it into the configuration register / MEM, and then start the image processing task.

[0003] When the ISP successfully processes the image of the current frame, it will export the processing results and related information from the information register / MEM and write it back to the shadow register / MEM for subsequent reading and analysis by the software system. At the same time, the ISP will also prepare for the processing of the next frame of the image, that is, it will read the configuration information of the next frame of the image from the shadow register / MEM and load it into the configuration register / MEM. It is worth noting that in the workflow of the ISP, the configuration register / MEM plays a write-only (wr) role, while the information register / MEM is responsible for the read-only (rd) task.

[0004] However, this prior art also has certain limitations. First, from the perspective of hardware resources, due to the large number of configuration and information registers / MEMs of the ISP, the use of shadow registers / MEM configuration will inevitably lead to a significant increase in hardware overhead. Specifically, the introduction of shadow registers / MEMs requires double the hardware resources to support. Second, from the perspective of CPU resource utilization, since each frame of the image needs to reconfigure the shadow registers / MEMs through the APB bus, this will lead to frequent CPU resource occupation and potential performance bottlenecks.

[0005] In summary, although the current shadow register / MEM configuration method plays a certain role in the ISP configuration process, its disadvantages such as large hardware overhead and high CPU resource occupancy cannot be ignored. Therefore, it is necessary to further optimize and improve this configuration method to improve the work efficiency and performance of the ISP. Summary of the invention

[0006] The embodiments of the present application provide a graphics processor, a method for operating the graphics processor, and an electronic device, which can solve the problems of high hardware overhead and low efficiency when the graphics processor is working in the prior art. The technical solution is as follows:

[0007] In a first aspect, an embodiment of the present application provides a graphics processor, including:

[0008] Selector, DMA module, FLASH memory and algorithm module;

[0009] The FLASH memory includes a configuration register, an information register, a configuration memory and an information memory, the configuration register and the configuration memory are set to a write-only mode, and the information register and the information memory are set to a read-only mode;

[0010] The image processor is provided with an SRAM outside, and the SRAM is provided with a first storage area, a second storage area, a third storage area and a fourth storage area;

[0011] The first input end of the selector is connected to the APB bus outside the image processor, the second input end of the selector is connected to the DMA module, the output end of the selector is connected to the first end of the FLASH memory, the second end of the FLASH memory is connected to the ISP algorithm module, and the control end of the selector is connected to the controller in the graphics processor;

[0012] The working process of the graphics processor includes:

[0013] The controller instructs the selector to conduct the first input end and the output end through the control end of the selector, then receives configuration information on the external APB bus, and configures the configuration register using the received configuration information, enables the DMA module, and configures the base address of the DMA module;

[0014] When the controller receives an external vertical synchronization signal, it instructs the selector to conduct the second input end and the output end, instructs the DMA module to write the information in the first storage area into the configuration register, read the data in the information register into the second storage area, write the data in the third storage area into the configuration memory, and write the data in the information memory into the fourth storage area;

[0015] After the controller detects that the DMA module completes the read and write operations, it instructs the selector to connect the first input end and the output end.

[0016] In a second aspect, an embodiment of the present application provides an operating method of an image processor, including:

[0017] The controller instructs the selector to conduct the first input end and the output end through the control end of the selector, then receives the configuration information on the external APB bus, and configures the configuration register using the received configuration information, enables the DMA module, and configures the base address of the DMA module;

[0018] When the controller receives an external vertical synchronization signal, it instructs the selector to conduct the second input end and the output end, instructs the DMA module to write the information in the first storage area into the configuration register, read the data in the information register into the second storage area, write the data in the third storage area into the configuration memory, and write the data in the information memory into the fourth storage area;

[0019] After the controller detects that the DMA module completes the read and write operations, it instructs the selector to connect the first input end and the output end.

[0020] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0021] The DMA module is used to directly transfer data between the SRAM and the ISP configuration register / memory, without the need for frequent CPU intervention, which simplifies the ISP configuration process and improves configuration efficiency. SRAM maps an area of ​​the same size as the ISP configuration register / memory, and through direct reading and writing of DMA, it realizes the rapid update and reading of ISP configuration and information. Compared with directly using multiple registers and memories, this mapping method reduces the occupation of hardware resources, making the area of ​​the entire SRAM smaller than the total area of ​​registers and memories used separately. SRAM is connected to the graphics processor through the AHB bus. Compared with the APB bus, the AHB bus has higher data transmission efficiency and bandwidth, thereby speeding up the reading and writing of ISP configuration information and data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a structural schematic diagram of an image processor in the prior art;

[0024] Figure 2 is a schematic diagram of the structure of an image processor provided in an embodiment of the present application;

[0025] Figure 3 is a mapping relationship diagram of the SRAM and FLASH memory in the image processor provided in the embodiment of the present application;

[0026] Figure 4 is a flowchart of an operating method of an image processor provided in an embodiment of the present application;

[0027] Figure 5 This is a timing diagram of the operation of a graphics processor provided by this application. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0029] See also Figure 1 and Figure 2 , is a schematic diagram of a graphics processor provided in an embodiment of the present application, including: a selector, a DMA module, a FLASH memory and an algorithm module;

[0030] The FLASH memory includes a configuration register, an information register, a configuration memory and an information memory, the configuration register and the configuration memory are set to a write-only mode, and the information register and the information memory are set to a read-only mode; an SRAM is set outside the image processor, and the SRAM is provided with a first storage area sram_0, a second storage area sram_1, a third storage area sram_2 and a fourth storage 3; a first input end of the selector is connected to an APB bus outside the image processor, a second input end of the selector is connected to a DMA module, an output end of the selector is connected to a first end of the FLASH memory, a second end of the FLASH memory is connected to an ISP algorithm module, and a control end of the selector is connected to a controller in a graphics processor.

[0031] The selector is responsible for selecting the signal path. According to the instructions of the controller, the selector can switch different input sources to the output end. The first input end is connected to the APB (Advanced Peripheral Bus) bus for receiving configuration information from the outside. The second input end is connected to the DMA module for receiving data transmitted by the DMA module. The output end is connected to the first end of the FLASH memory for sending the selected data or configuration information to the FLASH memory. The control end is connected to the controller and receives instructions from the controller to switch the input source.

[0032] The DMA module is used to realize direct data transfer between memory and peripherals without CPU intervention, thus improving data transfer efficiency. It reads data from SRAM and writes data to different storage areas in FLASH memory or SRAM. It is configured by the controller through configuration registers, including enabling the DMA module and setting the base address of the DMA module.

[0033] FLASH memory is a non-volatile memory used to store configuration information, image data, etc.

[0034] The configuration register is set to write-only mode and is used to store the configuration information of the DMA module and other necessary configuration parameters.

[0035] The information register is set to read-only mode and stores information about the FLASH memory or other components, such as version information, status information, etc.

[0036] The configuration memory is set to a write-only mode and is used to store the operating configuration or parameters of the image processor.

[0037] The information storage device is set to read-only mode to store image data or processing results, etc.

[0038] The ISP algorithm module is used to process image data, such as denoising, sharpening, color correction, etc. The configuration information and image data required for processing are received from the FLASH memory, and the processed image data can be stored back to the FLASH memory or transmitted to other components.

[0039] In an embodiment of the present application, the ISP algorithm module includes the following algorithm sub-modules: TPG test generator, BLC black level correction module, DPC bad pixel correction module, LSC lens shading correction module, BNRbayer noise reduction module, AWB automatic white balance module, AE automatic exposure module, CFA color interpolation module, GAMMA correction module, CCM color correction module, CSC color space conversion module, and SHARP&CE sharpening and color enhancement module.

[0040] Among them, the TPG test generator is used to generate test images to verify the performance and accuracy of the ISP algorithm module. In the R&D, debugging and testing stages of the ISP algorithm module, the TPG test generator is an indispensable tool.

[0041] The BLC black level correction module shifts the signal output by the image sensor up and down as a whole to eliminate the influence of dark current on the image signal. By subtracting a fixed black level value, the image data is closer to the actual brightness distribution.

[0042] The DPC bad pixel correction module detects and corrects bad pixels in the image sensor to improve image quality. By comparing the difference between a pixel and its surrounding pixels, the bad pixel is identified and replaced or repaired.

[0043] The LSC lens shading correction module corrects the uneven brightness of the image caused by the optical characteristics of the lens. By calculating the brightness compensation values ​​of different areas in the image, the overall or local brightness of the image is adjusted.

[0044] The BNR Bayer noise reduction module reduces the noise in the image and improves the image clarity. It uses a low-pass filter (such as mean filter, Gaussian filter, etc.) to smooth the image and eliminate noise.

[0045] The AWB automatic white balance module automatically adjusts the image color so that white objects appear white in any lighting condition. It estimates the color temperature of the light source by analyzing the color distribution in the image, especially the gray and white areas, and adjusts the intensity of the red, green and blue channels accordingly.

[0046] The AE automatic exposure module automatically determines the correct exposure to control the brightness of the image. By measuring the light distribution in the image, it calculates the ideal exposure settings (such as aperture size, shutter speed and ISO value).

[0047] The CFA color interpolation module converts the original image data in Bayer format into a full-color image and calculates the color value of each pixel using an interpolation algorithm (such as bilinear interpolation, bicubic interpolation, etc.).

[0048] The GAMMA correction module adjusts the brightness of the image nonlinearly to meet the visual characteristics of the human eye. It maps input values ​​of different brightness ranges to corresponding output values ​​through table lookup or formula calculation.

[0049] The CCM color correction module corrects the color error caused by color penetration between color blocks at the filter plate. The standard image is compared with the image obtained by the sensor to obtain a correction matrix, and the matrix is ​​applied to perform color correction on the image.

[0050] The CSC color space conversion module converts an image from one color space to another color space and processes the image data using color space conversion formulas (such as RGB to YUV conversion).

[0051] The SHARP&CE sharpening and color enhancement module improves the clarity and details of the image, while enhancing the color saturation and contrast of the image. The algorithm amplifies the edge or direction information of the brightness difference between adjacent areas, and adjusts the intensity of the color channel to achieve sharpening and color enhancement.

[0052] SRAM is a static random access memory used to temporarily store data during the image processor processing process. It includes a first storage area, a second storage area, a third storage area, and a fourth storage area, which are used for different data transmission and storage tasks.

[0053] See also Figure 4 The operation flow diagram of the image processor shown includes:

[0054] S401, the controller instructs the selector to connect the first input end and the output end through the control end of the selector, then receives the configuration information on the external APB bus, and configures the configuration register using the received configuration information, enables the DMA module and configures the base address of the DMA module.

[0055] The controller connects the APB bus to the configuration register of the FLASH memory through a selector. The configuration information on the APB bus is transferred to the configuration register. The controller uses this configuration information to configure the DMA module, including enabling the DMA module and setting the base address of the DMA module.

[0056] S402, when the controller receives an external vertical synchronization signal, it instructs the selector to connect the second input terminal and the output terminal, instructs the DMA module to write the information in the first storage area into the configuration register, reads the data in the information register into the second storage area, writes the data in the third storage area into the configuration memory, and writes the data in the information memory into the fourth storage area.

[0057] The controller receives a vertical synchronization signal, which indicates the beginning of a new frame of image data. The controller connects the DMA module to the FLASH memory through a selector.

[0058] The DMA module starts working and performs the following operations: Write the data in the first storage area of ​​the SRAM (for example, new configuration information) to the configuration register of the FLASH memory. Read data from the information register of the FLASH memory and store it in the second storage area of ​​the SRAM for subsequent processing or transmission. Write the data in the third storage area of ​​the SRAM (for example, the operating configuration or parameters of the image processor) to the configuration memory of the FLASH memory. Write the data in the information memory of the FLASH memory (for example, image data or processing results) to the fourth storage area of ​​the SRAM as a backup or data source for subsequent processing.

[0059] In the embodiment of the present application, the configuration register is used to store: the enabling information of each algorithm submodule, the correction threshold of BLC, the correction matrix configuration of CCM, the sharpness / color saturation / color contrast of SHARP&CE. The information register is used to store the indication information of whether the operation of each algorithm submodule is completed, the indication information of whether there is an error, the number of white points of AWB, and the regional average brightness of AE. The configuration memory is used to store: the location information of DPC bad points, and the correction coefficient of each partition of LSC. The information memory is used to store: histogram information, the number of pixel y values ​​divided into 0 to 255.

[0060] Among them, in the image processor, the register is an important component for storing configuration information, status information and processing results. According to the technical solution you provided, the configuration register, information register, configuration memory and information memory each have different functions. The following is a detailed division and explanation of the functions of these registers:

[0061] The configuration register is mainly used to store the configuration information of each algorithm submodule of the image processor, which determines how the image processor processes image data. Specifically, it includes:

[0062] Enable information of each algorithm submodule: determines which algorithm submodules (such as denoising, sharpening, color correction, etc.) will be enabled. By configuring this information, you can flexibly select the image processing algorithm to be applied to meet different image processing requirements.

[0063] BLC (Black Level Correction) correction threshold: Black level correction is used to adjust the black level of image data to eliminate black offset in the image. The correction threshold defines which pixel values ​​should be considered as black level and corrected accordingly.

[0064] Correction matrix configuration of CCM (Color Correction Matrix): The color correction matrix is ​​used to adjust the color of the image to make it more accurate and realistic. The correction matrix configuration defines the mapping relationship from the original color space to the target color space.

[0065] SHARP&CE (Sharpening and Color Enhancement) Sharpness / Color Saturation / Color Contrast: used to adjust the sharpness, color saturation and color contrast of the image to improve the visual effect and clarity of the image.

[0066] The information register is mainly used to store the status information of each module of the image processor. The status information reflects the current working status and processing results of the image processor. Specifically, it includes:

[0067] Information indicating whether each algorithm submodule has completed operation: indicates whether each algorithm submodule has completed processing of image data. By reading this information, the processing progress and status of the image processor can be understood.

[0068] Error indication information: Indicates whether an error or abnormality occurred during the processing. By reading this information, errors can be discovered and handled in a timely manner to ensure the normal operation of the image processor.

[0069] AWB (Automatic White Balance) White Point Count: Automatic white balance is used to adjust the white balance of an image to make it more natural and realistic. The white point count indicates the number of white points detected during the automatic white balance process, which helps determine the degree of white balance adjustment.

[0070] AE (Automatic Exposure) Zone Average Brightness: Automatic exposure is used to adjust the exposure of an image to make it brighter and clearer. Zone average brightness represents the average brightness value of each zone during the automatic exposure process, which helps determine the degree of exposure adjustment.

[0071] The configuration memory is mainly used to store the long-term configuration information of the image processor, which is still valid after the image processor is restarted. Specifically, it includes:

[0072] Dead pixel correction (DPC) dead pixel position information: Dead pixel correction is used to eliminate dead pixels in an image and improve image quality. Dead pixel position information records the location of dead pixels in an image so that correction can be performed during image processing.

[0073] Correction coefficient for each partition of LSC (Lens Shading Correction): Lens Shading Correction is used to eliminate the problem of uneven image brightness caused by lens shading effect. The correction coefficient for each partition defines the degree of brightness correction required for different areas.

[0074] The information memory is mainly used to store the temporary status information and processing results of the image processor, which are read and used during or after the image processing. Specifically, it includes:

[0075] Histogram information: Histograms are used to represent the brightness distribution or color distribution of image data. By reading the histogram information, you can understand the brightness statistics and color distribution characteristics of the image, which is helpful for further image processing and analysis.

[0076] The number of pixel y values ​​divided into 0 to 255: This indicates the number of pixels corresponding to each brightness value (0 to 255) in the image. By reading this information, you can understand the brightness distribution of the image and the statistical characteristics of the pixel values.

[0077] S403: After the controller detects that the DMA module completes the read and write operations, it instructs the selector to connect the first input terminal and the output terminal.

[0078] After the DMA module completes the read and write operations, it sends a completion signal to the controller. After the controller detects the completion signal from the DMA module, it reconnects the APB bus to the configuration register of the FLASH memory through the selector. The system returns to the initial state, waiting for the arrival of the next frame of image data and the reception of new configuration information.

[0079] See also Figure 5 , Figure 5 The working timing diagram of the image processor. The high level of VSYNC (vertical synchronization signal) indicates that the image frame is valid, and the low level indicates that the image frame is invalid. The high level of HREF indicates that the image line is valid, and the low level indicates that the image line is invalid. Time T1 indicates the time interval from the image frame valid to the first image line valid of image frame Frame1, time T2 indicates the time interval from the first image line valid to the invalid image frame of image frame Frame1, and time T3 indicates the image frame interval. Time T4 is consistent with time T1, and time T4 is consistent with time T2, belonging to the next frame Frame2. Generally, time T1 is relatively long, at the ms level. The DMA module will be triggered when the rising edge of VSYNC arrives, and then perform related operations. After all operations are completed, a completion interrupt is generated. The software can read and write sram_1 and sram_3 through the controller in the graphics processor in time T2 to perform related operations and prepare the configuration information of the next frame, such as white balance, exposure and GAMMA.

[0080] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0081] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processor, characterized in that: include: Selector, DMA module, FLASH memory and algorithm module; The FLASH memory includes a configuration register, an information register, a configuration memory and an information memory, the configuration register and the configuration memory are set to a write-only mode, and the information register and the information memory are set to a read-only mode; The image processor is provided with an SRAM outside, and the SRAM is provided with a first storage area, a second storage area, a third storage area and a fourth storage area; The first input end of the selector is connected to the APB bus outside the image processor, the second input end of the selector is connected to the DMA module, the output end of the selector is connected to the first end of the FLASH memory, the second end of the FLASH memory is connected to the ISP algorithm module, and the control end of the selector is connected to the controller in the graphics processor; The working process of the graphics processor includes: The controller instructs the selector to conduct the first input end and the output end through the control end of the selector, then receives configuration information on the external APB bus, and configures the configuration register using the received configuration information, enables the DMA module, and configures the base address of the DMA module; When the controller receives an external vertical synchronization signal, it instructs the selector to conduct the second input end and the output end, instructs the DMA module to write the information in the first storage area into the configuration register, read the data in the information register into the second storage area, write the data in the third storage area into the configuration memory, and write the data in the information memory into the fourth storage area; After the controller detects that the DMA module completes the read and write operations, it instructs the selector to connect the first input end and the output end.

2. The method according to claim 1, characterized in that The ISP algorithm module includes the following algorithm sub-modules: TPG test generator, BLC black level correction module, DPC bad pixel correction module, LSC lens shading correction module, BNR bayer noise reduction module, AWB automatic white balance module, AE automatic exposure module, CFA color interpolation module, GAMMA correction module, CCM color correction module, CSC color space conversion module, and SHARP&CE sharpening and color enhancement module.

3. The method according to claim 1 or 2, characterized in that: The DMA module transmits data via the AHB bus.

4. The method according to claim 3, characterized in that: The addresses of two storage units having a mapping relationship are the same.

5. The method according to claim 1, 2 or 4, characterized in that: The configuration register is used to store: the enabling information of each algorithm submodule, the correction threshold of BLC, the correction matrix configuration of CCM, the sharpness / color saturation / color contrast of SHARP&CE; The information register is used to store: indication information of whether the operation of each algorithm submodule is completed, indication information of whether there is an error, the number of white points of AWB, and the average brightness of the area of ​​AE; The configuration memory is used to store: the location information of the DPC bad point and the correction coefficient of each partition of the LSC; The information storage device is used to store: histogram information, and the number of pixel point y values ​​divided into 0-255.

6. A method for operating an image processor, characterized in that: include: The controller instructs the selector to conduct the first input end and the output end through the control end of the selector, then receives the configuration information on the external APB bus, and configures the configuration register using the received configuration information, enables the DMA module, and configures the base address of the DMA module; When the controller receives an external vertical synchronization signal, it instructs the selector to conduct the second input end and the output end, instructs the DMA module to write the information in the first storage area into the configuration register, read the data in the information register into the second storage area, write the data in the third storage area into the configuration memory, and write the data in the information memory into the fourth storage area; After the controller detects that the DMA module completes the read and write operations, it instructs the selector to connect the first input end and the output end.

7. An electronic device, characterized in that: include: SRAM and the image processor as claimed in any one of claims 1 to 5.

Images