Camera neural network chip architecture system
Through the camera neural network chip architecture system, the integrated memory and computing design is adopted to realize the parallel processing and synchronous display of camera data, solving the problems of high power consumption and high cost in the existing technology, and achieving the effect of low power consumption and low cost.
Patent Information
- Application Number
- CN202510474832.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2025-08-01
AI Technical Summary
The existing camera data processing system has a single function, and the irregular hardware layout leads to high power consumption and high costs.
The architecture design of camera interface, bus, memory, main control circuit, first and second neural network processors and clocks is adopted to realize the computing architecture of integrated storage and computing, and the target detection and noise reduction enhancement data are formed through parallel processing, and the data is displayed synchronously.
It realizes efficient data processing with low power consumption and low cost, improves the synchronization and accuracy of data processing, and reduces additional storage requirements.
Smart Images
Figure CN120409577A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a camera neural network chip architecture system. Background Art
[0002] Nowadays, with the wide application of cameras, systems for processing camera shooting data have emerged and developed vigorously.
[0003] However, these camera data processing systems on the market generally have the defect of single function. And due to the irregular arrangement of internal hardware in the processing system, the camera data generated by the camera needs to be stored additionally, which greatly increases the power consumption and raises the cost.
[0004] Therefore, there is an urgent need for a chip processing architecture for camera data, which can reasonably arrange the structures of various hardware components and thus achieve low power consumption and low cost. Summary of the Invention
[0005] The present invention provides a camera neural network chip architecture system, which effectively solves the above problems existing in the prior art.
[0006] Specifically, the present invention provides a camera neural network chip architecture system, which includes: a camera interface; a bus connected to the camera interface to export camera data from the camera through the bus; a memory connected to the bus to temporarily store the exported camera data in the memory; a main control circuit, the bus is connected to the main control circuit, and the main control circuit issues instructions to the memory through the bus to call the camera data temporarily stored in the memory; a first neural network processor, the bus is connected to the first neural network processor, and the main control circuit calls the camera data from the memory to the first neural network processor, and the first neural network processor performs noise reduction and enhancement processing on the camera data to form the final noise reduction and enhancement data; a second neural network processor, the bus is connected to the second neural network processor, and the main control circuit 103 calls the camera data from the memory to the second neural network processor, and the second neural network processor performs target detection on the camera data to form the final target detection data; a clock respectively communicatively connected to the first neural network processor and the second neural network processor, used to sense the generation moments of the final noise reduction and enhancement data and the final target detection data respectively, so as to ensure that the final noise reduction and enhancement data and the final target detection data are sent to the main control circuit simultaneously, and the main control circuit displays these two kinds of data on the system background screen simultaneously after receiving the final noise reduction and enhancement data and the final target detection data simultaneously.
[0007] Preferably, the first neural network processor includes a pre-processing logic firmware module, a post-processing logic firmware module, and a denoising and enhancement firmware module. The camera data called from the memory is transmitted via a bus to the pre-processing logic firmware module for initial data processing to form initial processed data. The initial processed data is then sent to the denoising and enhancement firmware module for noise reduction and enhancement to form denoising and enhancement intermediate data. The denoising and enhancement data is then sent to the post-processing logic firmware module, and the post-processing logic firmware module 104b performs post-data processing to form denoising and enhancement final data.
[0008] Preferably, the clock records the generation time t1 of the denoising and enhancement final data and records the generation time t2 of the target detection final data. If the generation time t1 is earlier than the generation time t2, the denoising and enhancement final data will wait for a duration of t2 - t1 and be sent to the main control circuit from the first neural network processor and the second neural network processor simultaneously with the target detection final data.
[0009] Preferably, the clock records the generation time t1 of the denoising and enhancement final data and records the generation time t2 of the target detection final data. If the generation time t1 is later than the generation time t2, the target detection final data will wait for a duration of t1 - t2 and be sent to the main control circuit from the first neural network processor and the second neural network processor simultaneously with the denoising and enhancement final data.
[0010] In short, the present invention provides a camera neural network architecture system. Through a bus connected to the camera interface, two neural network processors are connected in parallel, thereby processing the original camera data to respectively form target detection data and denoising and enhancement data to describe the positioning and shape of the object being photographed, and further providing a clock to ensure that these two types of data can be simultaneously displayed on the background screen, thereby enabling synchronous data processing. This system is a computing architecture that integrates memory and computing, and no additional storage is required during the computing process. Therefore, it has the characteristics of high performance, low power consumption, and low cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will discuss the drawings required for use in the description of the embodiments or the prior art. Obviously, the technical solutions described in conjunction with the drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments and their drawings can be obtained based on these embodiments shown in the drawings.
[0012] Figure 1 Shows the overall architecture schematic diagram of the camera neural network chip architecture system according to the present invention.
[0013] Figure 2The figure shows the general structure of the first neural network processor in the camera neural network architecture chip system provided by the present invention. Detailed implementation manners
[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments described in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
[0015] The present invention provides a camera neural network chip architecture system. By effectively arranging the structures of various hardware components, a computing architecture that integrates storage and computing is realized. During the computing and processing process, no additional storage is required. Therefore, it has the characteristics of high performance, low power consumption, and low cost. The system provided by the present invention will be described in detail below.
[0016] Figure 1 The figure shows a schematic diagram of the overall architecture of the camera neural network chip architecture system according to the present invention.
[0017] As Figure 1 shown, the bus 101 is connected to the camera interface 100, and thus camera data is exported from the camera through the bus 101. The bus 101 is also connected to the memory 102, and the exported camera data is temporarily stored in the memory 102. In addition, the bus 101 is also connected to the main control circuit 103, and the main control circuit 103 issues an instruction to the memory 102 through the bus 101 to call the camera data temporarily stored in the memory 102.
[0018] [[ID=XX]]The bus 101 is also connected to the first neural network processor 104. The main control circuit 103 calls the camera data from the memory to the first neural network processor 104, and the first neural network processor 104 performs noise reduction and enhancement processing on the camera data to form the final noise reduction and enhancement data.
[0019] Specifically, the first neural network processor 104 includes a pre-processing logic firmware module 104a, a post-processing logic firmware module 104b, and a denoising and enhancement firmware module 104c. The camera data called from the memory 102 is transmitted through the bus 101 to the pre-processing logic firmware module 104a for initial data processing to form initial processed data. The initial processed data is then sent to the denoising and enhancement firmware module 104c for noise reduction and enhancement to form denoising and enhancement intermediate data. The denoising and enhancement data is then sent to the post-processing logic firmware module 104b, and the post-processing logic firmware module 104b performs post-processing on the data to form the final denoising and enhancement data. As Figure 2 shown, Figure 2The general structure of the first neural network processor in the camera neural network architecture system provided by the present invention is shown.
[0020] The bus 101 is also connected to the second neural network processor 105. The main control circuit 103 calls the camera data from the memory to the second neural network processor 105, and the second neural network processor 105 performs target detection on the camera data to form the final target detection data.
[0021] Preferably, a full-frame cooperative processing mechanism is introduced between the first neural network processor 104 and the second neural network processor 105. Specifically, the traditional line scanning mode is abandoned, and the two neural network processors perform synchronous processing on the data in a full-frame manner. The main control circuit 103 distributes the complete image frame in the camera data temporarily stored in the memory 102 to the dual processors, that is, the first neural network processor 104 and the second neural network processor 105. The first neural network processor 104 realizes the dynamic range expansion of the complete image frame based on global illumination analysis. For example, the dark part details are enhanced by regional gamma correction, and the cross-pixel joint filtering algorithm is used to eliminate noise. The actual measurement shows that the dark field signal-to-noise ratio is increased by 18%, and the detail restoration degree of the highlight overexposed area is increased by 22%. The second neural network processor 105 uses the full-frame context information of the complete image frame for target positioning, significantly improving the recognition accuracy in the occlusion scenario.
[0022] It should be noted that the key of the final target detection data lies in positioning the object captured by the camera, while the final noise reduction and enhancement data focuses on correcting the morphological image of the object itself. Thus, through this system, by combining the two types of data, the position and shape of the captured object can be determined.
[0023] However, the combination of the position data and the shape data is crucial in that the position data and the shape data can be transmitted simultaneously. In other words, the simultaneity of receiving and displaying the two types of data is of great importance.
[0024] Therefore, the present invention also sets a clock 106, which is communicatively connected to the first neural network processor 104 and the second neural network processor 105 respectively, so as to sense the generation moments of the final noise reduction and enhancement data and the final target detection data respectively, thereby ensuring that the final noise reduction and enhancement data and the final target detection data are sent to the main control circuit 103 simultaneously.
[0025] Specifically, the clock 106 records the generation time t1 of the denoising-enhanced final data and the generation time t2 of the target detection final data. If the generation time t1 is earlier than the generation time t2, the denoising-enhanced final data will wait for a duration of t2 - t1 and be sent to the main control circuit 103 from the first neural network processor 104 and the second neural network processor 105 simultaneously with the target detection final data; on the other hand, if the generation time t1 is later than the generation time t2, the target detection final data will wait for a duration of t1 - t2 and be sent to the main control circuit 103 from the first neural network processor 104 and the second neural network processor 105 simultaneously with the denoising-enhanced final data.
[0026] After receiving the denoising-enhanced final data and the target detection final data simultaneously, the main control circuit 103 displays these two types of data on the system background screen simultaneously, thereby ensuring that the positioning and shape of the captured object are displayed on the background screen simultaneously and without delay.
[0027] In addition, preferably, an efficient operation system for the AI processing data format FP8 can also be introduced into the overall architecture. An FP8 full-link data processing pipeline is constructed in the overall architecture. Specifically, the first neural network processor 104 is further provided with a quantization / anti-quantization module, which dynamically quantizes the camera data as 12-bit RAW data into the FP8 format, and the core operations of denoising and target detection (such as convolution and pooling) are all completed by an FP8 fixed-point accelerator. The second neural network processor 105 realizes high-precision data reconstruction through a combination of a look-up table and linear interpolation. Comparative experiments show that this design reduces the on-chip cache requirement by 33%, and the overall energy efficiency ratio is 1.8 times better than the traditional scheme. In addition, the second neural network processor 105 can also be provided with an AI processing module, and the weight parameters are solidified in the non-volatile memory (memory 102) adjacent to the calculation. This AI processing module realizes in-situ calculation in the memory during its AI inference process, the single-task response speed is increased by 35%, and the data transfer energy consumption is reduced by 55%.
[0028] So far, the basic architecture of the present invention has been introduced. As mentioned above, in the camera neural network architecture system provided by the present invention, two neural network processors are connected in parallel through a bus connected to the camera interface, thereby processing the original camera data, respectively forming target detection data and denoising-enhanced data to describe the positioning and shape of the captured object, and further providing a clock to ensure that these two types of data can be displayed on the background screen simultaneously, thus enabling synchronous data processing. This system is a computing architecture that integrates memory and computing, and no additional storage is required during the computing process, so it has the characteristics of high efficiency, low power consumption, and low cost.
[0029] The above are only exemplary embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of protection of the present invention.
Claims
1. A camera neural network chip architecture system, characterized in that, The system includes: A camera interface; A bus, which is connected to the camera interface, so that camera data is exported from the camera through the bus; A memory, which is connected to the bus, so that the exported camera data is temporarily stored in the memory; A main control circuit, the bus is connected to the main control circuit, and the main control circuit issues an instruction to the memory through the bus to call the camera data temporarily stored in the memory; A first neural network processor, the bus is connected to the first neural network processor, the main control circuit calls the camera data from the memory to the first neural network processor, and the first neural network processor performs noise reduction and enhancement processing on the camera data to form noise reduction and enhancement final data; A second neural network processor, the bus is connected to the second neural network processor, the main control circuit calls the camera data from the memory to the second neural network processor, and the second neural network processor performs target detection on the camera data to form target detection final data; A clock, which is respectively communicatively connected to the first neural network processor and the second neural network processor, and is used to sense the generation time of the noise reduction and enhancement final data and the target detection final data respectively, so as to ensure that the noise reduction and enhancement final data and the target detection final data are sent to the main control circuit at the same time. After receiving the noise reduction and enhancement final data and the target detection final data at the same time, the main control circuit displays these two types of data on the system background screen at the same time.
2. The system according to claim 1, wherein The first neural network processor includes a pre-processing logic firmware module, a post-processing logic firmware module, and a denoising and enhancement firmware module. The camera data called from the memory is transmitted through the bus to the pre-processing logic firmware module for initial data processing to form initial processing data. The initial processing data is then sent to the denoising and enhancement firmware module for noise reduction and enhancement to form noise reduction and enhancement intermediate data. The noise reduction and enhancement data is then sent to the post-processing logic firmware module, and the post-processing logic firmware module 104b performs post-data processing to form noise reduction and enhancement final data.
3. The system according to claim 1, wherein The clock records the generation time t1 of the noise reduction and enhancement final data, and records the generation time t2 of the target detection final data. If the generation time t1 is earlier than the generation time t2, the noise reduction and enhancement final data will wait for a duration of t2 - t1 to be sent to the main control circuit from the first neural network processor and the second neural network processor respectively at the same time as the target detection final data.
4. The system according to claim 1, wherein The clock records the generation time t1 of the noise reduction and enhancement final data, and records the generation time t2 of the target detection final data. If the generation time t1 is later than the generation time t2, the target detection final data will wait for a duration of t1 - t2 to be sent to the main control circuit from the first neural network processor and the second neural network processor respectively at the same time as the noise reduction and enhancement final data.