Photoelectric sensing and computing integrated light field intelligent imaging method and device

Abstract

The invention provides a photoelectric sensing and computing integrated light field intelligent imaging method and device, and the method comprises the steps: enabling a light signal of a to-be-imaged scene or object to pass through an optical calculation module, and obtaining an optical calculation result, wherein the optical calculation result is light field imaging of the scene or the object to be imaged; obtaining an optical calculation result, and calculating the optical calculation result through electrical calculation operation to obtain an electrical calculation result; judging whether cascading is needed or not according to the electrical calculation result, and if yes, taking the electrical calculation result of the previous level as the input of the optical calculation module of the next level to form a cascading structure; and if not, outputting a final result. Light from an object or a scene to be imaged is used for light calculation, simple electric calculation is achieved by designing an analog circuit, additional analog-to-digital conversion and storage links are not needed, information loss caused by the photoelectric conversion process is avoided, therefore, different types of computer vision tasks can be efficiently completed with low power consumption in a sensing and calculation integrated mode.

Description

technical field [0001] The invention relates to the technical field of light field visual processing, in particular to a light field intelligent imaging method and device integrating photoelectricity, inductance and computing. Background technique [0002] In recent years, the development of deep neural networks has brought great progress to the field of light field vision processing, and has been implemented in many tasks, such as light field image recognition and object segmentation. In actual systems, in order to run artificial intelligence algorithms based on deep neural networks, specific hardware devices are required, such as graphics processing units (GPUs), embedded neural network processing units (NPUs) and FPGAs, etc., but these devices are faced with the following The problem. 1) For example, the high energy consumption and high bandwidth of the GPU make it unsuitable for deployment on edge computing smart terminals (such as commercial cameras, self-driving cars)...

AI Technical Summary

Problems solved by technology

[0008] 1) In the current optoelectronic computing technology, the processing technology of the waveguide-based scheme is complex, and the number of required components and the task dimension have a quadratic growth relationship. More complex tasks correspond to too many components, resulting in high processing costs. At the same time, noise and crosstalk It also becomes more serious with the increase of components, and it is difficult to apply to complex tasks

Solutions based on spatial light propagation basically use coherent light (laser), while most light sources in human life are incoherent light. Using coherent light for optoelectronic computing requires an additional light source, which is not conducive to deployment in edge computing devices

[0009] 2) In the current research on light field intelligent imaging, the optical signal from the scene or object to be imaged only passes through a single optical layer to complete information encoding or modulation, and all calculation and reconstruction tasks at the back end are completed by electronic calculation (usually neural network) , requires higher power consumption

On the other hand, such methods are still based on the framework of imaging first and then processing, which requires analog-to-digital conversion and additional storage links, which is not conducive to the realization of the sensor-computing integrated architecture.

Images