An intelligent remote sensing target detection and recognition system based on an optical deep neural network

Abstract

An intelligent remote sensing target detection and recognition system based on optical deep neural networks replaces the computational functions of traditional electronic devices by controlling the light field of remote sensing images, enabling the optical system to directly possess the functions of deep neural networks for image processing, classification, and recognition. The all-optical deep neural network intelligent remote sensing detection technology operates in the form of light and has outstanding characteristics such as all-optical computation, extremely simple hardware implementation, instantaneous processing of massive amounts of data, and extremely low power consumption. The all-optical deep neural network allows the intelligent remote sensing payload system to replace traditional electronic devices with all-optical computation in its overall design concept, greatly reducing the weight of the remote sensing system and fundamentally solving the dependence of intelligent target recognition of remote sensing payloads on massive computational resources, providing a practical and feasible technical route for the realization of on-orbit intelligence in remote sensing systems.

Description

Technical Field

[0001] This invention relates to an intelligent remote sensing target detection and recognition system based on optical deep neural networks, belonging to the fields of all-optical computing, deep learning, and optical remote sensing technology. Background Technology

[0002] Artificial intelligence (AI), as one of the most active research directions in information science today, has important applications in the aerospace field. Artificial neural networks (ANNs), as the most important model in AI, are widely used in various scenarios due to their excellent generalization ability and robustness. ANNs mimic the structure of the nervous system, establishing connections between neurons in different layers. Integrated circuit chips are the hardware carriers for training and testing mainstream neural network models; traditional neural networks can run on CPUs, GPUs, FPGAs, and application-specific integrated circuits (ASICs). However, regardless of the IC chip used as the carrier, the von Neumann architecture it employs separates the program space from the data space, resulting in a large amount of tidal data load between memory and computing units. Frequent tidal data reads and writes reduce computing speed and increase power consumption per computation. Currently, the main approaches to improving computational efficiency are increasing integration and in-memory computing. However, these approaches also face significant challenges. On the one hand, continuously shrinking transistor size to increase computational speed is unsustainable, as the shrinking transistor size will lead to increasingly significant quantum effects, making it difficult to further improve transistor efficiency. On the other hand, in-memory computing will face the challenge of large-scale modifications to existing neural network architectures, thereby reducing the portability and compatibility of neural network algorithms suitable for in-memory computing.

[0003] All-optical neural network technology uses photons as the basic carrier for information transmission and processing. Compared to traditional electronic technology, photonics technology has advantages such as high bandwidth, low loss, and high data transmission capacity, and is widely used in communication, imaging, radar, and signal processing. Combining this technology with traditional neural network models can leverage the unique advantages of all-optical technology and is expected to overcome the technical bottlenecks of long latency and high power consumption in traditional electrical neural networks. First, all-optical neural networks adopt an in-memory computing structure, avoiding the tidal data read / write problem of electrical neural networks, thus effectively reducing computational latency while improving computational speed. Second, all-optical neural networks have lower connection link losses, which can effectively improve power efficiency. Moreover, compared with traditional electrical devices, optical devices have greater bandwidth and shorter response time, making them more suitable for high-speed real-time computation of neural networks. In addition, for applications such as real-time intelligent processing of remote sensing images, where the front end is optical sensing, all-optical neural networks can process information directly at the physical layer, thus avoiding the problems of latency, power consumption, and signal-to-noise ratio degradation introduced by photoelectric conversion.

[0004] Existing all-optical neural network technology converts incoherent light field signals into laser pulse signals for optical calculations, or uses methods such as optocouplers. However, remote sensing images acquired in optical remote sensing scenarios are all composed of incoherent light, and there is currently a lack of feasible technical approaches for all-optical deep neural networks to directly and intelligently process incoherent natural light remote sensing images. Summary of the Invention

[0005] The technical problem solved by this invention is: addressing the lack of feasible technical approaches in the current technology for directly and intelligently processing incoherent natural light remote sensing images using fully optical deep neural networks, this invention proposes a remote sensing target intelligent detection and recognition system based on optical deep neural networks.

[0006] The present invention solves the above-mentioned technical problem through the following technical solution:

[0007] A remote sensing target intelligent detection and recognition system based on optical deep neural networks includes an optical lens subsystem, an optical deep neural network module, and an information acquisition module, wherein:

[0008] The optical lens subsystem acquires the optical information of the target and converts it into parallel light. The optical depth neural network module transforms, processes, and extracts the acquired spatial incoherent light information and outputs optical signals. The information acquisition module receives the optical signals and generates the processing results of the spatial incoherent light signals based on the optical signals.

[0009] The optical lens subsystem includes an optical collecting lens and an optical collimating lens group, wherein:

[0010] The optical collecting lens gathers the optical information from the target's optical signal into the optical depth neural network module for detection and recognition. The optical collimating lens group transforms the collected optical signal into parallel light.

[0011] The optical deep neural network module has a two-layer structure, wherein:

[0012] The first layer includes a linear optical operation module and a nonlinear optical operation module. The second layer includes a linear optical operation module, wherein:

[0013] The linear optical operation module includes an optical system and a high-precision optical mask. The linear optical operation module identifies the detection task information, determines the architecture of the optical deep neural network, calculates the input image and expected output image required by the first layer of the linear optical operation module, and calculates the input image and expected output image required by the second layer of the linear optical operation module. Based on the input images and expected output images obtained by each layer of the linear optical operation module, the light field transmittance distribution and phase distribution of the high-precision optical mask of each layer of the linear optical operation module are determined.

[0014] The nonlinear optical operation module uses a nonlinear crystal. A bias voltage is applied to the nonlinear crystal, and the nonlinear crystal forms a moiré lattice under the action of the bias voltage.

[0015] Under the action of a bias voltage, the moiré lattice nonlinearly modulates the incoherent light field intensity information input to the nonlinear optical operation module and outputs a light intensity signal. The light intensity signal has a nonlinear relationship with the input incoherent light field intensity information.

[0016] The number of structural layers within the optical deep neural network module is determined specifically according to the optical detection task.

[0017] The optical lens subsystem receives light field information containing remote sensing target information. After passing through the optical focusing lens and the optical collimating lens group, it is input into the optical depth neural network module in the form of a parallel beam. After optical processing by the linear optical operation module and the nonlinear optical operation module, it outputs an optical signal to the information acquisition module. The optical signal includes the type of remote sensing target, and identification is completed based on the type of remote sensing target.

[0018] Based on the optical depth neural network module, the image is stitched together to obtain the area array optical depth neural network module. The area array optical depth neural network module receives the light field information in the form of parallel beams for detection, and detects the specific location of the remote sensing target in the input image required by the linear optical operation module.

[0019] The advantages of this invention compared to the prior art are:

[0020] This invention provides a remote sensing target intelligent detection and recognition system based on optical deep neural networks. By using optical elements at the speed of light to process incoherent spatial light fields through deep neural networks, the overall design of intelligent remote sensing payload systems can eliminate the need for heavy electronic components, making it possible to realize on-orbit intelligence of remote sensing systems. At the same time, this remote sensing target intelligent detection and recognition system based on optical deep neural network computing adopts a minimalist hardware form, greatly reducing the energy requirements for image processing and computing. Meanwhile, the system's programmability and scalability are greatly improved, and high parallel processing will help solve target recognition problems in more complex scenarios. Attached Figure Description

[0021] Figure 1 A schematic diagram of a remote sensing target intelligent detection and recognition system based on optical deep neural networks provided for the invention;

[0022] Figure 2 A schematic diagram of an optical deep neural network module provided for the invention;

[0023] Figure 3 A schematic diagram of the input image and the expected output image required for the first-layer linear optical operation module provided for the invention;

[0024] Figure 4 A schematic diagram of the input image and the expected output image required for the second-layer linear optical operation module provided for the invention;

[0025] Figure 5 A schematic diagram of the light field transmittance distribution and phase distribution of the optical mask in the first-layer linear optical operation module provided for the invention;

[0026] Figure 6 A schematic diagram of the light field transmittance distribution and phase distribution of the optical mask in the second-layer linear optical operation module provided for the invention;

[0027] Figure 7 A schematic diagram of a moiré lattice provided for the invention;

[0028] Figure 8 A schematic diagram illustrating the nonlinear relationship between the output light intensity signal and the input incoherent light field intensity information provided for the invention;

[0029] Figure 9 A schematic diagram of an optical deep neural network module with a two-layer neural network structure provided for the invention;

[0030] Figure 10 A schematic diagram of intelligent remote sensing target recognition provided for the invention;

[0031] Figure 11 A schematic diagram of intelligent remote sensing target detection provided for the invention; Detailed Implementation

[0032] An intelligent remote sensing target detection and recognition system based on optical deep neural networks utilizes optical components at the speed of light to process incoherent spatial light fields via deep neural networks. This allows the overall design of the intelligent remote sensing payload system to eliminate the need for heavy electronic components, making on-orbit intelligence of the remote sensing system possible. Simultaneously, the use of minimalist hardware significantly reduces the energy requirements for image processing and computation, while greatly improving the system's programmability and scalability. High-parallel processing will help solve target recognition problems in more complex scenarios. The specific system structure is as follows:

[0033] It includes an optical lens subsystem, an optical deep neural network module, and an information acquisition module, among which:

[0034] The optical lens subsystem acquires the optical information of the target and converts it into parallel light. The optical depth neural network module transforms, processes, and extracts the acquired spatial incoherent light information and outputs optical signals. The information acquisition module receives the optical signals and generates the processing results of the spatial incoherent light signals based on the optical signals.

[0035] The optical lens subsystem includes an optical collecting lens and an optical collimating lens group, wherein:

[0036] The optical collecting lens gathers the optical information from the target's optical signal into the optical depth neural network module for detection and recognition, while the optical collimating lens group transforms the collected optical signal into parallel light.

[0037] The optical deep neural network module has a two-layer structure, in which:

[0038] The first layer includes a linear optical operation module and a nonlinear optical operation module. The second layer includes a linear optical operation module, wherein:

[0039] The linear optical operation module includes an optical system and a high-precision optical mask. The linear optical operation module identifies the detection task information, determines the architecture of the optical deep neural network, calculates the input image and expected output image required by the first layer of the linear optical operation module, and calculates the input image and expected output image required by the second layer of the linear optical operation module. Based on the input image and expected output image obtained by each layer of the linear optical operation module, the light field transmittance distribution and phase distribution of the high-precision optical mask of each layer of the linear optical operation module are determined.

[0040] The nonlinear optical operation module uses a nonlinear crystal. A bias voltage is applied to the nonlinear crystal, and the nonlinear crystal forms a moiré lattice under the action of the bias voltage.

[0041] Under the action of a bias voltage, the moiré lattice nonlinearly modulates the incoherent light field intensity information of the input nonlinear optical operation module and outputs a light intensity signal. The light intensity signal has a nonlinear relationship with the input incoherent light field intensity information.

[0042] The number of structural layers within the optical deep neural network module is determined specifically based on the optical detection mission.

[0043] The optical lens subsystem receives light field information containing remote sensing target information. After passing through the optical focusing lens and the optical collimating lens group, it is input into the optical depth neural network module in the form of a parallel beam. After optical processing by the linear optical operation module and the nonlinear optical operation module, the optical signal is output to the information acquisition module. The optical signal includes the type of remote sensing target, and identification is completed according to the type of remote sensing target.

[0044] Based on the optical depth neural network module, the image is stitched together to obtain the area array optical depth neural network module. The area array optical depth neural network module receives the light field information in the form of parallel beams for detection, and detects the specific location of the remote sensing target in the input image required by the linear optical operation module.

[0045] The following is a further explanation based on specific embodiments:

[0046] In the current embodiment, a remote sensing target intelligent detection and recognition system based on optical deep neural network computation, such as... Figure 1 As shown, the remote sensing target intelligent detection and recognition system based on optical deep neural networks includes: an optical lens system, an optical deep neural network module, and an information acquisition module.

[0047] The optical lens system is used to collect the optical information of the target and convert it into parallel light; the optical deep neural network module is used to transform, process and extract the collected spatial incoherent light information; and the information acquisition module is used to receive the output signal of the optical deep neural network module and generate the processing result of the spatial incoherent light signal based on the output signal.

[0048] An optical lens system includes an optical collecting lens and an optical collimating lens group.

[0049] The optical collecting lens gathers the optical information of the target into the remote sensing target intelligent detection and recognition system of optical deep neural network, while the optical collimating lens group transforms the collected optical signal into collimated light.

[0050] The optical deep neural network module includes: a first-layer linear optical operation module, a non-linear optical operation module, and a second-layer linear optical operation module.

[0051] like Figure 2 As shown, the first-layer linear optical operation module, the nonlinear optical operation module, and the second-layer linear optical operation module construct a two-layer deep neural network.

[0052] Specifically, the first-layer linear optical operation module consists of a 4f optical system and a high-precision optical mask, and the second-layer linear optical operation module consists of a 4f optical system and a high-precision optical mask.

[0053] Specifically, the nonlinear optical operation module is composed of a moiré lattice.

[0054] First, based on the recognition and detection task, the architecture of the deep neural network is determined, and the input image required by the first-layer linear optical operation module and the expected output image are calculated, such as... Figure 3 As shown.

[0055] Similarly, based on the recognition and detection task, the architecture of the deep neural network is determined, and the input image and expected output image required by the second-layer linear optical operation module are calculated, such as... Figure 4 As shown.

[0056] Using the input image required by the first-layer linear optical manipulation module and the expected output image, the light field transmittance distribution and phase distribution of the high-precision optical mask in the first-layer linear optical manipulation module are obtained, such as... Figure 5 As shown.

[0057] Using the input image required by the second-layer linear optical manipulation module and the expected output image, the light field transmittance distribution and phase distribution of the high-precision optical mask in the second-layer linear optical manipulation module are obtained, such as... Figure 6 As shown.

[0058] The nonlinear optical manipulation module is composed of a moiré lattice. For example... Figure 7 As shown, a bias voltage is applied to a nonlinear crystal, forming a moiré lattice inside the nonlinear crystal. Under the action of an external bias voltage, the moiré lattice can nonlinearly control the intensity information of the input incoherent optical field. The output light intensity signal exhibits a nonlinear relationship with the input incoherent optical field intensity information, such as... Figure 8 As shown.

[0059] Using the aforementioned linear and nonlinear optical manipulation modules, a multi-layered optical deep neural network module can be constructed, as illustrated in the diagram below. Figure 9 As shown, this is an optical deep neural network module with a two-layer neural network structure.

[0060] A remote sensing target intelligent detection and recognition system based on optical deep neural network computation can be used to identify and classify remote sensing targets, such as... Figure 10 As shown.

[0061] Specifically, the light field information containing remote sensing target information is input into the optical lens system. After passing through the optical focusing lens and the optical collimating lens group, it is input into the optical depth neural network module in the form of a parallel beam. In the module, it undergoes linear optical operations and nonlinear optical operations. The output signal is displayed by the information acquisition module, indicating the type of remote sensing target and completing the identification process.

[0062] A remote sensing target intelligent detection and recognition system based on optical deep neural network computation can be used to identify and detect remote sensing targets, such as... Figure 11 As shown.

[0063] Specifically, the obtained optical depth neural network modules are stitched together to form an area array optical depth neural network module. The light field information containing remote sensing target information is input into the optical lens system. After passing through the optical focusing lens and the optical collimating lens group, it is input into the area array optical depth neural network module in the form of a parallel beam. In the module, linear and nonlinear optical operations are performed, and the output signal is displayed by the information acquisition module to indicate the type of remote sensing target. Based on the detection results in the area array optical depth neural network module, the position of the detected target in the input remote sensing image is obtained, thus completing the identification and detection process.

[0064] The remote sensing target intelligent detection and identification system can design diverse focal plane forms according to the configuration of the optical remote sensing payload system. In addition to the spliced ​​area array form, special focal plane designs can also be performed.

[0065] In this embodiment, the remote sensing target intelligent detection and recognition system based on optical deep neural networks is a detection and recognition system that can directly process spatial incoherent light fields without converting them into coherent ones. It is a novel system oriented towards the characteristics of spatial light fields and particularly suitable for spatial optical remote sensing applications. Furthermore, by constructing multiple optical deep neural network modules and stitching them together to form an array-type optical deep neural network module, it can simultaneously provide the target's position information in the light field image while recognizing the target, thus completing the target detection task.

[0066] In remote sensing target intelligent detection and recognition systems, optical deep neural network modules include linear control modules and nonlinear control modules. Multiple linear control modules and nonlinear control modules can be coupled in a cascade manner to form multi-layer deep neural networks, enabling more complex recognition and detection functions.

[0067] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0068] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A remote sensing target intelligent detection and recognition system based on optical deep neural networks, characterized in that: It includes an optical lens subsystem, an optical deep neural network module, and an information acquisition module, among which: The optical lens subsystem acquires the optical information of the target and converts it into parallel light. The optical depth neural network module transforms, processes, and extracts the acquired spatial incoherent light information and outputs optical signals. The information acquisition module receives the optical signals and generates the processing results of the spatial incoherent light signals based on the optical signals. The optical deep neural network module has a two-layer structure, in which: The first layer includes a linear optical operation module and a nonlinear optical operation module. The second layer includes a linear optical operation module, wherein: The linear optical operation module includes an optical system and a high-precision optical mask. The linear optical operation module identifies the detection task information, determines the architecture of the optical deep neural network, calculates the input image and expected output image required by the first layer of the linear optical operation module, and calculates the input image and expected output image required by the second layer of the linear optical operation module. Based on the input image and expected output image obtained by each layer of the linear optical operation module, the light field transmittance distribution and phase distribution of the high-precision optical mask of each layer of the linear optical operation module are determined. The nonlinear optical operation module uses a nonlinear crystal. A bias voltage is applied to the nonlinear crystal, and the nonlinear crystal forms a moiré lattice under the action of the bias voltage. Under the action of a bias voltage, the moiré lattice nonlinearly modulates the incoherent light field intensity information input to the nonlinear optical operation module and outputs a light intensity signal. The light intensity signal has a nonlinear relationship with the input incoherent light field intensity information. The optical lens subsystem receives light field information containing remote sensing target information. After passing through the optical collecting lens and the optical collimating lens group, it is input into the optical depth neural network module in the form of a parallel beam. After optical processing by the linear optical operation module and the nonlinear optical operation module, it outputs an optical signal to the information acquisition module. The optical signal includes the type of remote sensing target, and identification is completed according to the type of remote sensing target. Based on the optical depth neural network module, the image is stitched together to obtain the area array optical depth neural network module. The area array optical depth neural network module receives the light field information in the form of parallel beams for detection, and detects the specific location of the remote sensing target in the input image required by the linear optical operation module.

2. The remote sensing target intelligent detection and recognition system based on optical deep neural networks according to claim 1, characterized in that: The optical lens subsystem includes an optical collecting lens and an optical collimating lens group, wherein: The optical collecting lens gathers the optical information from the target's optical signal into the optical depth neural network module for detection and recognition. The optical collimating lens group transforms the collected optical signal into parallel light.

Images