Calculation network for space-ground cooperative detection

Abstract

A computing network for space-ground collaborative detection comprises an on-satellite optical computing network and a ground electric computing network, and can directly detect and identify simple targets on a satellite, preprocess complex targets, compress the scale of mass remote sensing data and reduce on-satellite energy consumption and downloading difficulty. And for a simple target identification result and a complex target compression preprocessing result, a compression vector downloaded by an optical network is reconstructed and classified in a ground electric computing network, so that the overall accuracy of the system is improved.

Description

Technical Field

[0001] The present invention relates to a computing network for space-ground collaborative detection, and belongs to the field of intelligent optical computing. Background Art

[0002] With the rapid development and widespread application of artificial intelligence technology, modern satellite earth observation systems have gradually integrated artificial intelligence technology to improve the efficiency and accuracy of data processing and analysis. However, the limitations of traditional computing methods in processing large-scale data are becoming increasingly prominent. Especially in modern satellite earth observation systems, with the increase in the number of satellites and the improvement in sensor resolution, the amount of data is showing an explosive growth trend. This massive amount of data needs to be efficiently processed and analyzed in a short time to extract useful information and support various applications, such as weather forecasting, natural disaster monitoring, agricultural resource management, etc.

[0003] The ability of intelligent recognition and detection depends largely on the complexity of the deep neural network architecture, and the system performance is completely dependent on the scale and performance level of the existing electronic computing platform. However, traditional electronic computing-based methods are often inefficient when processing such large-scale data, and are usually subject to physical limitations such as operating speed, energy consumption, and bandwidth. Especially in space-based scenarios, space-based platforms are constrained by power consumption, volume, and other factors, and the computing resources and energy consumption they can provide are limited, making it difficult to deploy complex models. This leads to bottleneck problems in on-orbit data processing based on traditional electronic devices, and data applications face great challenges; at the same time, at the device implementation level, due to the high probability of mutual interference between multi-layer neural network signals, it brings certain difficulties to the hardware implementation of neural networks that require high-density connections. Summary of the invention

[0004] The technical problem solved by the present invention is: in view of the difficulty in realizing the design and connection of a neural network with high density connection in the current prior art, a computing network for space-ground collaborative detection is proposed.

[0005] The present invention solves the above technical problems by the following technical solutions:

[0006] A computing network for space-ground collaborative detection, including an on-board optical computing network and a ground-based electrical computing network, wherein:

[0007] The on-board optical computing network receives images captured after simple target detection, sequentially performs linear modulation and nonlinear modulation on the simple target detection images, performs fitting processing on the modulated images to obtain simple target fitting images, and uses the simple target fitting images to perform simple target recognition; receives images captured after complex target detection, sequentially performs linear modulation and nonlinear modulation on the complex target detection images, performs fitting processing on the modulated images to obtain complex target fitting images, and uses the complex target fitting images to perform compression preprocessing; and sends simple target recognition results and complex target compression preprocessing results to the ground electrical computing network;

[0008] The ground electronic computing network performs initial feature reconstruction and nonlinear signal reconstruction on simple target recognition results and complex target preprocessing results, and outputs the target reconstructed image.

[0009] The on-board optical computing network includes three linear modulation layers and two nonlinear modulation layers, wherein:

[0010] The first linear modulation layer receives the detected and captured images of simple targets and complex targets, performs transmission-based intensity linear modulation on the detected and captured images to obtain initial feature matrices of the simple targets and complex targets, and sends the linearly modulated images to the first nonlinear modulation layer;

[0011] The first nonlinear modulation layer performs nonlinear modulation on the linearly modulated image and limits the gain value of the nonlinearly modulated image to a preset multiple, and sends the nonlinearly modulated image to the second linear modulation layer;

[0012] The second linear modulation layer extracts features from the nonlinearly modulated image, performs feature compression, and sends the feature-compressed image to the second nonlinear modulation layer;

[0013] The second nonlinear modulation layer performs fitting processing on the feature compressed image and sends it to the third linear modulation layer;

[0014] The third linear modulation layer performs image recognition or image compression preprocessing on the image after fitting processing according to the target type.

[0015] When the target type is a simple target, simple target recognition is performed on the simple target fitting image in the third linear modulation layer to obtain simple target information as a simple target recognition result; the simple target recognition is completed in the on-board optical computing network;

[0016] When the target type is a complex target, the complex target fitting image is compressed and preprocessed in the third linear modulation layer to obtain a complex target compression vector as a complex target compression preprocessing result; the complex target compression vector is subsequently calculated in the ground electrical computing network according to the requirements of the ground-ground collaborative detection mission.

[0017] The first linear modulation layer includes a liquid crystal display and a lens;

[0018] The liquid crystal display receives the images captured by simple target and complex target detection, performs imaging processing, loads the initial feature matrix of the simple target and complex target, and performs linear modulation according to the initial feature matrix;

[0019] The lens performs down-sampling processing on the linear modulated image and outputs the linear modulated image to the first nonlinear modulation layer.

[0020] The first non-linear modulation layer comprises an image intensifier, wherein:

[0021] The image intensifier receives the incident linearly modulated image, realizes nonlinear modulation through the local amplification effect and controls the gain to be 800 times to 3000 times.

[0022] The second linear modulation layer includes a microlens array, a liquid crystal display, and a lens, wherein:

[0023] Microlens array, to replicate the image after nonlinear modulation;

[0024] A liquid crystal display receives all the nonlinearly modulated images after replication and performs transmission intensity modulation, loads the corresponding modulation matrix to modulate the image to achieve feature extraction, and obtains a feature matrix for reflecting different categories of image information for display;

[0025] The lens performs down-sampling and compression processing on the linear modulated image, obtains a linear modulated and feature compressed image whose focus is much smaller than the original image, and outputs it to the second nonlinear modulation layer.

[0026] The second non-linear modulation layer comprises an image intensifier, wherein:

[0027] An image intensifier receives an incident linear modulated image, performs nonlinear modulation and image fitting, and sends the fitted image to a third linear modulation layer;

[0028] The image intensifier of the second nonlinear modulation layer is the same as that of the first nonlinear modulation layer.

[0029] The ground electrical computing network includes a sampling network, an initial reconstruction network and a deep reconstruction network, wherein:

[0030] The sampling network presets a compression sampling matrix, automatically compresses the simple target recognition results and the complex target compression preprocessing results according to the compression sampling matrix, and sends the compressed image to the initial reconstruction network;

[0031] The initial reconstruction network performs feature image conversion on the compressed image according to the image feature requirements of the space-ground collaborative exploration mission, and sends the initial reconstructed feature image to the deep reconstruction network;

[0032] The deep reconstruction network receives the initial reconstructed feature image, performs nonlinear signal reconstruction, and outputs the target reconstructed image.

[0033] The initial reconstruction network uses convolutional layers and Reshape+Concate layers to achieve feature image conversion;

[0034] In the initial reconstruction network structure, the convolution layer is designed as:

[0035] The convolution kernel size is 1*1, the number of convolution kernel channels is 1024, the number of convolution kernels is 16384, the image size after input compression is 2*2, the number of input channels is 1024, the output feature map size is 2*2, and the output channels are 16384;

[0036] The Reshape+Concate layer is used to convert the 16384*2*2 feature image into an initial reconstructed feature image of 1*256*256 and output the deep reconstruction network.

[0037] The deep reconstruction network includes five convolutional layers with the same structure, and each layer is designed as follows:

[0038] The first convolution layer has a kernel size of 3*3, a kernel channel number of 1, a kernel number of 64, a stride of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 1, the output feature map size is 256*256, and the output channel number is 64;

[0039] The second convolution layer has a convolution kernel size of 3*3, 64 convolution kernel channels, 64 convolution kernels, a step size of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 64;

[0040] The third and fourth convolutional layers have the same design as the second convolutional layer;

[0041] The fifth convolution layer has a kernel size of 3*3, 64 kernel channels, 1 kernel number, 1 stride, and 1*1 padding. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 1.

[0042] The advantages of the present invention compared with the prior art are:

[0043] (1) The present invention provides a computing network for space-ground collaborative detection. It uses the parallelism and high speed of optical signals to convert the computing process of neural networks into the transmission and processing of optical signals. It has the characteristics of high bandwidth, high parallelism, high information transmission volume and high speed. A two-stage optical computing network construction and optimization method is proposed. A lightweight optical computing network is deployed on the satellite to handle simple tasks. For complex tasks, the optical neural network can convert massive remote sensing data into compressed vectors and transmit them to the ground processing system, realizing fast and efficient processing of large-scale data in complex scenarios.

[0044] (2) The present invention adopts a two-stage optical computing network for space-ground collaboration. For simple tasks, optical neural networks are directly used on the satellite for processing. For complex tasks, optical neural networks are used for pre-processing and compression before being transmitted to the ground processing system. This greatly reduces the difficulty of transmitting and processing massive data, and improves the resource bottleneck problem of the existing technology that makes it difficult to deploy large-scale networks on the satellite. The optical neural network truly realizes the processing solution of light speed and zero energy consumption. The two-stage network collaborative processing greatly reduces the overall processing difficulty and energy consumption while ensuring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of a two-stage optoelectronic coupling network system for space-ground collaborative detection provided by the present invention;

[0046] Figure 2 A physical implementation diagram of the optical neural network provided by the present invention;

[0047] Figure 3 A diagram of the structure of an optical neural network provided by the present invention;

[0048] Figure 4 A network structure diagram of a ground processing system in a two-stage network provided by the present invention;

[0049] Figure 5 A schematic diagram of the Quickdraw data set used for the simple tasks provided by the present invention;

[0050] Figure 6 Schematic diagram of available benchmark data sets for complex task remote sensing image scene classification provided by the present invention;

[0051] Figure 7 Schematic diagram of the original image, optical network compression vector and reconstructed image provided by the present invention. DETAILED DESCRIPTION

[0052] A computing network for space-ground collaborative detection includes an on-board optical computing network and a ground-based electrical computing network. The network can directly detect and identify simple targets on the satellite, pre-process complex targets, compress the size of massive remote sensing data, and reduce on-board energy consumption and difficulty in downlinking. The network reconstructs and classifies the compressed vectors transmitted from the optical network in the ground-based electrical computing network for simple target recognition results and complex target compression pre-processing results, thereby improving the overall accuracy of the system.

[0053] The on-board optical computing network receives images captured after simple target detection, sequentially performs linear modulation and nonlinear modulation on the simple target detection images, performs fitting processing on the modulated images to obtain simple target fitting images, and uses the simple target fitting images to perform simple target recognition; receives images captured after complex target detection, sequentially performs linear modulation and nonlinear modulation on the complex target detection images, performs fitting processing on the modulated images to obtain complex target fitting images, and uses the complex target fitting images to perform compression preprocessing; and sends simple target recognition results and complex target compression preprocessing results to the ground electrical computing network;

[0054] The ground electronic computing network performs initial feature reconstruction and nonlinear signal reconstruction on simple target recognition results and complex target preprocessing results, and outputs the target reconstructed image.

[0055] The on-board optical computing network consists of three linear modulation layers and two nonlinear modulation layers, where:

[0056] The first linear modulation layer receives the detected and captured images of simple targets and complex targets, obtains the initial feature matrix of the simple targets and complex targets after performing transmission-based intensity linear modulation on the detected and captured images, and sends the linearly modulated images to the first nonlinear modulation layer; wherein the initial feature matrix is ​​the linearly modulated image, and through linear modulation, the features of the captured image can be preliminarily extracted.

[0057] The first nonlinear modulation layer performs nonlinear modulation on the linearly modulated image and limits the gain value of the nonlinearly modulated image to 800 times, and sends the nonlinearly modulated image to the second linear modulation layer;

[0058] The second linear modulation layer extracts features from the nonlinearly modulated image, performs feature compression, and sends the feature-compressed image to the second nonlinear modulation layer;

[0059] The second nonlinear modulation layer performs fitting processing on the feature compressed image and sends it to the third linear modulation layer;

[0060] The third linear modulation layer performs image recognition or image compression preprocessing on the image after fitting processing according to the target type.

[0061] When the target type is a simple target, simple target recognition is performed on the simple target fitting image in the third linear modulation layer to obtain simple target information as a simple target recognition result; the simple target recognition is completed in the on-board optical computing network;

[0062] When the target type is a complex target, the complex target fitting image is compressed and preprocessed in the third linear modulation layer to obtain a complex target compression vector as a complex target compression preprocessing result; the complex target compression vector is subsequently calculated in the ground electrical computing network according to the requirements of the ground-ground collaborative detection mission.

[0063] The first linear modulation layer includes a liquid crystal display and a lens;

[0064] The liquid crystal display receives the images captured by simple target and complex target detection, performs imaging processing, loads the initial feature matrix of the simple target and complex target, and performs linear modulation according to the initial feature matrix;

[0065] The lens performs down-sampling processing on the linear modulated image and outputs the linear modulated image to the first nonlinear modulation layer.

[0066] The first non-linear modulation layer comprises an image intensifier, wherein:

[0067] An image intensifier receives an incident linearly modulated image and performs nonlinear modulation;

[0068] Nonlinear modulation is achieved through the local amplification effect of the microchannel plate and the gain is approximately 800 times, and can reach up to 3000 times.

[0069] The second linear modulation layer includes a microlens array, a liquid crystal display, and a lens, wherein:

[0070] Microlens array, to replicate the image after nonlinear modulation;

[0071] The liquid crystal display receives all the nonlinearly modulated images after replication and performs transmission intensity modulation, loads the corresponding modulation matrix to modulate the image to achieve feature extraction, and obtains a feature matrix that can reflect image information of different categories;

[0072] The lens performs down-sampling and compression processing on the linearly modulated image. The focused image is much smaller than the original image, and the linearly modulated and feature-compressed image is output to the second nonlinear modulation layer.

[0073] The second non-linear modulation layer comprises an image enhancer, wherein:

[0074] An image intensifier receives an incident linear modulated image, performs nonlinear modulation and image fitting, and sends the fitted image to a third linear modulation layer;

[0075] The image intensifier of the second nonlinear modulation layer is the same as that of the first nonlinear modulation layer.

[0076] The ground electrical computing network includes a sampling network, an initial reconstruction network, and a deep reconstruction network, where:

[0077] The sampling network presets a compression sampling matrix, automatically compresses the simple target recognition results and the complex target compression preprocessing results according to the compression sampling matrix, and sends the compressed image to the initial reconstruction network;

[0078] The initial reconstruction network performs feature image conversion on the compressed image according to the image feature requirements of the space-ground collaborative exploration mission, and sends the initial reconstructed feature image to the deep reconstruction network;

[0079] The deep reconstruction network receives the initial reconstructed feature image, performs nonlinear signal reconstruction, and outputs the target reconstructed image.

[0080] The initial reconstruction network uses convolutional layers and Reshape+Concate layers to achieve feature image conversion;

[0081] In the initial reconstruction network structure, the convolution layer is designed as:

[0082] The convolution kernel size is 1*1, the number of convolution kernel channels is 1024, the number of convolution kernels is 16384, the image size after input compression is 2*2, the number of input channels is 1024, the output feature map size is 2*2, and the output channels are 16384;

[0083] The Reshape+Concate layer is used to convert the 16384*2*2 feature image into an initial reconstructed feature image of 1*256*256 and output the deep reconstruction network.

[0084] The deep reconstruction network consists of five convolutional layers with the same structure, and the design of each layer is as follows:

[0085] The first convolution layer has a kernel size of 3*3, a kernel channel number of 1, a kernel number of 64, a stride of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 1, the output feature map size is 256*256, and the output channel number is 64;

[0086] The second convolution layer has a convolution kernel size of 3*3, 64 convolution kernel channels, 64 convolution kernels, a step size of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 64;

[0087] The third and fourth convolutional layers have the same design as the second convolutional layer;

[0088] The fifth convolution layer has a kernel size of 3*3, 64 kernel channels, 1 kernel number, 1 stride, and 1*1 padding. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 1.

[0089] The following is further described in conjunction with the accompanying drawings and preferred embodiments of the specification:

[0090] In the current embodiment, the computing network includes an on-board optical computing network and a ground-based electrical computing network, as shown in the schematic diagram. Figure 1 As shown in the figure, the on-board optical computing network includes three linear layers and two nonlinear layers. The optical network can detect and identify simple targets directly on the satellite, and the processing process consumes zero energy at the speed of light. For complex targets, the network can pre-process the targets, compress the scale of massive remote sensing data, and reduce the energy consumption on the satellite and the difficulty of downlink. The first linear layer first captures the image, performs linear modulation to obtain the initial feature matrix of the image; nonlinear modulation is performed in the first nonlinear layer to highlight the image features and limit the range of output values; it is sent to the second linear modulation layer for further feature extraction and further compresses the image to facilitate subsequent classification and downlink tasks; then the second nonlinear layer is used to increase the expression ability of the model, which can better fit the data; the third linear modulation layer performs different actions according to the difficulty of the task. For simple tasks, linear classification is performed directly, and image recognition is performed according to the output vector. For complex tasks, the image is further compressed to form vector points, reducing the difficulty and energy consumption of downlinking massive remote sensing data. The ground electrical computing network includes a sampling network, an initial reconstruction network, and a deep reconstruction network, which can reconstruct and classify the compressed vectors transmitted by the optical network and improve the overall accuracy of the system.

[0091] The optical computing network forms a five-layer optical neural network through linear control of amplitude modulation and optical nonlinear operation, including three layers of linear modulation layer of optical computing network and two layers of nonlinear modulation layer of optical computing network. Its physical form is as follows: Figure 2 As shown, the network form is as Figure 3 shown.

[0092] The first linear modulation layer consists of a liquid crystal display (LCD) and a lens (len). The captured optical image is imaged onto the LCD for intensity modulation, where the LCD can load the trained modulation matrix. The weight vector W mEach weight value is encoded into the transmittance of the spatial light modulator (SLM) pixel based on intensity modulation, and the transmittance is proportional to the weight value. Each pixel value of the input vector is then overlaid on the corresponding pixel on the SLM, thereby completing the transmissive intensity modulation. Since the input image captured from the environment is incoherent, the intensity of the spatial pattern modulated by the LCD pixel corresponds to the product between two non-negative numbers, as shown in Equation 1:

[0093] Y out (x,y)=X(x,y)·W(x,y)#(1)

[0094] Optical fan-in is completed through a lens, that is, the modulated image is downsampled. Optical fan-in implements the summation step in the vector-vector dot product operation. In the present invention, the optical fan-in focuses the modulation matrix modulated by the LCD to the active area of ​​the optical nonlinearity through a lens, so that the focused image size is smaller than the resolution of the latter, thereby completing the analog summation of each output vector element. This summation method skips the digital circuit summation, reduces the number of pixels, allows the use of smaller detectors for reception, and more importantly, reduces the energy cost of the digital memory used to store intermediate results, thereby greatly reducing energy consumption. Its mathematical expression is as shown in Formula 2:

[0095]

[0096] Optical nonlinearity is also very important in optical neural networks. If there is no nonlinear activation, even if multi-layer matrix linear operations are physically implemented, they are still equivalent to single-layer operations. In the present invention, optical nonlinearity can be implemented by a commercial image intensifier, and the modulated image is focused on the commercial image intensifier. The incident light generates free electrons from the photocathode, which are locally amplified by the microchannel plate (MCP) and then produce new, amplified bright spots when they hit the phosphor screen. The local saturation of MCP amplification leads to a saturated nonlinear response. Although this nonlinear approach is not all-optical, the local on-site implementation retains the spatial parallelism of ONN and avoids the time and energy cost required for readout / input when calculating nonlinear activation on a separate electronic processor. Its nonlinear effect is shown in Equation 3:

[0097] y=a(1-e -bx )+c(1-e -dx )#(3)

[0098] The second linear modulation layer adds an optical fan-out operation before the modulation layer, which is performed through a micro lens array (MLA). Optical fan-out is to replicate the output light signal into multiple copies, and each copy is exactly the same. In the present invention, it is achieved through a micro lens array. A micro lens array is a device for optical imaging and light field replication, and is often used to achieve multi-viewpoint imaging, holographic display, light field photography, etc. It consists of many tiny lenses, each of which can capture and focus light. By reasonably designing the arrangement and parameters of the lenses, multiple perspectives can be obtained simultaneously or the light field can be replicated in a system. The mathematical expression of this operation is shown in Formula 4:

[0099]

[0100] After that, it goes through LCD modulation and lens vector summation, thus completing a fully connected layer modulation using optics and then inputting it into the second nonlinear layer.

[0101] The components of the second optical nonlinear module are the same as those of the first optical nonlinear module, but the number of specific nonlinear modulation nodes is different. Claim 3 is repeated to complete the second nonlinear modulation.

[0102] Simple classification tasks can be completed on board: the last linear modulation layer first uses MLA to fan-out the optical image to four copies, and then uses LCD to modulate the copies separately. After the modulation is completed, a lens is used to perform a fan-in operation, which becomes a 2×2 output light vector. The output light vector can be used for image classification and can also be used for image reconstruction, with a compression ratio of 400:1.

[0103] For complex tasks in space-based scenarios, it is necessary to transmit compressed vectors to the ground reconstruction network for processing. The original image is 40×40 in size and becomes 2×2 after compression.

[0104] The reconstruction network includes a sampling network, an initial reconstruction network, and a deep reconstruction network. The sampling network first compresses and samples the input image, automatically learns the compressed sampling matrix, and avoids complex manual design. The initial reconstruction network uses a convolutional layer and a Reshape+Concate layer to simulate the initial reconstruction process, which is the minimum mean square error linear estimation in the traditional Block Compressed Sensing (BCS) reconstruction. The deep reconstruction network implements the nonlinear signal reconstruction process and contains a five-layer convolutional network. The network structure is as follows: Figure 4 shown.

[0105] The structure of the sampling network: convolution kernel size 32×32, number of convolution kernel channels 1, number of convolution kernels 1024, step size 32×32, padding 0. The input feature map size of the sampling network is 64×64, the number of input channels is 1, the output feature map size is 2×2, and the number of output channels is 1024.

[0106] Initial reconstruction network structure: convolution kernel size 1×1, number of convolution kernel channels 1024, number of convolution kernels 16384, step size 1×1, padding 0. The input feature map size of the initial reconstruction network is 2×2, the number of input channels is 1024, the output feature map size is 2×2, and the output channels are 16384.

[0107] The Reshape+Concate layer converts the feature map of size 16384×2×2 into a feature map of size 1×256×256, and then connects it to the deep reconstruction network.

[0108] Initial reconstruction network structure: convolution kernel size 1×1, number of convolution kernel channels 1024, number of convolution kernels 16384, step size 1×1, padding 0. The input feature map size of the initial reconstruction network is 2×2, the number of input channels is 1024, the output feature map size is 2×2, and the output channels are 16384.

[0109] The Reshape+Concate layer converts the feature map of size 16384×2×2 into a feature map of size 1×256×256, and then connects it to the deep reconstruction network.

[0110] Example:

[0111] In order to verify the computational network, the Quickdraw dataset and the remote sensing image scene classification available benchmark dataset created by Northwestern Polytechnical University were used as simple and complex tasks to verify the performance of the optical neural network and the two-stage network, respectively.

[0112] For simple tasks such as the Quickdraw dataset, Figure 5As shown in the figure, the data set is directly processed by the optical neural network deployed on the satellite, which can reach the upper limit of the accuracy of the electronic multi-layer perceptron (MLP), ensuring the accuracy of the optical neural network under extremely low energy consumption. Its physical implementation process can be described as loading the 40×40 Quickdraw image onto the digital micromirror device (DMD) to form an input light field of incoherent broadband illumination, and upsampling the image to 400×400 through the first lens to more completely adjust the input light field. The light field is multiplied by the LCD to complete the through-type intensity adjustment. At this time, the trained modulation matrix loaded by the LCD is 400*400. After the second lens, it is downsampled to a size of 40×40, and the image size is still 40×40 after the nonlinear activation of the MCP. The second linear modulation layer first uses MLA to optically fan-out the optical image to 16 copies, and then uses LCD to modulate the copies respectively. After the modulation is completed, the lens is used to perform fan-in operation. At this time, the image scale becomes 4×4, and after nonlinearity, it passes through the linear modulation layer again to become a 2×2 output light vector. The output light vector can be used for image classification and image reconstruction, and the compression ratio is 400:1. After testing, the optical neural network proposed in the present invention has a direct classification accuracy of 85% for simple tasks, which is similar to the accuracy of the electronic MLP network.

[0113] The two-stage space-ground collaborative optoelectronic coupling network is more suitable for complex tasks in space-based scenarios, such as Figure 6 As shown in the figure, this is the dataset used, so the available benchmark dataset for remote sensing image scene classification is used for verification. The image size of this dataset is 256×256, so the optical network first compresses the image size to 64×64 before transmitting it to the ground processing system. The compression ratio is 16:1, and the optical network compression is performed at the speed of light, with energy consumption close to 0. For complex tasks, the accuracy of using only optical neural networks is only 45%, so optical networks are used for pre-processing and compression before transmitting it to the ground processing system, which greatly alleviates the difficulty of transmitting and processing massive data. If only compressed images are classified, the classification accuracy is 60%, so we use electronic neural networks. First, the compressed image of the optical network is decoded and reconstructed, and the 64×64 compressed vector is reconstructed to 256×256, and then classified again. The accuracy can reach 72%. The original image, optical network compressed vector and reconstructed image are shown as follows: Figure 7 shown.

[0114] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

[0115] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A computing network for space-ground collaborative detection, characterized by: It includes on-board optical computing network and ground-based electrical computing network, including: The on-board optical computing network receives images captured after simple target detection, sequentially performs linear modulation and nonlinear modulation on the simple target detection images, performs fitting processing on the modulated images to obtain simple target fitting images, and uses the simple target fitting images to perform simple target recognition; receives images captured after complex target detection, sequentially performs linear modulation and nonlinear modulation on the complex target detection images, performs fitting processing on the modulated images to obtain complex target fitting images, and uses the complex target fitting images to perform compression preprocessing; and sends simple target recognition results and complex target compression preprocessing results to the ground electrical computing network; The ground electronic computing network performs initial feature reconstruction and nonlinear signal reconstruction on simple target recognition results and complex target preprocessing results, and outputs the target reconstructed image.

2. The computing network for space-ground collaborative detection according to claim 1, characterized in that: The on-board optical computing network includes three linear modulation layers and two nonlinear modulation layers, wherein: The first linear modulation layer receives the detected and captured images of simple targets and complex targets, performs transmission-based intensity linear modulation on the detected and captured images to obtain initial feature matrices of the simple targets and complex targets, and sends the linearly modulated images to the first nonlinear modulation layer; The first nonlinear modulation layer performs nonlinear modulation on the linearly modulated image and limits the gain value of the nonlinearly modulated image to a preset multiple, and sends the nonlinearly modulated image to the second linear modulation layer; The second linear modulation layer extracts features from the nonlinearly modulated image, performs feature compression, and sends the feature-compressed image to the second nonlinear modulation layer; The second nonlinear modulation layer performs fitting processing on the feature compressed image and sends it to the third linear modulation layer; The third linear modulation layer performs image recognition or image compression preprocessing on the image after fitting processing according to the target type.

3. The computing network for space-ground collaborative detection according to claim 2, characterized in that: When the target type is a simple target, simple target recognition is performed on the simple target fitting image in the third linear modulation layer to obtain simple target information as a simple target recognition result; Simple target recognition is accomplished using the onboard optical computing network; When the target type is a complex target, compression preprocessing is performed on the complex target fitting image in the third linear modulation layer to obtain a complex target compression vector as a complex target compression preprocessing result; The complex target compression vector is subsequently calculated in the ground-based electronic computing network according to the requirements of the space-ground collaborative detection mission.

4. The computing network for space-ground collaborative detection according to claim 2, characterized in that: The first linear modulation layer includes a liquid crystal display and a lens; The liquid crystal display receives the images captured by simple target and complex target detection, performs imaging processing, loads the initial feature matrix of the simple target and complex target, and performs linear modulation according to the initial feature matrix; The lens performs down-sampling processing on the linear modulated image and outputs the linear modulated image to the first nonlinear modulation layer.

5. The computing network for space-ground collaborative detection according to claim 2, characterized in that: The first non-linear modulation layer comprises an image intensifier, wherein: The image intensifier receives the incident linearly modulated image, realizes nonlinear modulation through the local amplification effect and controls the gain to be 800 times to 3000 times.

6. The computing network for space-ground collaborative detection according to claim 2, characterized in that: The second linear modulation layer includes a microlens array, a liquid crystal display, and a lens, wherein: Microlens array, to replicate the image after nonlinear modulation; A liquid crystal display receives all the nonlinearly modulated images after replication and performs transmission intensity modulation, loads the corresponding modulation matrix to modulate the image to achieve feature extraction, and obtains a feature matrix for reflecting different categories of image information for display; The lens performs down-sampling and compression processing on the linear modulated image, obtains a linear modulated and feature compressed image whose focus is much smaller than the original image, and outputs it to the second nonlinear modulation layer.

7. The computing network for space-ground collaborative detection according to claim 2, characterized in that: The second non-linear modulation layer comprises an image intensifier, wherein: An image intensifier receives an incident linear modulated image, performs nonlinear modulation and image fitting, and sends the fitted image to a third linear modulation layer; The image intensifier of the second nonlinear modulation layer is the same as that of the first nonlinear modulation layer.

8. The computing network for space-ground collaborative detection according to claim 2, characterized in that: The ground electrical computing network includes a sampling network, an initial reconstruction network and a deep reconstruction network, wherein: The sampling network presets a compression sampling matrix, automatically compresses the simple target recognition results and the complex target compression preprocessing results according to the compression sampling matrix, and sends the compressed image to the initial reconstruction network; The initial reconstruction network performs feature image conversion on the compressed image according to the image feature requirements of the space-ground collaborative exploration mission, and sends the initial reconstructed feature image to the deep reconstruction network; The deep reconstruction network receives the initial reconstructed feature image, performs nonlinear signal reconstruction, and outputs the target reconstructed image.

9. The computing network for space-ground collaborative detection according to claim 8, characterized in that: The initial reconstruction network uses convolutional layers and Reshape+Concate layers to achieve feature image conversion; In the initial reconstruction network structure, the convolution layer is designed as: The convolution kernel size is 1*1, the number of convolution kernel channels is 1024, the number of convolution kernels is 16384, the image size after input compression is 2*2, the number of input channels is 1024, the output feature map size is 2*2, and the output channels are 16384; The Reshape+Concate layer is used to convert the 16384*2*2 feature image into an initial reconstructed feature image of 1*256*256 and output the deep reconstruction network.

10. The computing network for space-ground collaborative detection according to claim 8, characterized in that: The deep reconstruction network includes five convolutional layers with the same structure, and each layer is designed as follows: The first convolution layer has a kernel size of 3*3, a kernel channel number of 1, a kernel number of 64, a stride of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 1, the output feature map size is 256*256, and the output channel number is 64; The second convolution layer has a convolution kernel size of 3*3, 64 convolution kernel channels, 64 convolution kernels, a step size of 1*1, and a padding of 1*1. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 64; The third and fourth convolutional layers have the same design as the second convolutional layer; The fifth convolution layer has a kernel size of 3*3, 64 kernel channels, 1 kernel number, 1 stride, and 1*1 padding. The input feature map size is 256*256, the input channel number is 64, the output feature map size is 256*256, and the output channel number is 1.

Images