Image processing method, system, application and computer-readable storage medium

Abstract

The present application relates to the field of semiconductor technology, and specifically discloses a method for image processing based on a light-emitting unit, a system thereof, an application, and a computer-readable storage medium, wherein the light-emitting unit is connected to a bias power supply, wherein the light-emitting unit is used to detect, process, or recognize an image, and the light-emitting unit includes R×C pixel units arranged in an array, and each pixel unit includes M Micro-LEDs with different numbers; the method includes: obtaining the output photocurrent of the Micro-LED; inputting the output photocurrent of the Micro-LED into a neural network to generate a processing result of an image corresponding to the output photocurrent of the Micro-LED. In the above manner, the present application utilizes the high optoelectronic modulation bandwidth, MIMO parallel communication, and high communication speed characteristics of the Micro-LED array to construct a neural network with the detection, processing, and recognition functions of visual information, meeting the requirements of low power consumption, low latency, and high-speed image processing.

Description

Background Art

[0002] In the existing technology, machine vision uses cameras to capture images frame by frame, convert them into digital signals for transmission, and finally use deep learning neural networks to achieve applications such as target detection, image recognition, and image enhancement. However, with the development of the times, people's demand for time is getting higher and higher, and the characteristics of traditional machine vision such as high latency, high power consumption, and high data storage requirements are becoming more and more prominent. Summary of the invention

[0003] Based on this, the present application proposes a light-emitting unit-based image processing method and its system, application, and computer-readable storage medium, which utilizes the high optoelectronic modulation bandwidth, MIMO parallel communication, and high communication speed characteristics of the Micro-LED array to achieve a low-power, low-latency, and high-speed image processing method.

[0004] To solve the above technical problems, a technical solution adopted in the present application is: proposing a picture processing method based on a light-emitting unit, the light-emitting unit is connected to a bias power supply, wherein the light-emitting unit is used to detect, process or identify pictures, and the light-emitting unit includes R×C pixel units arranged in an array, and each pixel unit includes M Micro-LEDs with different numbers; the method includes: obtaining the output photocurrent of the Micro-LED; inputting the output photocurrent of the Micro-LED into a neural network, and calculating the bias control weight based on the output photocurrent of the Micro-LED to generate a processing result of the picture corresponding to the output photocurrent of the Micro-LED.

[0005] To solve the above technical problems, another technical solution adopted in the present application is: proposing a picture processing system based on a light-emitting unit, including: a light-emitting unit, used to detect, identify or compress pictures, the light-emitting unit includes R×C pixel units arranged in an array, each pixel unit includes M Micro-LEDs with different numbers; a bias power supply, connected to the light-emitting unit, the detection photocurrent generated by the bias power supply based on the bias control flows through the Micro-LED; a neural network operation structure, used to collect the output photocurrent of the Micro-LED, and calculate the bias control weight based on the output photocurrent of the Micro-LED to generate a processing result of the picture corresponding to the output photocurrent of the Micro-LED.

[0006] In order to solve the above technical problems, another technical solution adopted in the present application is: proposing an application of an image processing system based on a light-emitting unit in a visible light communication device.

[0007] In order to solve the above technical problems, another technical solution adopted in the present application is: proposing a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned image processing method based on the light-emitting unit are implemented.

[0008] Compared with the prior art, the present application has the following advantages and beneficial effects: utilizing the high optoelectronic modulation bandwidth, MIMO parallel communication and high communication speed characteristics of the Micro-LED array to construct a neural network with the detection, processing and recognition functions of visual information, meeting the requirements of low power consumption, low latency and high-speed image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0010] Figure 1 It is a flowchart of the first embodiment of the image processing method based on the light-emitting unit of the present application;

[0011] Figure 2 It is a flowchart of the second embodiment of the image processing method based on the light-emitting unit of the present application;

[0012] Figure 3 It is a flowchart of the third embodiment of the image processing method based on the light-emitting unit of the present application;

[0013] Figure 4 It is a schematic diagram of the framework of the light-emitting unit of the present application;

[0014] Figure 5 It is a schematic diagram of the framework of the neural network of this application;

[0015] Figure 6 It is a structural schematic diagram of the image processing system based on the light-emitting unit of the present application;

[0016] Figure 7 This is the relationship between the output photocurrent of Micro-LED under the bias power supply and the incident light power density (the horizontal axis is the light power density (power density), the unit is W / cm 2 ; The vertical axis is photocurrent (photocurrent), the unit is A);

[0017] Figure 8 is a set of randomly generated noise images;

[0018] Fig. 9It is the relationship between the output photocurrent and the incident light power density under different light power densities (the horizontal axis is the bias voltage, in V; the vertical axis is the photocurrent, in A);

[0019] Fig.10 It is a schematic diagram of the structure of the Micro-LED of this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] The present application proposes a method for image processing based on a light emitting unit, where the light emitting unit can be used as a light emitting device of a visible light communication device. Figure 4 The light-emitting unit 11 includes R rows×C columns of pixel units 111 arranged in an array, wherein each pixel unit 111 includes M Micro-LEDs 1111 with different numbers, such as 1, 2, 3, ... M. The sizes of the Micro-LEDs 1111 are equal or different, and the Micro-LEDs 1111 can be in any shape, such as round, square, polygonal, which is not limited here.

[0022] Micro-LED1111 is a new type of light-emitting LED based on GaN epitaxial wafers. Compared with ordinary LEDs on the market, it is smaller in size, generally below 100μm. Micro-LED1111 can be used as a light source for Gbps-level visible light communication, and can also be used as a photodetector to achieve high-speed parallel visible light communication, which is conducive to image imaging and image processing.

[0023] It can be understood that the light-emitting unit 11 is a Micro-LED array. A small Micro-LED array can realize functions such as accelerating the operation of a convolution layer or a pooling layer, and a large Micro-LED array can be used to realize functions such as image imaging, image recognition, and image compression coding.

[0024] Furthermore, a set of noise pictures with noise is randomly generated (the noise picture set includes several pictures with a pixel size of R×C, and the pictures are numbered from small to large according to the noise strength). The pictures are input into the Micro-LED array, and the Micro-LED array detects the pictures and images a picture with light and dark stripes with a pixel size of R×C on the imaging plane.

[0025] The above-mentioned Micro-LED 1111 refers to an LED with a size less than 100μm, more precisely, it refers to an LED 1111 with a size of approximately 1μm-100μm. At a pulse transmission rate of 1GHz, the pulse response of the received image of Micro-LED 1111 can reach 400ns, and the image processing rate can reach 2.5 megaframes. Therefore, when the Micro-LED array is used as a photodetector, the high photoelectric modulation bandwidth, MIMO parallel communication and high communication speed characteristics of the Micro-LED array can be used to achieve a low-power, low-latency and high-speed image processing method.

[0026] Furthermore, the light power density of the incident light received by each Micro-LED 1111 in the same pixel unit 111 is consistent, while the light power density of the incident light between the pixel units 111 is determined by the requirements of the external machine vision. The total output photocurrent I in this application is k is the sum of the output photocurrents of all Micro-LEDs 1111 in the R×C pixel units 111 that have the same number as the kth Micro-LED 1111 in the i-th row and j-th column pixel unit 111. Figure 4 For example, the total output photocurrent I1 is the sum of the output photocurrents of all No. 1 Micro-LEDs 1111 in the R×C pixel units 111. The total output photocurrent I M is the sum of the output photocurrents of all the Mth Micro-LEDs 1111 in the R×C pixel units 111. It can be understood that the total output photocurrents I1, I M Represents a feature of the picture respectively. In the small Micro-LED array, the total output photocurrent I1, I M Can also be used to indicate a corresponding valid category.

[0027] like Figure 6 As shown, the light-emitting unit 11 is connected to the bias power supply 12 , and the bias current generated by the bias power supply 12 flows through the Micro-LED 1111 .

[0028] See also Figure 1 , the method comprises the following steps:

[0029] S10: Obtain the output photocurrent of Micro-LED1111.

[0030] Specifically, picture A in a set of noise pictures is input to the light-emitting unit 11, and the light-emitting unit 11 detects picture A and images the picture A on the imaging plane. At this time, the output photocurrent of all Micro-LEDs 1111 in the light-emitting unit 11 corresponding to picture A is obtained.

[0031] S20: Input the output photocurrent of Micro-LED 1111 into the neural network, and calculate the bias control weight based on the output photocurrent of Micro-LED 1111 to generate a processing result of the image corresponding to the output photocurrent of Micro-LED 1111.

[0032] Specifically, noise is used to enhance the robustness of image recognition. Noise is caused by external noise (such as insufficient modulation accuracy of the spatial modulator SLM) and crosstalk, and noise is not determined by the Micro-LED device itself. However, the effect caused by noise is related to the output photocurrent of Micro-LED1111, and all pictures in each group of noise pictures are numbered from small to large according to the noise strength. Therefore, the output photocurrent of all Micro-LED1111 in the light-emitting unit 11 corresponding to picture A is input into the neural network. After the operation and processing of the neural network, the number of picture A in the group of noise pictures can be obtained. In addition to generating classification results for pictures, simple encoding and compression of pictures can also be performed.

[0033] Compared with the prior art, the present application has the following advantages and beneficial effects: utilizing the high optoelectronic modulation bandwidth, MIMO parallel communication and high communication speed characteristics of the Micro-LED array to construct a neural network with the detection, processing and recognition functions of visual information, meeting the requirements of low power consumption, low latency and high-speed image processing.

[0034] In one embodiment, if Figure 5 As shown in the figure, the neural network is a four-layer network, including an input layer (InputLayer), an amplification layer (Multiply Layer), a full connection layer (Full Connet Layer) and a Softmax classification layer (Softmax Layer) connected in sequence.

[0035] See also Figure 2 , step S20 comprises:

[0036] S21: Input the output photocurrent of Micro-LED1111 to the input layer to obtain the sampling result P ijk , where P ijk Represents the sampling result of the kth Micro-LED 1111 in the i-th row and j-th column pixel unit 111, 1≤i≤R, 1≤j≤C, 1≤k≤M, and i, j, and k are all integers.

[0037] S22: Sampling result P through the enlargement layer ijk Perform amplification calculation to obtain the amplified sampling result Y ijk , Y ijk =α×Pijk , where α is the magnification factor.

[0038] S23: The total output photocurrent I is output by the fully connected layer k , W ijk Represents the bias control weight of the kth Micro-LED 1111 in the i-th row and j-th column.

[0039] S24: The processing result O of the image output by the Softmax classification layer x ,in,

[0040]

[0041] Specifically, the output photocurrents of all Micro-LEDs 1111 in the light-emitting unit 11 corresponding to the A picture are input to the input layer of the neural network, and after the calculations of steps S21 to S24, the O x , O x Indicates that the number of picture A in the set of noise pictures is X.

[0042] In one embodiment, the method comprises the following steps:

[0043] S30: training the neural network.

[0044] Among them, see Figure 3 , step S30 comprises:

[0045] S31: Initialize the bias value of the bias power supply 12.

[0046] Specifically, in order to make the bias current corresponding to the bias value be the preset value, the bias value of the bias power supply 12 needs to be initialized. After initialization, the bias current corresponding to the bias value is at Figure 7 At the marked position, at this time, the bias value of the bias power supply 12 is about 0.4V.

[0047] S32: inputting several groups of noisy pictures in the randomly generated noise picture set into the light-emitting unit 11 in sequence, wherein each group of noise pictures includes a plurality of pictures with a pixel size of R×C.

[0048] Noise is used to enhance the robustness of image recognition. It should be noted that the images of each group of noise images in step S32 are obtained by sampling, so the noise of the images of each group of noise images is random.

[0049] S33: When the light-emitting unit detects, processes or identifies picture A, the output photocurrent of Micro-LED 1111 corresponding to picture A is input into the neural network, and the processing result of picture A is output through calculation of the neural network.

[0050] It should be noted that after the light-emitting unit 11 detects the noise picture set (the noise picture set includes at least one of an "N"-shaped picture, a "T"-shaped picture, a "V"-shaped picture, and an "X"-shaped picture), it generates the following images based on the shape of the Micro-LED array 11: Figure 8 The four groups of pictures shown, each group of pictures includes at least 4 imaging pictures with a pixel size of 3×3. It can be understood that the imaging pictures with a pixel size of 3×3 are obtained by imaging 3×3 pixel units 111 arranged in an array.

[0051] Specifically, the output photocurrent of Micro-LED1111 corresponding to picture A is input to Figure 6 The neural network shown in Figure 2 undergoes Figure 2 After the neural network operation shown, the category number of picture A is output, that is, picture A is the picture numbered X in the noise picture set.

[0052] S34: Design a loss function according to the processing result of picture A and the actual number of picture A in the noise picture set, and determine the calculation result of the loss function.

[0053] Specifically, the calculation result of the loss function during training is calculated according to the number of the picture output by the neural network in the noise picture set and the error between the actual number of the picture in the noise picture set.

[0054] S35: Use gradient descent method to solve bias control weight W ijk The resulting error and weight back propagation update results.

[0055] S36: After each group of noise image training is completed, complete the bias control weight W ijk Updates.

[0056] Below Figure 5 The neural network shown and Figure 8 Taking the noise picture set shown as an example, the implementation details of the technical solution of the embodiment of the present application are elaborated in detail:

[0057] Example 1

[0058] The image processing method of this application is used for Figure 8 The noise picture set shown is used for picture recognition, which specifically includes the following steps:

[0059] (1) Initialize the bias voltage of the bias power supply 12 so that the initial bias current is at Figure 7 At the marked position, at this time, the bias value of the bias power supply 12 is about 0.4V;

[0060] (2) Randomly generate Figure 8There are 4 groups of different types of noise pictures shown, each group of noise pictures includes N pictures, where N is greater than 4 and N is an integer. Preferably, N is greater than or equal to 30, and it is ensured that each picture is included in each extracted picture set;

[0061] (3) Each image in each group of noise images is sequentially input into the Micro-LED array (i.e., the light-emitting unit 11), and the Micro-LED array detects the image and forms an image with light and dark stripes on an imaging plane;

[0062] (4) When the Micro-LED array detects the zth image (1≤z≤N), the output photocurrent of all Micro-LEDs 1111 in the Micro-LED array is obtained;

[0063] (5) Training the neural network, inputting the output photocurrent of all Micro-LEDs 1111 into the neural network, and the neural network outputs the number of the image in the noise image set;

[0064] (6) Calculate the loss function based on the number of the image output by the neural network in the noisy image set and the actual number of the image in the noisy image set, and use the gradient descent method to solve the error caused by the bias control weight and the update result of the weight back propagation;

[0065] (7) Determine whether z is equal to N (i.e., whether the zth picture is the last picture). If not, repeat steps (4)-(6) until the zth picture is the last picture.

[0066] (8) adjusting and updating the calculated bias value of the bias power supply 12 according to the minimum value;

[0067] (9) After each set of noise image training is completed, the bias control weights are updated.

[0068] (10) After completing the above training, the output photocurrent of Micro-LED 1111 corresponding to any one of the above four groups of noise images is input into the neural network, and image recognition can be achieved, and the final result of image recognition can reach 100%.

[0069] Example 2

[0070] The image processing method of this application is used for Figure 8 The noise picture set shown is subjected to picture coding compression, specifically comprising the following steps:

[0071] (1) Initialize the bias voltage of the bias power supply 12 so that the initial bias current is at Figure 7 At the marked position, at this time, the bias value of the bias power supply 12 is about 0.4V;

[0072] (2) Randomly generate Figure 8 There are 4 sets of different types of noise picture sets shown, each set of noise pictures includes N pictures, where N is greater than 4 and N is an integer, preferably, N is greater than or equal to 30;

[0073] (3) Each image in each group of noise images is sequentially input into the Micro-LED array (i.e., the light-emitting unit 11), and the Micro-LED array detects the image and forms an image with light and dark stripes on an imaging plane;

[0074] (4) When the Micro-LED array detects the zth image (1≤z≤N), the output photocurrent of all Micro-LEDs 1111 in the Micro-LED array is obtained;

[0075] (5) Training the neural network, inputting the output photocurrent of all Micro-LEDs 1111 into the neural network, and the neural network outputs the number of the image in the noise image set;

[0076] (6) Calculate the loss function based on the number of the image output by the neural network in the noisy image set and the actual number of the image in the noisy image set, and use the gradient descent method to solve the error caused by the bias control weight and the update result of the weight back propagation;

[0077] (7) Determine whether z is equal to N (i.e., whether the zth picture is the last picture). If not, repeat steps (4)-(6) until the zth picture is the last picture.

[0078] (8) adjusting and updating the calculated bias value of the bias power supply 12 according to the minimum value;

[0079] (9) After each set of noise image training is completed, the bias control weights are updated.

[0080] (10) After completing the above training, record the result matrix I of the fully connected layer, and only output 4 pixels, successfully compressing 9 pixels into 4 pixels;

[0081] (11) Record the size and value of the weight matrix W and calculate the inverse matrix W of the matrix -1 , through P = IW -1 To restore the original pixel values ​​of 9 pixels, complete the decoding and restoration of the image, achieve input and output of exactly the same image, and reduce the amount of data by more than 50%.

[0082] In addition, in an exemplary embodiment of the present application, a picture processing system 10 based on the light emitting unit 11 is also provided. Figure 6As shown, the image processing system 10 includes: a light emitting unit 11, a bias power supply 12 and a neural network operation unit 13.

[0083] The light-emitting unit 11 is used to form a picture. The light-emitting unit 11 includes R×C pixel units 111 arranged in an array. Each pixel unit 111 includes M Micro-LEDs 1111 with different numbers.

[0084] The bias power supply 12 is connected to the light-emitting unit 11 , and the bias current generated by the bias power supply 12 flows through the Micro-LED 1111 .

[0085] The neural network operation unit 13 is used to obtain the output photocurrent of Micro-LED 1111 and process the output photocurrent of Micro-LED 1111 to generate a processing result of a picture corresponding to the output photocurrent of Micro-LED 1111 .

[0086] In one embodiment, the neural network operation unit 13 includes an input layer, an amplification layer, a fully connected layer, and a Softmax classification layer which are connected in sequence.

[0087] The input layer is used to process the output photocurrent of Micro-LED1111 to obtain the sampling result P ijk , where P ijk Represents the sampling result of the kth Micro-LED 1111 in the i-th row and j-th column pixel unit 111, 1≤i≤R, 1≤j≤C, 1≤k≤M, and i, j, and k are all integers.

[0088] The amplification layer is used to enlarge the sampling result P ijk Perform amplification calculation to obtain the amplified sampling result Y ijk , where Y ijk =α×P ijk , where α is the magnification factor.

[0089] The fully connected layer is used to output the total output photocurrent I k ,in W ijk Represents the bias control weight of the kth Micro-LED 1111 in the i-th row and j-th column.

[0090] Softmax classification layer, used to output the image processing result O x ,in

[0091]

[0092] like Fig.10As shown, the basic structure of Micro-LED 1111 includes: a substrate 101, a buffer layer 102, an N-type gallium nitride layer 103, a quantum well active layer 104, a P-type gallium nitride layer 105, an ITO layer 106, an N-type electrode 107 and a P-type electrode 108.

[0093] The substrate 101 is any substrate such as a sapphire substrate, a silicon substrate, a GaN substrate, a SiC substrate, a flexible PET substrate, etc. The buffer layer 102 is grown on the substrate 101. The N-type gallium nitride layer 103 is grown on the side of the buffer layer 102 away from the substrate 101. The quantum well active layer 104 is grown on the side of the N-type gallium nitride layer 103 away from the buffer layer 102. The P-type gallium nitride layer 105 is grown on the side of the N-type gallium nitride layer 103 away from the quantum well active layer 104. The ITO layer 106 is grown on the side of the P-type gallium nitride layer 105 away from the quantum well active layer 104. The N-type electrode 107 is grown on the side of the N-type gallium nitride layer 103 away from the buffer layer 102, and is spaced apart from the quantum well active layer 104, and the N-type electrode 107 is connected to the bias power supply 12. The P-type electrode 108 is grown on a side of the P-type gallium nitride layer 105 away from the quantum well active layer 104 , and the P-type electrode 108 is used for grounding.

[0094] The bias power supply 12 may be a controllable DC power supply or an FPGA onboard power supply.

[0095] In such a connection, Micro-LED1111 shows an obvious effect of bias voltage controlling photocurrent output. When Micro-LED1111 is used as a photodetector, the relationship between the output photocurrent size and the incident light power density under different light power densities is shown in the figure below: Fig. 9 As shown in the figure (the horizontal axis is the bias voltage, the unit is V; the vertical axis is the photocurrent, the unit is A). When Micro-LED1111 is used as a photodetector, the relationship between the output photocurrent of Micro-LED1111 under the action of the bias power supply and the incident light power density is as follows: Figure 7 As shown (the horizontal axis is the light power density (power density), the unit is W / cm 2 ; The vertical axis is photocurrent (photocurrent), the unit is A).

[0096] Since the various functional modules of the image processing system 10 of the example embodiment of the present application correspond to the steps of the example embodiment of the above-mentioned image processing method, for details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the above-mentioned image processing method of the present application.

[0097] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0098] In addition, in an exemplary embodiment of the present application, an application of the above-mentioned image processing system 10 based on the light-emitting unit 11 in a visible light communication device is also provided.

[0099] In addition, in an exemplary embodiment of the present application, an electronic device capable of executing the steps of the picture processing method of the above embodiment is also provided.

[0100] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to as a "circuit", "module" or "system" herein.

[0101] The electronic device is presented in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: the at least one processor mentioned above, the at least one memory mentioned above, and a bus connecting different system components (including the memory and the processor).

[0102] Among them, the memory stores program code, and the program code can be executed by the processor, so that the processor executes the steps of each embodiment of the above-mentioned image processing method of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0103] The memory may include readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read only memory (ROM).

[0104] The bus may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0105] The electronic device can also communicate with one or more external devices (such as keyboards, pointing devices, Bluetooth devices, etc.), and can also communicate with one or more devices that enable users to interact with the electronic device, and / or communicate with any device (such as routers, modems, etc.) that enables the electronic device to communicate with one or more other computing devices. This communication can be carried out through an input / output (I / O) interface. In addition, the electronic device can also communicate with one or more networks (such as local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter. As shown in the figure, the network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0106] In addition, in an exemplary embodiment of the present application, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method of the above embodiment are implemented.

[0107] Compared with the prior art, the present application has the following advantages and beneficial effects: utilizing the high optoelectronic modulation bandwidth, MIMO parallel communication and high communication speed characteristics of the Micro-LED array to construct a neural network with the detection, processing and recognition functions of visual information, meeting the requirements of low power consumption, low latency and high-speed image processing.

[0108] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0109] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0110] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A picture processing method based on a light emitting unit, characterized in that: The light-emitting unit is connected to a bias power supply, wherein the light-emitting unit is used to detect, process or recognize an image, and the light-emitting unit includes R×C pixel units arranged in an array, and each of the pixel units includes M Micro-LEDs with different numbers; The method comprises: Inputting a target image into the light-emitting unit to obtain an output photocurrent of the Micro-LED; The output photocurrent of the Micro-LED is input into a trained neural network to generate a processing result of the target image corresponding to the output photocurrent of the Micro-LED, wherein the trained neural network includes a trained bias control weight of the Micro-LED.

2. The method according to claim 1, characterized in that The neural network is a four-layer network, including an input layer, an amplification layer, a fully connected layer, and a Softmax classification layer connected in sequence; The step of inputting the output photocurrent of the Micro-LED into the trained neural network to generate a processing result of the target image corresponding to the output photocurrent of the Micro-LED includes: The output photocurrent of the Micro-LED is input to the input layer to obtain a sampling result P ijk , where P ijk represents the sampling result of the kth Micro-LED in the i-th row and j-th column pixel unit, 1≤i≤R, 1≤j≤C, 1≤k≤M, and i, j, and k are all integers; The sampling result P is processed by the amplification layer ijk Perform amplification calculation to obtain the amplified sampling result Y ijk , where Y ijk =α×P ijk , where α is the magnification factor; The total output photocurrent I is output by the fully connected layer k , Where W ijk represents the bias control weight of the k-th Micro-LED in the i-th row and j-th column; The Softmax classification layer outputs the processing result of the target image. x ,in 3. The method according to claim 2, characterized in that The total output photocurrent I k is: the sum of the output photocurrents of all Micro-LEDs in the R×C pixel units that have the same number as the kth Micro-LED in the pixel unit in the i-th row and j-th column.

4. The method according to claim 1, characterized in that The method further comprises: training the neural network; The step of training the neural network comprises: Initializing a bias voltage value of the bias power supply; Inputting several groups of pictures with noise from the randomly generated noise picture set into the light-emitting unit in sequence, wherein each group of the noise picture set includes a plurality of pictures with a pixel size of R×C, and the noise is used to enhance the robustness of picture recognition; When the light-emitting unit detects, processes or recognizes the image, the output photocurrent of the Micro-LED corresponding to the image is input into the neural network, and the processing result of the image is output through calculation by the neural network; Designing a loss function according to the processing result of the picture and the actual number of the picture in the noise picture set, and determining a calculation result of the loss function; Using the gradient descent method to solve the error caused by the bias control weight and the update result of the weight back propagation; After the training of each group of the noise picture sets is completed, the bias control weight is updated.

5. A picture processing system based on a light emitting unit, characterized in that: include: A light-emitting unit, used for detecting, identifying or compressing an image, wherein the light-emitting unit comprises R×C pixel units arranged in an array, and each of the pixel units comprises M Micro-LEDs with different numbers; A bias power supply connected to the light-emitting unit, wherein a detection photocurrent generated by the bias power supply based on bias control flows through the Micro-LED; A neural network operation structure is used to obtain the output photocurrent of the Micro-LED after the target image is input into the light-emitting unit, and then input the output photocurrent of the Micro-LED into the trained neural network to generate a processing result of the target image corresponding to the output photocurrent of the Micro-LED, wherein the trained neural network includes a trained bias control weight of the Micro-LED.

6. The image processing system according to claim 5, characterized in that: The neural network operation structure includes an input layer, an amplification layer, a fully connected layer and a Softmax classification layer connected in sequence; The input layer is used to process the output photocurrent of the Micro-LED to obtain a sampling result P ijk , where P ijk represents the sampling result of the kth Micro-LED in the i-th row and j-th column pixel unit, 1≤i≤R, 1≤j≤C, 1≤k≤M, and i, j, and k are all integers; The amplification layer is used to amplify the sampling result P ijk Perform amplification calculation to obtain the amplified sampling result Y ijk , where Y ijk =α×P ijk , where α is the magnification factor; The fully connected layer is used to output the total output photocurrent I k ,in W ijk represents the bias control weight of the kth Micro-LED in the i-th row and j-th column, and the total output photocurrent I k is: the sum of the output photocurrents of all Micro-LEDs in the R×C pixel units that have the same number as the kth Micro-LED in the pixel unit in the i-th row and j-th column; The Softmax classification layer is used to output the processing result of the target image. x ,in 7. The image processing system according to claim 5, characterized in that: The basic structure of Micro-LED includes: substrate; a buffer layer grown on the substrate; An N-type gallium nitride layer is grown on a side of the buffer layer away from the substrate; A quantum well active layer grown on a side of the N-type gallium nitride layer away from the buffer layer; A P-type gallium nitride layer, grown on a side of the quantum well active layer away from the N-type gallium nitride layer; An ITO layer is grown on a side of the P-type gallium nitride layer away from the quantum well active layer; An N-type electrode, grown on a side of the N-type gallium nitride layer away from the buffer layer and spaced apart from the quantum well active layer, the N-type electrode being connected to the bias power supply; A P-type electrode is grown on a side of the P-type gallium nitride layer away from the quantum well active layer, and the P-type electrode is used for grounding.

8. The image processing system according to claim 6, characterized in that: The bias power supply is a controllable DC power supply or an FPGA onboard power supply.

9. Application of the image processing system based on the light-emitting unit as claimed in any one of claims 5 to 8 in a visible light communication device.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images