A Visible Light Camera Communication Method Based on Convolutional Neural Networks

Abstract

This invention relates to the field of visible light communication technology and provides a visible light camera communication method based on convolutional neural networks. At the transmitting end: a pseudo-random binary sequence is generated using Matlab; the pseudo-random binary sequence is mapped to PWM symbols via a microcontroller unit, and the modulated PWM symbols are transmitted to free space through the RGB-LED three color channels. At the receiving end: the red, green, and blue color channels are separated in the CMOS camera using a Bayer color filter array hardware component; a convolutional neural network is used as a classifier to separate PWM symbols with different duty cycles within the same color channel; and real-time demodulation of the transmitting signal is achieved through a convolutional neural network-based mobile application. The receiving end uses a convolutional neural network-based application to demodulate the signal, enabling real-time demodulation of the transmitted signal, reducing system complexity, and improving the reliability of the OCC system.

Description

Technical Field

[0001] This invention relates to the field of visible light communication technology, specifically to a visible light camera communication method based on a convolutional neural network. Background Technology

[0002] Optical camera communication (OCC) can be divided into two categories based on exposure mode: global exposure and rolling shutter exposure. In global exposure, all pixels of the CMOS camera are captured simultaneously. Unlike global exposure, in rolling shutter exposure, the CMOS image sensor does not acquire all light signals simultaneously; each image frame is generated line by line based on the captured pixels. Although rolling shutter exposure can achieve a higher data rate compared to global exposure, increasing the data rate of OCC remains a challenge due to the low frame rate of CMOS cameras. Most existing OCC technologies use direct line-of-sight links for data transmission. Due to the uneven distribution of LED light emission, a halo effect occurs, causing fluctuations in signal pixel values. Furthermore, signal transmission in direct line-of-sight links is easily obstructed, limiting the availability and mobility of OCC systems. Therefore, there is a need to provide a visible light camera communication method based on convolutional neural networks to solve or alleviate the above problems. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a visible light camera communication method based on convolutional neural networks, so as to solve or alleviate the problems existing in the above-mentioned background technology.

[0004] This invention is implemented as follows: a visible light camera communication method based on a convolutional neural network, the method comprising the following steps:

[0005] At the transmitting end: a pseudo-random binary sequence is generated by Matlab; the pseudo-random binary sequence is mapped into a PWM symbol by a microcontroller unit, and the modulated PWM symbol is sent to free space through the three color channels of RGB-LED. The three I / O ports of the microcontroller unit control the on / off state of the LED by outputting high and low levels.

[0006] At the receiving end: In the CMOS camera, the three color channels of red, green and blue are separated by the hardware component of the Bayer color filter array. A convolutional neural network is used as a classifier to separate PWM symbols with different duty cycles in the same color channel. Real-time demodulation of the transmitting end signal is achieved through a mobile application based on the convolutional neural network.

[0007] As a further aspect of the present invention: the transmitting end includes a DC power supply, a microcontroller unit, a drive controller, and an LED; the receiving end includes a smartphone camera and an application based on a convolutional neural network, and the transmitting end and the receiving end use a non-line-of-sight link for signal transmission.

[0008] As a further aspect of the present invention: the microcontroller unit is an STM32 MCU.

[0009] As a further aspect of the present invention: the transmitting end data is mapped to PWM symbols by a color-multiplexed modulation method. The pseudo-random binary sequence is first mapped to red, green and blue color symbols respectively, and the same color channel symbols are then mapped to PWM symbols with different duty cycles.

[0010] As a further aspect of the present invention: when the LED is turned on, a bright stripe is generated corresponding to the exposure line; when the LED is turned off, a dark stripe is generated corresponding to the exposure line, so that bright and dark stripes of different colors and widths are displayed on the image captured by the CMOS camera.

[0011] As a further aspect of the present invention: the Bayer color filter array consists of a matrix arranged with three color filters: red, green and blue. Each pixel covers only one of the color filters, and the Bayer color filter array can capture information of a specific color channel.

[0012] As a further aspect of the present invention: a video stream is captured by the smartphone camera at the receiving end, the video stream is converted into images frame by frame, the column pixel value of the center column of the image is selected as the column matrix, and the selected column matrix is ​​divided into several sub-column matrices at equal intervals according to the PWM symbol resolution. The sub-column matrices are then input into a convolutional neural network for training. Through training, the convolutional neural network learns the pixel differences between PWM symbols with different duty cycles, thereby separating PWM symbols with different widths in the same color channel.

[0013] Compared with the prior art, the beneficial effects of the present invention are:

[0014] This invention is applicable to visible light camera communication under non-line-of-sight links, overcoming the problem of data transmission links being easily blocked in direct line-of-sight links and improving the availability of OCC systems. The transmitting end adopts a color-multiplexed modulation format to improve the transmission capability of OCC systems. The receiving end uses a convolutional neural network-based application to demodulate the signal, which can realize real-time demodulation of transmitted signals, reducing system complexity and improving the reliability of OCC systems. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a visible light camera communication method based on a convolutional neural network. Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0018] like Figure 1 As shown, this embodiment of the invention provides a visible light camera communication method based on a convolutional neural network, the method comprising the following steps:

[0019] At the transmitting end: a pseudo-random binary sequence is generated by Matlab in the computer; the pseudo-random binary sequence is mapped into a PWM symbol by a microcontroller unit (MCU); the modulated PWM symbol is sent to free space through the three color channels of RGB-LED; the three I / O ports of the microcontroller unit control the on / off state of the LED by outputting high and low levels.

[0020] At the receiving end: In the CMOS camera, the three color channels of red, green and blue are separated by the hardware component of the Bayer color filter array. A convolutional neural network is used as a classifier to separate PWM symbols with different duty cycles in the same color channel. Real-time demodulation of the transmitting end signal is achieved through a mobile application based on the convolutional neural network.

[0021] It should be noted that Optical Camera Communication (OCC) is a promising visible light communication technology. With the widespread application of Complementary Metal-Oxide-Semiconductor (CMOS) cameras in smart devices, OCC has attracted extensive attention from researchers. Most existing OCC technologies use direct line-of-sight links for data transmission. Due to the uneven distribution of LED light emission, a halo effect occurs, causing fluctuations in signal pixel values. The embodiments of this invention aim to solve or alleviate the above problems.

[0022] In this embodiment of the invention, the transmitting end includes a DC power supply, a microcontroller unit, a driver controller, and an LED; the receiving end includes a smartphone camera and a convolutional neural network-based application. The transmitting end and the receiving end transmit signals via a non-line-of-sight link, and the microcontroller unit is an STM32 MCU. This embodiment of the invention maps the transmitting end data to PWM symbols using color-multiplexed modulation. The pseudo-random binary sequence is first mapped to red, green, and blue color symbols, with symbols for the same color channel then mapped to PWM symbols with different duty cycles. Since the CMOS camera uses a rolling shutter exposure mode, each image frame is generated line by line based on the captured pixels. When the LED is on, a bright stripe is generated for the corresponding exposure line; when the LED is off, a dark stripe is generated for the corresponding exposure line, resulting in bright and dark stripes of different colors and widths appearing on the image captured by the CMOS camera. Because the voltage provided by the MCU is relatively low, an LED driver circuit is added between the LED and the MCU to increase the LED current. After the transmitting end optical signal propagates through the non-line-of-sight link, it is recorded as a video stream by the smartphone camera at the receiving end. At the receiving end, the captured video stream is converted into images frame by frame. After image preprocessing, the images are input into a CNN for decoding, and finally the transmitted data is recovered.

[0023] In this embodiment of the invention, different color channels are separated using the RGB filter of a CMOS camera. In a CMOS camera, the red, green, and blue color channels can be separated using a hardware component called a Bayer color filter array. The Bayer color filter array consists of a matrix arranged with red, green, and blue color filters. Each pixel covers only one of the color filters, and the Bayer color filter array can capture information from a specific color channel. This invention separates symbols within the same color channel using a convolutional neural network. A video stream is captured by the camera of a smartphone at the receiving end. The video stream is converted into images frame by frame. The central column of pixels in the image is selected as the column matrix. Based on the PWM symbol resolution, the selected column matrix is ​​divided into several sub-column matrices at equal intervals and input into a convolutional neural network (CNN) for training. Through training, the CNN learns the pixel differences between PWM symbols with different duty cycles, thereby separating PWM symbols with different widths within the same color channel. This enables real-time demodulation of the transmitted signal via a mobile app.

[0024] In this embodiment of the invention, color multiplexing technology is employed to improve the transmission capability of the OCC system and compensate for the low transmission rate of OCC. For example, in this embodiment, the binary data sequence is modulated in groups of three. Data "000", "001", and "010" are transmitted through the red channel of the RGB-LED, with duty cycles of 25%, 50%, and 75%, respectively; "011", "100", and "101" are transmitted through the green channel, with duty cycles of 25%, 50%, and 75%, respectively; and "110" and "111" are transmitted through the blue channel, with duty cycles of 25% and 50%, respectively. The data packet consists of 5 header symbols and 40 payload symbols modulated by OOK. Due to the rolling shutter exposure mechanism, data loss can occur during transitions between image frames. Therefore, the data packet is sent twice to ensure complete capture. At the receiving end, by adjusting the ISO and exposure time of the smartphone camera, photos with bright and dark stripes of different colors and widths can be captured.

[0025] In this embodiment of the invention, when demodulating based on a convolutional neural network, the video stream captured by the smartphone camera is first converted into images frame by frame. In a CMOS camera, the red, green, and blue color channels can be separated using a hardware component called a Bayer color filter array, and these are read as a data matrix. The Bayer color filter array consists of a matrix arranged with red, green, and blue color filters. Each pixel covers only one of the color filters, thus capturing information for a specific color channel. Due to uneven LED light distribution, a halo effect occurs, causing fluctuations in signal pixel values. Although the impact of the halo effect can be greatly mitigated in non-line-of-sight links, it still exists. To solve this problem, a second-order polynomial is used to fit the data to obtain the corresponding fitted value, and the pixel value is divided by the fitted value to reduce the data fluctuations caused by the halo effect. After finding the data header, the selected column matrix is ​​divided into equally spaced sub-column matrices and input into the CNN for classification. Each sub-matrix can recover three data symbols. Finally, the data is mapped to a bitstream for bit error rate analysis.

[0026] The above description only details the preferred embodiments of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0027] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0028] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the disclosure in the specification and embodiments. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A visible light camera communication method based on a convolutional neural network, characterized in that, The method includes the following steps: At the transmitting end: a pseudo-random binary sequence is generated by Matlab; the pseudo-random binary sequence is mapped into a PWM symbol by a microcontroller unit, and the modulated PWM symbol is sent to free space through the three color channels of RGB-LED. The three I / O ports of the microcontroller unit control the on / off state of the LED by outputting high and low levels. At the receiving end: In the CMOS camera, the red, green, and blue color channels are separated using a Bayer color filter array hardware component. A convolutional neural network (CNN) is used as a classifier to separate PWM symbols with different duty cycles in the same color channel. Real-time demodulation of the transmitting signal is achieved through a CNN-based mobile application. The receiving end uses a smartphone camera to capture a video stream, which is then converted into images frame by frame. The central column of pixels in the image is selected as the column matrix. Based on the PWM symbol resolution, the selected column matrix is ​​divided into several sub-column matrices at equal intervals and input into the CNN for training. Through training, the CNN learns the pixel differences between PWM symbols with different duty cycles, thereby separating PWM symbols with different widths in the same color channel.

2. The visible light camera communication method based on convolutional neural networks according to claim 1, characterized in that, The transmitting end includes a DC power supply, a microcontroller unit, a drive controller, and an LED; the receiving end includes a smartphone camera and a convolutional neural network-based application. The transmitting end and the receiving end use a non-line-of-sight link for signal transmission.

3. The visible light camera communication method based on convolutional neural networks according to claim 1, characterized in that, The microcontroller unit is an STM32 MCU.

4. The visible light camera communication method based on convolutional neural networks according to claim 1, characterized in that, The transmitter data is mapped to PWM symbols by using color multiplexing modulation. The pseudo-random binary sequence is first mapped to red, green and blue color symbols respectively. The symbols of the same color channel are then mapped to PWM symbols with different duty cycles.

5. The visible light camera communication method based on convolutional neural networks according to claim 1, characterized in that, When the LED is turned on, a bright stripe is generated in the corresponding exposure row; when the LED is turned off, a dark stripe is generated in the corresponding exposure row, resulting in bright and dark stripes of different colors and widths appearing in the image captured by the CMOS camera.

6. The visible light camera communication method based on convolutional neural networks according to claim 1, characterized in that, The Bayer color filter array consists of a matrix with three color filters arranged in red, green and blue. Each pixel covers only one of the color filters, and the Bayer color filter array can capture information of a specific color channel.

Images