Multi-wavelength wireless optical communication method and system

Abstract

The invention belongs to the technical field of wireless communication, and relates to a multi-wavelength wireless optical communication method and system. The method comprises the following steps: converting a mixed optical signal into a frame image sequence; identifying the dimming level of the mixed optical signal based on the frame image sequence, calculating the G component channel energy of each pixel point of each frame of image in the frame image sequence, and dividing the frame of image based on the G component channel energy of each pixel point to obtain RGBamp in the frame of image; an IR image and a pure infrared band image; determining the RGBamp in each frame of image; performing multi-path signal extraction on the IR image to obtain the RGBamp; rGB three-channel signals of the IR image are obtained; extracting the RGBamp by using a principal component analysis method; a first infrared signal in the IR image; performing signal extraction on the pure infrared band image in each frame of image to obtain a second infrared signal of the frame of image; and obtaining a demodulated target signal based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame of image.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a multi-wavelength wireless optical communication method and system. Background Art

[0002] Visible light communication technology (VLC) is an important part of wireless communication technology. Its principle is based on optical communication technology. It uses laser diodes (LD) or light emitting diodes (LED) as light sources to send signals. The signals propagate in free space. After the transmitter is modulated, the receiver captures and demodulates the optical signal through a photodetector or other receiving device to complete data communication. There are many ways to implement visible light communication systems, such as using photodetectors to sense optical signals, optical cameras to capture optical signals, and infrared receivers to receive infrared information.

[0003] Optical Camera Communication (OCC) is an important branch of VLC technology. The receiving end uses an optical camera to receive signals, making the system more flexible and lower in cost. However, due to the limited frame rate of optical cameras, in order to transmit more data, multi-link / multi-wavelength-based OCC has gradually become a research hotspot. For example, RGB-LED is used as the transmitter to send multi-channel RGB optical signals through light of different wavelengths. Wavelength Division Multiplexing (WDM) or Color-shift Keying (CSK) technology is used to use light of different wavelengths (such as red light, green light, and blue light) as different communication carriers, thereby greatly improving the communication rate and the system's concurrency capability. This enables the simultaneous transmission of multiple communication links in the same space, improving communication bandwidth and system capacity, and supporting more complex and intensive device communication needs, and can be better applied to scenarios such as smart homes and smart office buildings.

[0004] Some studies have found that due to the insufficient penetration of RGB optical signals, they are easily affected by environmental factors during the transmission process, causing signal attenuation and signal quality degradation. In addition, the RGB-LED-based system cannot achieve full-range precise dimming. If PWM is used to achieve the dimming function, data transmission will lose the transmission data of the PWM OFF time slot as the PWM duty cycle changes. If pure infrared band signals are mixed in RGB optical signals, not only can the transmission rate be increased, but also because pure infrared band signals have longer wavelengths and stronger penetration, the signal quality can still be guaranteed even in adverse environments. In addition, a full range of dimming levels of 0-1 can be achieved, and data loss caused by dimming can be compensated. However, since pure infrared band signals have their own RGB three-channel components, this will affect the three-channel data in the original RGB optical signal. When demodulating the signal, the existing receiving end cannot accurately separate the RGB signal and the pure infrared signal in the mixed signal, nor can it remove the influence of the infrared signal on the RGB signal, that is, it is impossible to demodulate the RGB optical signal mixed with the infrared signal, resulting in distortion or loss of key information in the demodulated signal at the receiving end, which in turn affects the reliability of the visible light communication system.

[0005] In summary, the existing visible light communication method cannot demodulate the RGB light signal mixed with the infrared signal, resulting in the problem that the demodulated signal is distorted or loses key information, which in turn affects the reliability of the visible light communication system. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the visible light communication method in the prior art cannot demodulate the RGB light signal mixed with the infrared signal, resulting in distortion or loss of key information in the demodulated signal, thereby affecting the reliability of the visible light communication system.

[0007] In order to solve the above technical problems, the present invention provides a multi-wavelength wireless optical communication method, which is applied to a receiving end of a multi-wavelength wireless optical communication system, comprising: Receive a mixed light signal sent by a transmitting end, and convert the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and an infrared band light signal; Identify the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; Perform multi-channel signal extraction on the RGB&IR image in each frame of image to obtain the RGB three-channel signal of the RGB&IR image; and use the principal component analysis method to extract the first infrared signal in the RGB&IR image; Extracting the signal of the pure infrared band image in each frame of image to obtain the second infrared signal of the frame of image; Based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image, a target sub-signal corresponding to the frame image is obtained; based on the target sub-signals corresponding to all frame images, a demodulated target signal is obtained.

[0008] Preferably, the frame image is divided based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image, including: Based on the G component channel energy of all pixels, a G component channel energy curve is obtained; Determine a threshold line based on a preset ratio of the maximum value of the G component channel energy curve, and use the first intersection point of the threshold line and the G component channel energy curve as an initial demarcation point; Taking the initial boundary point as the starting point, traverse the points on the G component channel energy curve in the reverse direction of the x-axis; obtain points whose slope signs are opposite to the slope signs of the initial boundary point, and obtain a target point set; The point in the target point set that is closest to the initial boundary point is taken as the target boundary point, and the frame image is divided based on the target boundary point to obtain the RGB&IR image and the pure infrared band image in the frame image.

[0009] Preferably, extracting multi-channel signals from the RGB & IR images in each frame of the image includes: Convert each frame of image into a YCrCb format image, and calculate the standard deviation of the Cb / Cr channel based on the values ​​of the Cb / Cr channel at all pixels in the YCrCb format image; Performing color correction on the YCrCb format image by using the standard deviation of the Cb / Cr channel as a color correction value; Gaussian filtering is performed on the color-corrected YCrCb format image, the Gaussian-filtered YCrCb format image is converted into an RGB format image to obtain a target RGB image, and multi-channel signal extraction is performed on the target RGB image.

[0010] Preferably, the calculation formula of the standard deviation of the Cb / Cr channel is expressed as: , Among them, among them, Indicates the standard deviation of the Cb / Cr channel; Indicates the first Line The value of the Cb / Cr channel at the pixel point of the column; Indicates the number of row pixels in a YCrCb format image; Indicates the number of columns of pixels in a YCrCb format image.

[0011] Preferably, extracting the first infrared signal from the RGB&IR image using principal component analysis includes: Based on the three-channel pixel gray value matrix of RGB&IR image, the decentralized three-channel pixel gray value matrix and its covariance matrix are obtained; Perform eigenvalue decomposition on the covariance matrix of the decentralized three-channel pixel grayscale value matrix to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; A principal component matrix is ​​obtained based on the eigenvectors corresponding to the h largest eigenvalues, and the decentralized three-channel pixel gray value matrix is ​​projected based on the principal component matrix to obtain a target matrix; The first vector in the target matrix is ​​used as the first infrared signal in the RGB&IR image.

[0012] Preferably, the calculation formula of the decentralized three-channel pixel gray value matrix is ​​expressed as: , in, Represents a decentralized three-channel pixel grayscale value matrix; Represents the three-channel pixel grayscale value matrix of RGB&IR image; express A vector of all 1s; Represents the mean vector composed of the mean values ​​of each dimension in the three-channel pixel grayscale value matrix of the RGB&IR image; represents transpose; The calculation formula of the covariance matrix of the decentralized three-channel pixel grayscale value matrix is ​​expressed as: , in, Represents the covariance matrix of the decentralized three-channel pixel grayscale value matrix; The calculation formula of the target matrix is ​​expressed as: , in, represents the target matrix; represents the principal component matrix.

[0013] Preferably, after obtaining the RGB&IR image and the pure infrared band image in each frame of image, the following steps are further included: Based on dimming recognition and separation, an RGB&IR image sequence is obtained, and wavelet decomposition is performed on the RGB&IR image sequence to obtain energy distribution of the RGB&IR image sequence; Performing a linear fit on the energy distribution of the RGB&IR image sequence, and using the least squares method to solve the slope of the fitted line; Normalizing the slope of the fitting straight line to obtain a proportional coefficient of the infrared band light signal in the mixed light signal; It is determined whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitting end, so that the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

[0014] Preferably, the formula for performing wavelet decomposition on the RGB&IR image sequence is: , in, Represents RGB&IR image sequence; represents the scale coefficient; represents the scaling function; represents the detail coefficient; represents the wavelet function; Indicates the maximum scale of wavelet decomposition, that is, the number of levels of wavelet decomposition; Represents the position index, which is used to indicate the different positions of the signal at different scales; Represents the scale index, indicating the Level of detail coefficient; The energy distribution of RGB&IR image sequence is expressed as: , in, Indicates that the RGB&IR image sequence is Levels of energy; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates A vector consisting of detail coefficients and approximation coefficients; represents the coefficient index; represents the coefficient index; The solution formula for the slope of the fitting line is expressed as: , in, represents the slope of the fitted line; The first observations; The output variable observations; represents the number of observations; The formula for normalizing the slope of the fitted straight line to obtain the proportional coefficient is expressed as: , in, Represents the proportionality factor.

[0015] The present invention also provides a multi-wavelength wireless optical communication system, comprising a transmitting end and a receiving end, wherein the receiving end comprises: A signal capture module, used to receive a mixed light signal sent by a transmitting end, and convert the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and a pure infrared band light signal; A dimming recognition and image separation module, which is used to recognize the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; The first signal extraction module is used to perform multi-channel signal extraction on the RGB&IR image in each frame of the image to obtain the RGB three-channel signal of the RGB&IR image; and extract the first infrared signal in the RGB&IR image by using the principal component analysis method; A second signal extraction module is used to extract the signal of the pure infrared band image in each frame of image to obtain the second infrared signal of the frame of image; The data fusion module is used to obtain the target sub-signal corresponding to each frame image based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image; and obtain the demodulated target signal based on the target sub-signals corresponding to all frame images.

[0016] Preferably, the receiving end further includes a feedback module, which specifically includes: A wavelet transform submodule is used to obtain an RGB&IR image sequence based on dimming recognition separation, and perform wavelet decomposition on the RGB&IR image sequence to obtain energy distribution of the RGB&IR image sequence; A linear fitting submodule, used for performing a linear fitting on the energy distribution of the RGB&IR image sequence and solving the slope of the fitting line using the least square method; A proportionality coefficient calculation submodule is used to normalize the slope of the fitting straight line to obtain a proportionality coefficient of the infrared band light signal in the mixed light signal; The feedback signal generating submodule is used to determine whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitting end, so that the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

[0017] After the multi-wavelength wireless optical communication method provided by the present application converts the received mixed light signal into a frame image sequence, since the color filter array in the receiving end is arranged as a red light filter, a blue light filter and two green light filters, and each pixel can only sense light of one color, the receiving end is more sensitive to the green component in the signal. Therefore, the present application can amplify the gap between the mixed signal and the pure infrared signal in the frame image by calculating the G component channel energy of each pixel in each frame image, and then divide the image to obtain the RGB&IR image and the pure infrared band image in each frame image; and, since there will still be low-rate infrared signals in the separated RGB&IR images, the principal component analysis method, as a non-parametric method, can effectively extract the mixed signal by performing orthogonal linear transformation in the direction of maximum variance. single signal, therefore, in addition to extracting the RGB three-channel signal from the RGB&IR image, the present application also uses the principal component analysis method to extract the first infrared signal from the RGB&IR image; then the signal is extracted from the pure infrared band image in each frame of the image to obtain the second infrared signal of the frame image; finally, based on the RGB three-channel signal, the first infrared signal and the second infrared signal in all frame images, the multi-channel signal sent by the transmitter can be accurately restored; the present application separates the RGB&IR image and the pure infrared band image, and then extracts the low-rate infrared signal from the RGB&IR image, thereby separating the infrared signal from the RGB&IR signal to the maximum extent and eliminating the influence of the infrared signal on the RGB signal, so as to accurately restore the multi-channel signal sent by the transmitter and improve the reliability of multi-wavelength wireless optical communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 A flow chart of the multi-wavelength wireless optical communication method provided by this application; Figure 2 A schematic diagram of the data frame structure sent by the transmitter provided in this application; Figure 3 A schematic diagram of the comparison of R channels with and without feedback links provided in this application; wherein, Figure 3 (a) is a schematic diagram of the R channel with a feedback link. Figure 3 (b) is a schematic diagram of the R channel when there is no feedback link; Figure 4 A schematic diagram of the structure of a multi-wavelength wireless optical communication system provided for this application; Figure 5 This is a schematic diagram of the bit error rate of each channel as the dimming level changes provided by the present application. DETAILED DESCRIPTION

[0019] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0020] See also Figure 1 , Figure 1 The figure shows a flow chart of a multi-wavelength wireless optical communication method provided by the present application. The method is applied to a receiving end of a multi-wavelength wireless optical communication system. The method specifically includes: S10: receiving a mixed light signal sent by the transmitting end, and converting the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and a pure infrared band light signal.

[0021] Specifically, in some embodiments of the present application, Figure 2 As shown, the steps of sending the mixed optical signal at the transmitting end include: Step 1: Multiplexed data and infrared data are modulated and output in parallel to be loaded into the ON time slot of PWM; higher rate pure infrared data output is loaded into the OFF time slot of PWM; Specifically, in the OFF time slot of PWM, since there is no other signal, the embodiment of the present application chooses to send pure infrared data at a higher rate, that is, the transmission rate in the ON time slot of PWM is , the transmission rate in the PWM OFF time slot is , thereby increasing the transmission rate of multi-wavelength wireless optical communication systems.

[0022] Step 2: Calculate the signal sending rate; Specifically, for the OCC system based on CMOS camera, there is a processing time between two consecutive frame images, which is called "blind period". During this period, the receiving end cannot receive any data. Therefore, in order to overcome the data loss problem caused by the blind time, it is necessary to set a suitable signal transmission rate. The calculation formula of the signal transmission rate is: , in, Indicates the signal sending rate; Indicates the number of times the data packet needs to be repeated per frame; Indicates the data frame length of a complete data packet when there is only a single signal; Indicates the frame rate; Indicates the time during which information can be read within one frame; Indicates the blind time in a frame where no information can be read; is the duration of one frame; and The rate is L Calculated under different circumstances, It is the maximum fixed transmission rate that the camera can receive when there are no blurred fringes; The above signal transmission rate calculation formula constrains that the data lost during the blind time is not greater than the total frame length that needs to be transmitted, and that the length of the data sent during the time when the signal can be read is not less than the total frame length.

[0023] Step 3: Transmitting the determined mixed optical signal to the optical driver based on the signal transmission rate, and connecting to the LED light source for transmission; S20: Identify the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; Specifically, dimming recognition is required after obtaining the frame image sequence. By identifying the ON time slot and OFF time slot of PWM, the multi-channel data and infrared data sent by the transmitter in the ON time slot and the higher-rate pure infrared data sent in the OFF time slot can be distinguished, so that the RGB&IR image and the pure infrared band image of each frame image can be obtained; Since the color filter array in the CMOS camera is arranged as one red light filter, one blue light filter and two green light filters, and each pixel can only sense light of one color, the CMOS camera is more sensitive to the green component. Therefore, the embodiment of the present application performs dimming recognition based on the G component of each pixel in each frame of the image; Specifically, the G component grayscale value of each pixel extracted is first squared to calculate the G component channel energy of each pixel. The calculation formula is expressed as: , in, Indicates The G component channel energy of each pixel; Indicates The gray value of the G component of each pixel; Indicates the number of pixels; By calculating the G component channel energy of each pixel, the gap between RGB&IR images and pure infrared band images can be enlarged, thereby better separating RGB images and infrared band images; In some embodiments of the present application, the frame image is divided based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image, including: Based on the G component channel energy of all pixels, a G component channel energy curve is obtained; Determine a threshold line based on a preset ratio of the maximum value of the G component channel energy curve, and use the first intersection point of the threshold line and the G component channel energy curve as an initial demarcation point; Taking the initial boundary point as the starting point, traverse the points on the G component channel energy curve in the reverse direction of the x-axis; obtain the points whose slope signs are opposite to the slope signs of the initial boundary points to obtain the target point set; The point in the target point set that is closest to the initial boundary point is taken as the target boundary point, and the frame image is divided based on the target boundary point to obtain the RGB&IR image and the pure infrared band image in the frame image.

[0024] Specifically, after determining the initial dividing point, since the initial dividing point is generally at the rising edge of the curve, it is not the most accurate dividing point. Therefore, in the embodiment of the present application, the initial dividing point is used as the starting point, and the points on the G component channel energy curve where the slope is constant are traversed forward until the slope changes to obtain the target dividing point.

[0025] For example, in a specific example of the present application, the threshold line is determined based on 50% of the maximum value of the G component channel energy curve, and the threshold line is expressed as , Indicates the highest value of the G component channel energy curve; the first intersection of the G component channel energy curve and the threshold line is taken as the initial dividing point, and the points with the same slope are traversed forward on the energy curve where the initial dividing point is located until the slope changes and the target dividing point is obtained. The RGB&IR image and the pure infrared band image obtained based on the target dividing point are divided.

[0026] S30: performing multi-channel signal extraction on the RGB&IR image in each frame of the image to obtain the RGB three-channel signal of the RGB&IR image; and extracting the first infrared signal in the RGB&IR image by using the principal component analysis method; Specifically, since there is a low-rate infrared signal in the RGB&IR image, it is necessary to extract the low-rate infrared signal in the RGB&IR image, and the infrared signal will have its own RGB three-channel data, which will make the original normal RGB three channels affected by the infrared signal. The traditional method can only process a single signal, and it is impossible to extract the infrared signal mixed in the RGB signal; principal component analysis (PCA) is a non-parametric method that can effectively extract a single signal from a mixed signal by performing an orthogonal linear transformation in the direction of the maximum variance. Therefore, in the embodiment of the present application, the principal component analysis method is used to extract the first infrared signal in the RGB&IR image, which specifically includes: Based on the three-channel pixel gray value matrix of RGB&IR image, the decentralized three-channel pixel gray value matrix and its covariance matrix are obtained; Specifically, the calculation formula of the decentralized three-channel pixel gray value matrix is ​​expressed as: , in, Represents a decentralized three-channel pixel grayscale value matrix; Represents the three-channel pixel grayscale value matrix of RGB&IR image; express A vector of all 1s; Represents the mean vector composed of the mean values ​​of each dimension in the three-channel pixel grayscale value matrix of the RGB&IR image; represents transpose; The calculation formula of the covariance matrix of the decentralized three-channel pixel grayscale value matrix is ​​expressed as: , in, Represents the covariance matrix of the decentralized three-channel pixel grayscale value matrix; Perform eigenvalue decomposition on the covariance matrix of the decentralized three-channel pixel grayscale value matrix to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; The principal component matrix is ​​obtained based on the eigenvectors corresponding to the h largest eigenvalues, and the decentralized three-channel pixel gray value matrix is ​​projected based on the principal component matrix to obtain the target matrix; Specifically, the calculation formula of the target matrix is ​​expressed as: , in, represents the target matrix; represents the principal component matrix; The first vector in the target matrix is ​​taken as the first infrared signal in the RGB&IR image.

[0027] Optionally, in some embodiments of the present application, before using the principal component analysis method to extract the first infrared signal in the RGB&IR image, the image can be converted into the HSV format, and then the HSV format image is filtered using an average filter with a filter size of 5 to highlight the stripes carrying infrared signals in the RGB&IR image, and the image is converted into the RGB format after filtering; by performing average filtering on the RGB&IR image, the infrared signal in the RGB&IR image can be made more prominent, thereby accurately extracting the low-rate infrared signal in the RGB&IR image.

[0028] S40: extracting signals from the pure infrared band image in each frame of image to obtain a second infrared signal of the frame of image; Optionally, in some embodiments of the present application, a principal component analysis method may be used to extract the second infrared signal in the frame image.

[0029] S50: obtaining a target sub-signal corresponding to each frame image based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image; and obtaining a demodulated target signal based on the target sub-signals corresponding to all frame images.

[0030] In some embodiments of the present application, the specific implementation steps of step S50 include: Convert the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame of the image into a "0" and "1" matrix respectively; Based on the pixel difference between the starting position and the ending position of each signal frame header in each frame image and the length of the frame header, the downsampling point of each signal in the frame image is calculated, and based on the downsampling point, the "0" and "1" matrices corresponding to the RGB three-channel signal, the first infrared signal and the second infrared signal in the frame image are downsampled respectively to obtain the first downsampling matrix, the second downsampling matrix, the third downsampling matrix and the fourth downsampling matrix of each frame image; Specifically, the calculation formula of the downsampling point is expressed as: , in, represents the downsampling point; Indicates the pixel difference between the start and end positions of the signal frame header in the frame image; Indicates the length of the frame header; A target sub-signal corresponding to each frame image is obtained based on the first down-sampling matrix, the second down-sampling matrix, the third down-sampling matrix and the fourth down-sampling matrix; and a demodulated target signal is obtained based on the target sub-signals corresponding to all frame images.

[0031] After the multi-wavelength wireless optical communication method provided by the present application converts the received mixed light signal into a frame image sequence, since the color filter array in the receiving end is arranged as a red light filter, a blue light filter and two green light filters, and each pixel can only sense light of one color, the receiving end is more sensitive to the green component in the signal. Therefore, the present application can amplify the gap between the mixed signal and the pure infrared signal in the frame image by calculating the G component channel energy of each pixel in each frame image, and then divide the image to obtain the RGB&IR image and the pure infrared band image in each frame image; and, since there will still be low-rate infrared signals in the separated RGB&IR images, the principal component analysis method, as a non-parametric method, can effectively extract the mixed signal by performing orthogonal linear transformation in the direction of maximum variance. single signal, therefore, in addition to extracting the RGB three-channel signal from the RGB&IR image, the present application also uses the principal component analysis method to extract the first infrared signal from the RGB&IR image; then the signal is extracted from the pure infrared band image in each frame of the image to obtain the second infrared signal of the frame image; finally, based on the RGB three-channel signal, the first infrared signal and the second infrared signal in all frame images, the multi-channel signal sent by the transmitter can be accurately restored; the present application separates the RGB&IR image and the pure infrared band image, and then extracts the low-rate infrared signal from the RGB&IR image, thereby separating the infrared signal from the RGB&IR signal to the maximum extent and eliminating the influence of the infrared signal on the RGB signal, so as to accurately restore the multi-channel signal sent by the transmitter and improve the reliability of multi-wavelength wireless optical communication.

[0032] Optionally, since the infrared signal has its own RGB channel components, each channel in the RGB signal will be affected by the infrared signal. Therefore, in some embodiments of the present application, before performing signal extraction on the RGB&IR image in step S30, the image needs to be color corrected and filtered, and the specific steps include: Convert each frame of image into a YCrCb format image, and calculate the standard deviation of the Cb / Cr channel based on the values ​​of the Cb / Cr channel at all pixels in the YCrCb format image; Specifically, the calculation formula of the standard deviation of the Cb / Cr channel is expressed as: , in, Indicates the standard deviation of the Cb / Cr channel; Indicates the first Line The value of the Cb / Cr channel at the pixel point of the column; Indicates the number of row pixels in a YCrCb format image; Indicates the number of column pixels in the YCrCb format image; The standard deviation of the Cb / Cr channel is used as the color correction value to perform color correction on the YCrCb format image; Gaussian filtering is performed on the color-corrected YCrCb format image, the Gaussian-filtered YCrCb format image is converted into an RGB format image to obtain a target RGB image, and multi-channel signal extraction is performed on the target RGB image.

[0033] Optionally, since an infrared signal that is too high will seriously affect other signals and make them unable to be demodulated, and an infrared signal that is too low will make it impossible to extract the infrared signal, a feedback link based on wavelet decomposition is designed in the embodiment of the present application, so that the transmitting end can adjust the intensity of the infrared signal in the mixed optical signal.

[0034] Specifically, based on the fact that infrared signals have the greatest impact on the R channel, the embodiment of the present application uses RGB&IR images as the data source for wavelet decomposition. The basic idea of ​​wavelet decomposition is to use wavelet functions to decompose and reconstruct signals, and decompose the signals into wavelet components of different scales and frequencies. These components include approximate coefficients and detail coefficients, which respectively represent the overall trend and detail characteristics of the signal.

[0035] In some embodiments of the present application, the feedback step of the feedback link includes: after obtaining the RGB&IR image and the pure infrared band image in each frame of the image: Based on dimming recognition and separation, the RGB&IR image sequence is obtained, and the RGB&IR image sequence is decomposed by wavelet to obtain the energy distribution of the RGB&IR image sequence; Specifically, the formula for wavelet decomposition of RGB&IR image sequence is: , in, Represents RGB&IR image sequence; represents the scale coefficient; represents the scaling function; represents the detail coefficient; represents the wavelet function; Indicates the maximum scale of wavelet decomposition, that is, the number of levels of wavelet decomposition; Represents the position index, which is used to indicate the different positions of the signal at different scales; Represents the scale index, indicating the Level of detail coefficient; The energy distribution of RGB&IR image sequence is expressed as: , in, Indicates that the RGB&IR image sequence is Levels of energy; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates A vector consisting of detail coefficients and approximation coefficients; represents the coefficient index; Perform linear fitting on the energy distribution of RGB & IR image sequences and use the least squares method to solve the slope of the fitted line; Specifically, the solution formula for the slope of the fitting line is expressed as: , in, represents the slope of the fitted line; The first observations; The output variable observations; represents the number of observations; The slope of the fitting straight line is normalized to obtain the proportional coefficient of the infrared band light signal in the mixed light signal; Specifically, the formula for obtaining the proportional coefficient by normalizing the slope of the fitting line is expressed as: , in, Represents the proportionality factor.

[0036] It is determined whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitter, so that the transmitter adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

[0037] Specifically, generally, the initial voltage of the transmitting end is set to the rated voltage of the infrared lamp. Therefore, after receiving the feedback signal, the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal by reducing the voltage of the infrared lamp.

[0038] For example, Figure 3 A schematic diagram of the comparison of R channels with and without feedback links provided in this application; wherein, Figure 3 (a) is a schematic diagram of the R channel with a feedback link. Figure 3 (b) is a schematic diagram of the R channel when there is no feedback link.

[0039] Based on the multi-wavelength wireless optical communication method provided in the above embodiment, the embodiment of the present application also provides a multi-wavelength wireless optical communication system, such as Figure 4As shown, the system includes a transmitting end and a receiving end; Specifically, the transmitter includes: A dimming signal input module, used to dim the current signal using the dimming signal; A sending rate calculation module, used to calculate the signal sending rate; A data packet setting module, used for outputting data packet structure information; The original signal generation module is used to generate the data to be sent; The dimming data generation module is used to generate a mixed light signal, send RGB signals and low-rate infrared signals in the ON time slot of PWM, and send high-rate infrared signals in the OFF time slot of PWM; The optical driver is used to drive the LED light source and load the mixed optical signal to be transmitted onto the light source and transmit it through free space.

[0040] The receiving end includes: A signal capture module, used to receive a mixed light signal sent by a transmitting end, and convert the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and a pure infrared band light signal; A dimming recognition and image separation module, which is used to recognize the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; The first signal extraction module is used to perform multi-channel signal extraction on the RGB&IR image in each frame of the image to obtain the RGB three-channel signal of the RGB&IR image; and extract the first infrared signal in the RGB&IR image by using the principal component analysis method; A second signal extraction module is used to extract the signal of the pure infrared band image in each frame of image to obtain the second infrared signal of the frame of image; The data fusion module is used to obtain the target sub-signal corresponding to each frame image based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image; and obtain the demodulated target signal based on the target sub-signals corresponding to all frame images.

[0041] Optionally, in some embodiments, the receiving end further includes a feedback module, which specifically includes: A wavelet transform submodule is used to obtain an RGB&IR image sequence based on dimming recognition separation, and perform wavelet decomposition on the RGB&IR image sequence to obtain an energy distribution of the RGB&IR image sequence; A linear fitting submodule, used for performing a linear fitting on the energy distribution of the RGB&IR image sequence and solving the slope of the fitting line using the least square method; A proportionality coefficient calculation submodule is used to normalize the slope of the fitting straight line to obtain a proportionality coefficient of the infrared band light signal in the mixed light signal; The feedback signal generating submodule is used to determine whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitting end, so that the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

[0042] The effectiveness of the above multi-wavelength wireless optical communication method and system is verified by specific examples below: A random sequence is generated at the transmitting end of the multi-wavelength wireless optical communication system, and the data is divided into the R channel, G channel, B channel and infrared channel based on the data frame structure setting. The low-rate infrared data in the RGB three channels and the infrared channel are loaded in the ON time slot of PWM, and data packets of different lengths are selected for experiments: L is 40 and 90 (L=40 / 90 is the data packet length under full duty cycle); in the OFF time slot of PWM, the high-rate infrared data rate of the infrared channel is a fixed rate of 9.428Kbit / s, which is the maximum transmission rate of unblurred fringes that can be received by the CMOS camera. The frame header length of the data packet is set to 4, and the modulation format of the data is PAM2.

[0043] After the frame structure is constructed, the rate of sending the signal is determined. In this embodiment, the optical camera equipped with a CMOS image sensor used at the receiving end has a "blind" period of about 30% of each frame time. The camera frame rate at the receiving end is set to 30fps, and the data packet is sent twice in each frame image to ensure that at least one complete data packet in a frame can be captured by the camera. The signal transmission rate can be determined: when L=40, R=3.428Kbit / s; when L=90, R=7.714Kbit / s.

[0044] The signal output by the dimming data generation module is output to STM32 through the serial port. In order to enhance the driving capability of STM32's GPIO, the IO port is connected to the LED driver (dual MOS tube amplifier circuit) and then to the RGB LED (CREE XM-L) and IR LED (FANGPU FOVNNSTF3W940NM). Different dimming levels (D = 0%, 10%, 20%, 30%, …, 100%) are obtained by adjusting the different duty cycles of PWM.

[0045] The light signal is transmitted through the channel, and the signal is received by the CMOS camera (OnePlus 8 Pro) at the receiving end. In order to receive the infrared signal and eliminate the blooming effect, it is necessary to remove the infrared filter of the CMOS camera in advance and place a piece of oil paper in front of the camera. The illuminance meter (UT383) is placed next to the camera to measure the current light intensity. The video is output to the computer and demodulated using MATLAB. The distance between the system light and the camera is 20cm, the video resolution is 1920×1080, the exposure time is set to 1 / 15360 s, the sensitivity is set to 6400, and the frame rate is 30fps.

[0046] The feedback module is used to perform wavelet analysis on the R channel of the original frame image. In order to find the appropriate voltage value of the IR LED, it is first set to the rated voltage of 1.8V. The extracted R channel grayscale value is as follows: Figure 3 As shown in (a), it can be seen that at this time, due to the excessively high infrared voltage, the grayscale value of the R channel is saturated and overflowed, and the pixels with low grayscale values ​​are pulled up. The DB4 wavelet function is used to decompose it, and the number of decomposition layers is 5. The energy of each layer is calculated. The ones with the largest energy values ​​can basically represent the signal trend of the signal, so only the first three energy values ​​are selected, and the calculated normalized slope is 2.687. Next, the infrared voltage is reduced at a resolution of 0.1V to find a suitable voltage. Figure 3 (b) in the figure uses the case where the infrared voltage is 1.3V. Repeating the above steps, the normalized slope is 0.4262. Through experiments before communication, it is found that when the slope is in the range of (0,1), the input of the infrared signal is appropriate, thus determining the input infrared voltage.

[0047] After the feedback module determines the intensity of the infrared signal, the data of each channel is extracted: first, the grayscale value of the G channel is extracted from the original RGB frame image, and the energy value of each pixel is calculated. The dividing point of the two images is found using the above-mentioned method of finding the dividing point and used as the dividing line to separate the images of the PWM ON and OFF time slots.

[0048] Then, the mixed signal image is used to extract the RGB channel data. The RGB format image is first converted to YCrCb format, and then the standard deviation of the YCrCb channel is calculated and used as the color balance value. The image is filtered using a two-dimensional Gaussian smoothing kernel with a standard deviation of 1, and finally the RGB three-channel signal of the processed image is extracted.

[0049] The mixed signal image also needs to extract the infrared signal, convert its image format into HSV format, and then filter it with an average filter with a filter size of 5 to highlight the stripes carrying infrared signals. Then the image is converted into RGB format, and finally PCA is used to extract low-rate infrared signals.

[0050] The pure infrared signal image is used to extract the infrared signal, and the high-rate infrared signal is directly extracted by applying PCA.

[0051] The signals of each channel are respectively subjected to threshold judgment using a third-order fitting method to obtain the information of "0" and "1" being sent.

[0052] The data after threshold judgment is downsampled through the downsampling module. In some examples, more frame images can be taken to calculate the corresponding downsampling values, and the final downsampling value of the system is determined by taking the average, so as to reduce the error.

[0053] The signal from the downsampling module is input to the data fusion module, and the data from each channel obtained previously are integrated to obtain a complete data packet.

[0054] Finally, the data from the data integration module is input into the performance evaluation. Figure 5 The bit error rate (BER) of different data packets under different dimming levels is shown. This experiment selects two cases, L = 40 and L = 90. It can be seen from the figure that when the dimming level is 0, that is, only infrared signals are currently transmitted, the interference is minimal, and the bit error rate is the lowest in all cases. As the dimming level increases, the four channel signals interfere with each other, resulting in an increase in the bit error rate of the infrared channel. When L = 40, as the dimming level changes from 0 to 1, the bit error rate of the infrared signal increases from 1.62×10 -3 Increased to 2.44×10 -3 The average bit error rate is from 1.62×10 -3 Increased to 2.99×10 -3 ; When L = 90, the bit error rate is from 1.62×10 -3 Increased to 3.7×10 -2 The average bit error rate is from 1.62×10 -3 Increased to 5.01×10 -2 , but the bit error rate of the three channels of RGB remains almost unchanged. This phenomenon is more obvious when the packet length L = 90, because the bit error rate of pure infrared and the bit error rate of mixed signals are quite different, so the change of dimming level will cause a greater impact on the bit error rate.

[0055] In addition, this embodiment also compares the data rate of this system with that of a common system in which the transmitting end only transmits RGB signals. This system adds an infrared link compared to the RGB system; and compares the dimming. Both this system and the RGB system meet the dimming requirement that the light intensity increases with the dimming level.

[0056] In summary, the multi-wavelength optical camera communication system with the introduction of infrared proposed in this application makes up for the deficiencies of the optical camera-based OCC system in dimming control and improves the data transmission efficiency. The transmitting end mixes the infrared signal with the multi-wavelength signal and reasonably utilizes the OFF time slot of the dimming signal PWM to send a higher-rate infrared signal, which makes up for the insufficient transmission rate under the frame rate limit. The receiving end demodulates the mixed optical signal and dynamically adjusts the intensity of the infrared signal through a feedback link based on wavelet decomposition, especially in the case of multi-signal interference, to ensure the effective extraction of multi-wavelength signals and infrared signals, and to ensure the accuracy and stability of signal extraction.

[0057] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0058] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0061] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A multi-wavelength wireless optical communication method, characterized in that: The receiving end used in the multi-wavelength wireless optical communication system includes: Receive a mixed light signal sent by a transmitting end, and convert the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and an infrared band light signal; Identify the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; Perform multi-channel signal extraction on the RGB&IR image in each frame of image to obtain the RGB three-channel signal of the RGB&IR image; and use the principal component analysis method to extract the first infrared signal in the RGB&IR image; Extracting the signal of the pure infrared band image in each frame of image to obtain the second infrared signal of the frame of image; Based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image, a target sub-signal corresponding to the frame image is obtained; based on the target sub-signals corresponding to all frame images, a demodulated target signal is obtained.

2. The multi-wavelength wireless optical communication method according to claim 1, characterized in that: The frame image is divided based on the G component channel energy of each pixel point, and the RGB&IR image and pure infrared band image in the frame image include: Based on the G component channel energy of all pixels, a G component channel energy curve is obtained; Determine a threshold line based on a preset ratio of the maximum value of the G component channel energy curve, and use the first intersection point of the threshold line and the G component channel energy curve as an initial demarcation point; Taking the initial boundary point as the starting point, traverse the points on the G component channel energy curve in the reverse direction of the x-axis; obtain points whose slope signs are opposite to the slope signs of the initial boundary point, and obtain a target point set; The point in the target point set that is closest to the initial boundary point is taken as the target boundary point, and the frame image is divided based on the target boundary point to obtain the RGB&IR image and the pure infrared band image in the frame image.

3. The multi-wavelength wireless optical communication method according to claim 1, characterized in that: The multi-channel signal extraction of RGB & IR images in each frame includes: Convert each frame of image into a YCrCb format image, and calculate the standard deviation of the Cb / Cr channel based on the values ​​of the Cb / Cr channel at all pixels in the YCrCb format image; Performing color correction on the YCrCb format image by using the standard deviation of the Cb / Cr channel as a color correction value; Gaussian filtering is performed on the color-corrected YCrCb format image, the Gaussian-filtered YCrCb format image is converted into an RGB format image to obtain a target RGB image, and multi-channel signal extraction is performed on the target RGB image.

4. The multi-wavelength wireless optical communication method according to claim 3, characterized in that: The calculation formula of the standard deviation of the Cb / Cr channel is expressed as: ， in, Indicates the standard deviation of the Cb / Cr channel; Indicates the first Line The value of the Cb / Cr channel at the pixel point of the column; Indicates the number of row pixels in a YCrCb format image; Indicates the number of columns of pixels in a YCrCb format image.

5. The multi-wavelength wireless optical communication method according to claim 1, characterized in that: Extracting the first infrared signal from the RGB&IR image using the principal component analysis method includes: Based on the three-channel pixel gray value matrix of RGB&IR image, the decentralized three-channel pixel gray value matrix and its covariance matrix are obtained; Perform eigenvalue decomposition on the covariance matrix of the decentralized three-channel pixel grayscale value matrix to obtain multiple eigenvalues ​​and eigenvectors corresponding to each eigenvalue; A principal component matrix is ​​obtained based on the eigenvectors corresponding to the h largest eigenvalues, and the decentralized three-channel pixel gray value matrix is ​​projected based on the principal component matrix to obtain a target matrix; The first vector in the target matrix is ​​used as the first infrared signal in the RGB&IR image.

6. The multi-wavelength wireless optical communication method according to claim 5, characterized in that: The calculation formula of the decentralized three-channel pixel grayscale value matrix is ​​expressed as: ， in, Represents a decentralized three-channel pixel grayscale value matrix; Represents the three-channel pixel grayscale value matrix of RGB&IR image; express A vector of all 1s; Represents the mean vector composed of the mean values ​​of each dimension in the three-channel pixel grayscale value matrix of the RGB&IR image; represents transpose; The calculation formula of the covariance matrix of the decentralized three-channel pixel grayscale value matrix is ​​expressed as: ， in, Represents the covariance matrix of the decentralized three-channel pixel grayscale value matrix; The calculation formula of the target matrix is ​​expressed as: ， in, represents the target matrix; represents the principal component matrix.

7. The multi-wavelength wireless optical communication method according to claim 1, characterized in that: After obtaining the RGB&IR image and pure infrared band image in each frame, it also includes: Based on dimming recognition and separation, an RGB&IR image sequence is obtained, and wavelet decomposition is performed on the RGB&IR image sequence to obtain energy distribution of the RGB&IR image sequence; Performing a linear fit on the energy distribution of the RGB&IR image sequence, and using the least squares method to solve the slope of the fitted line; Normalizing the slope of the fitting straight line to obtain a proportional coefficient of the infrared band light signal in the mixed light signal; It is determined whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitting end, so that the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

8. The multi-wavelength wireless optical communication method according to claim 7, characterized in that: The formula for wavelet decomposition of the RGB&IR image sequence is: ， Among them, among them, Represents RGB&IR image sequence; represents the scale coefficient; represents the scaling function; represents the detail coefficient; represents the wavelet function; Indicates the maximum scale of wavelet decomposition, that is, the number of levels of wavelet decomposition; Represents the position index, which is used to indicate the different positions of the signal at different scales; Represents the scale index, indicating the Level of detail coefficient; The energy distribution of RGB&IR image sequence is expressed as: ， in, Indicates that the RGB&IR image sequence is Levels of energy; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates The level of detail coefficients and approximation coefficient length vectors; Indicates A vector consisting of detail coefficients and approximation coefficients; represents the coefficient index; The solution formula for the slope of the fitting line is expressed as: ， in, represents the slope of the fitted line; The first observations; The output variable observations; represents the number of observations; The formula for normalizing the slope of the fitted line to obtain the proportional coefficient is expressed as: ， in, Represents the proportionality factor.

9. A multi-wavelength wireless optical communication system, characterized in that: It includes a transmitting end and a receiving end, and the receiving end includes: A signal capture module, used to receive a mixed light signal sent by a transmitting end, and convert the mixed light signal into a frame image sequence; wherein the mixed light signal includes an RGB&IR light signal and a pure infrared band light signal; A dimming recognition and image separation module, which is used to recognize the dimming level of the mixed light signal based on the frame image sequence, calculate the G component channel energy of each pixel point of each frame image in the frame image sequence, and divide the frame image based on the G component channel energy of each pixel point to obtain the RGB&IR image and the pure infrared band image in the frame image; The first signal extraction module is used to perform multi-channel signal extraction on the RGB&IR image in each frame of the image to obtain the RGB three-channel signal of the RGB&IR image; and extract the first infrared signal in the RGB&IR image by using the principal component analysis method; A second signal extraction module is used to extract the signal of the pure infrared band image in each frame of image to obtain the second infrared signal of the frame of image; The data fusion module is used to obtain the target sub-signal corresponding to each frame image based on the RGB three-channel signal, the first infrared signal and the second infrared signal in each frame image; and obtain the demodulated target signal based on the target sub-signals corresponding to all frame images.

10. The multi-wavelength wireless optical communication system according to claim 9, characterized in that: The receiving end also includes a feedback module, which specifically includes: A wavelet transform submodule is used to obtain an RGB&IR image sequence based on dimming recognition separation, and perform wavelet decomposition on the RGB&IR image sequence to obtain an energy distribution of the RGB&IR image sequence; A linear fitting submodule, used for performing a linear fitting on the energy distribution of the RGB&IR image sequence and solving the slope of the fitting line using the least square method; A proportionality coefficient calculation submodule is used to normalize the slope of the fitting straight line to obtain a proportionality coefficient of the infrared band light signal in the mixed light signal; The feedback signal generating submodule is used to determine whether the proportional coefficient is within a preset value range. If the proportional coefficient is not within the preset value range, a feedback signal is generated and sent to the transmitting end, so that the transmitting end adjusts the intensity of the infrared band optical signal in the mixed optical signal based on the feedback signal.

Images