A method for integrating recognition and communication using a reflective optical camera

Abstract

This invention discloses an integrated method for recognition and communication using a reflective optical camera, comprising a recognition method and a communication method. The recognition method includes the following steps: representing a mixed image using stripe features, target background features, and noise; reconstructing the mixed image, assigning the highest weight to the variable most relevant to the target background features; further refining the reconstructed image; and inputting the refined reconstructed image into a convolutional neural network for target recognition and classification. The communication method includes the following steps: dividing the image into an upper region and a lower region; storing the gray values ​​of the pixels with the highest gray values ​​found in the search into separate sets; accumulating the gray value data of the pixels with the highest gray values ​​in the same column to form a gray value sequence to be demodulated; and decoding to reconstruct the transmitted information. This invention solves the problems of the receiving camera's inability to recognize target objects in the environment during communication and the low communication quality in complex backgrounds.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence technology and communication technology, and relates to a method for achieving target recognition and communication of a reflective optical camera in an optical communication scenario by using dissociation characterization technology and camera optical communication signal sampling optimization algorithm. Background Technology

[0002] Optical camera communication (OCC) technology, implemented using light-emitting diodes (LEDs) and embedded cameras, has attracted widespread attention due to its abundant spectrum availability and ease of deployment. Based on the rolling shutter effect, the signal of camera optical communication appears as alternating bright and dark stripes in the image detected by the camera sensor. Therefore, the camera, as the receiving end, can demodulate the signal, thus achieving data communication. Using this technology, LED lights in optical camera communication systems can simultaneously perform illumination and communication functions. However, in reflective camera optical communication systems, the camera, as the receiving end, is limited to receiving and demodulating the signal transmitted by the LED light on the reflective surface; it cannot receive images. Furthermore, the bright and dark stripes formed by the light communication signal transmitted by the LED light and the environmental information received by the camera superimpose on the image and interfere with each other. Therefore, in reflective optical camera communication systems, the camera, as the receiving end, cannot identify target objects in the environment while communicating. Meanwhile, since the reflective surface is often in a complex environment, the bright and dark stripes formed by the camera's optical communication signal are blocked or absorbed by the environment due to the interference of the complex reflective surface. This results in uneven distribution of the optical communication signal in the grayscale sequence to be demodulated and a low signal-to-noise ratio, which affects the reliability of communication. Summary of the Invention

[0003] To address the problems in existing technologies, particularly the inability of the receiving camera to identify target objects in the environment during communication and the low communication quality in complex backgrounds in reflective optical camera communication systems, this invention provides an integrated method for target identification and communication in reflective optical cameras. By employing a deconstruction representation technique from deep learning, the bright and dark stripe features formed by the signal in camera imaging are decoupled from environmental information features, reconstructing the target object image from the environmental information for target identification. Furthermore, a sampling technique for camera optical communication signals in complex backgrounds is applied to reduce the bit error rate and improve communication reliability. This invention overcomes the problems of target identification failure and low communication quality in reflective optical camera communication systems, improving their performance and expanding their application scenarios.

[0004] The technical solution adopted in this invention is:

[0005] A method for integrating recognition and communication using a reflective optical camera, comprising a recognition method and a communication method;

[0006] The identification method includes the following steps:

[0007] S1. Represent the mixed image of the communication signal and the target background received by the receiving camera using the stripe features formed by the communication signal, the target background features, and the noise.

[0008] S2. Apply an attention mechanism to perform hybrid image reconstruction, so that the variables most relevant to the target background features of the reconstructed image are given the highest weights;

[0009] S3. Apply conditional adversarial network structures to further refine the reconstructed images;

[0010] S4. Input the refined reconstructed image into a convolutional neural network for target recognition and classification;

[0011] The communication method includes the following steps:

[0012] T1. Select one line from the received image containing optical communication information as the baseline, and divide the entire image into an upper region and a lower region.

[0013] T2. Stepwise search for the pixel with the largest gray value in each column of the upper and lower regions of the image, and store the gray values ​​of the pixels with the largest gray values ​​found in the upper and lower regions into sets respectively;

[0014] T3. Accumulate the grayscale data of the pixels with the largest grayscale values ​​in the same column in the upper and lower regions that have been searched to form a grayscale sequence to be demodulated.

[0015] T4. Decode the grayscale sequence to be demodulated using a thresholding method to restore the transmitted information.

[0016] Preferably, in step S1, the images are mixed. Represented as:

[0017]

[0018] in, The stripe feature formed by communication signals; For target background features; It is noise.

[0019] Preferably, step S1 further includes the following steps:

[0020] Creating hidden variables In the hidden space Mixed images Describe it;

[0021] Create proxy distribution , will hidden variables For mixed images posterior distribution Approximation is performed using the KL(Kullback–Leibler) divergence, with the following formula:

[0022]

[0023] in, For mixed images Distribution; For mixed images and latent variables The joint distribution;

[0024] For the encoder-decoder network structure in a variational autoencoder, its variational lower bound is:

[0025]

[0026] in, and These are the parameterized representations of the encoder and decoder in the network structure, respectively; The mean squared error reconstruction loss; Distributions generated for the encoder and standard isotropic Gaussian distributions KL divergence between them It is the identity matrix; Latent variables The true distribution;

[0027] Decoder The output result is The loss function of the variational autoencoder is then:

[0028]

[0029] in, These are hyperparameters that can be manually adjusted;

[0030] Then hidden variables They can be represented as two statistically independent sets of latent variables, as shown below:

[0031]

[0032] in, Hidden space The latent variable in the middle represents the characteristics of the light signal stripes; Hidden space The latent variables represent the background features of the target.

[0033] Preferably, step S2 specifically includes:

[0034] The attention mechanism is applied to select relevant features for image reconstruction while masking irrelevant features, by learning latent variables. The weighted combination of all variables in the model gives the highest weight to the variable most relevant to the target background features of the reconstructed image.

[0035] Preferably, step S3 specifically includes:

[0036] The conditional adversarial network architecture includes a generator. Markov discriminator Markov discriminant By generator The generated images are distinguishable from the real data distribution, encouraging the generator. Fitting Prior The provided real data distribution has a loss function defined as:

[0037]

[0038] in, These are the latent variables obtained after steps S1 and S2;

[0039] Combining the loss function of the variational autoencoder, the overall loss function is:

[0040]

[0041]

[0042] .

[0043] Preferably, in step T2, the gray values ​​of the pixels with the largest gray values ​​found in the upper and lower regions are stored in sets respectively. and middle, and They are represented as follows:

[0044]

[0045]

[0046] in, This refers to the grayscale value data of the pixel with the largest grayscale value in each column of the upper region of the image. This refers to the grayscale value data of the pixel with the largest grayscale value in each column in the lower region of the image. and These represent the horizontal and vertical pixel counts in the received full image containing optical communication information, respectively.

[0047] The beneficial effects of this invention are:

[0048] 1. This invention removes the limitation of the camera in a reflective optical camera communication system being used only as a signal receiver, enabling it to receive images and identify targets in the images;

[0049] 2. This invention uses a new camera optical communication signal sampling algorithm, which improves the decoding accuracy and reduces the bit error rate;

[0050] 3. This invention realizes the integrated function of target recognition and communication in a reflective optical camera communication system. Attached Figure Description

[0051] Figure 1 This is a flowchart corresponding to an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the process for identifying the corresponding component in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram illustrating the effect of reconstructing an image in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram illustrating the partitioning of the entire image in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram comparing the pixel grayscale selected by the cumulative sampling optimization method in this embodiment of the invention with the pixel grayscale selected by the traditional method of taking the middle row for sampling; Detailed Implementation

[0056] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] Example

[0058] To address the issues of poor communication quality in reflective optical camera communication systems, where the receiving camera cannot identify target objects in the environment during communication and under complex backgrounds, this embodiment provides an integrated method for recognition and communication using a reflective optical camera. The entire system consists of two functional modules: a recognition module and a communication module. The process is as follows: Figure 1 As shown, the implementation process is explained in detail below.

[0059] For the recognition module, the recognition method and steps are as follows:

[0060] A1. The image of the communication signal received by the receiving camera mixed with the target background. It can be divided into and Two main features: the former represents the stripe features formed by communication signals in the image, and the latter represents the target background features in the image, in contrast to noise. Together can be represented as:

[0061]

[0062] Due to mixed images Since feature decoupling cannot be easily performed in pixel space, variational autoencoder techniques are used to create latent variables. In the hidden space China Describe this. Because the latent variables cannot be easily inferred... For mixed images posterior distribution Create a new agent distribution And it is approximated using the KL divergence. Its formula can be written as:

[0063]

[0064] in, For mixed images Distribution; For mixed images and latent variables The joint distribution;

[0065] For the encoder-decoder network structure in a variational autoencoder, its variational lower bound can be written as:

[0066]

[0067] in, and These are the parameterized representations of the encoder and decoder in the network structure, respectively. The mean squared error reconstruction loss; The distribution generated for the encoder (including the mean) with standard deviation ) and standard isotropic Gaussian distribution KL divergence between them It is the identity matrix; Latent variables The true distribution. At this point, assuming the decoder... The output result is The loss function of the variational autoencoder is then:

[0068]

[0069] in, These are hyperparameters that can be manually adjusted; The distribution generated for the encoder (mean is The standard deviation is ) and a standard isotropic Gaussian distribution (mean 0, standard deviation is the identity matrix) KL divergence between )

[0070] Latent variables It can be represented as two statistically independent sets of latent variables:

[0071]

[0072] in, Hidden space The latent variable in the middle represents the characteristics of the light signal stripes; Hidden space The latent variables represent the background features of the target.

[0073] A2. Although latent variables Able to hide space While the latent variable vector for generating background target features is decoupled, it is still impossible to determine the vector. Therefore, an attention mechanism is applied to select relevant features for image reconstruction while masking irrelevant features. Specifically, this is achieved by learning latent variables... The weighted combination of all variables in the model gives the highest weight to the variable most relevant to the reconstructed background target features.

[0074] A3. Applying conditional adversarial network structures to further refine the reconstructed images, the network structure includes a generator. Markov discriminator Markov discriminant By generator The generated images are distinguishable from the real data distribution, encouraging the generator. Fitting Prior The loss function of the provided real data distribution can be defined as:

[0075]

[0076] in, These are the latent variables obtained after steps A1 and A2;

[0077] Based on the loss function of the variational autoencoder described above, the loss function of the entire functional module can be defined as:

[0078]

[0079]

[0080]

[0081] A4. Input the reconstructed results into a general convolutional neural network for target recognition and classification. The technical flowchart of the recognition module is shown below. Figure 2 As shown in the image. An example of the reconstructed image is shown below. Figure 3 As shown.

[0082] For the communication module, the communication method and steps are as follows:

[0083] B1. After the receiving camera receives the entire image containing optical communication information, it selects one line as the baseline and divides the entire received image in two to facilitate subsequent pixel searches in the upper and lower regions of the image. For example... Figure 4 As shown, select The line in question serves as its baseline, dividing the entire received image into upper and lower parts.

[0084] B2. Proceed by searching for the pixel with the largest gray value in each column of the top and bottom sides of the image, and store the gray values ​​of the high signal-to-noise ratio pixels found into a set. and In Chinese, this is specifically expressed as:

[0085]

[0086]

[0087] in, This refers to the grayscale value data of the pixel with the highest signal-to-noise ratio in each column of the upper region of the image. This refers to the grayscale value data of the pixel with the highest signal-to-noise ratio in each column of the lower region of the image. and These are the horizontal and vertical pixel counts in the entire received image containing optical communication information, respectively. This indicates the resolution of the grayscale image received by the receiving camera.

[0088] B3. Accumulate the grayscale data of high signal-to-noise ratio pixels in the same column in the searched upper and lower regions, that is, accumulate the grayscale value set. and The grayscale values ​​in the corresponding columns are accumulated to form a grayscale sequence to be demodulated. The actual effect is as follows: Figure 5 As shown, the pixel grayscale selected by the cumulative sampling optimization method based on maximum search (e.g.) Figure 5 The upper part of the curve (in the image) is more suitable for selecting pixel grayscale values ​​than the traditional method of sampling the middle row (e.g., ...). Figure 5 The lower part of the curve has a larger signal amplitude and a higher signal-to-noise ratio, and the consistency of the optical communication signal to be demodulated is also better, which is conducive to reducing the bit error rate and improving the communication capacity and reliability of optical camera communication in complex backgrounds.

[0089] B4. Decode the obtained grayscale sequence to be demodulated using a conventional thresholding method to restore the transmitted information.

[0090] In summary, this invention, targeting the object recognition module, first involves manually capturing photographic images in a reflective camera optical communication scenario and preprocessing them to establish a dataset for neural network training. Then, variational autoencoder technology is applied to generate a latent variable space with decoupling feature capabilities, and an attention mechanism is introduced to focus the features to be decoupled onto those relevant to object recognition, i.e., the background target, while masking irrelevant features. Furthermore, this technology further employs a conditional adversarial generative network structure, combined with the variational autoencoder, to refine the reconstructed background target features, thereby improving object recognition accuracy.

[0091] For the communication function module, an accumulated sampling method based on pixel maximum search is applied to progressively search for the pixel with the largest gray value in the upper and lower (or left and right) areas of the communication region and accumulate them to form a gray value sequence to be demodulated. This amplifies the signal strength of the optical camera's communication receiver, mitigates the impact of complex backgrounds on the obstruction and absorption of bright and dark stripes, and reduces problems such as low signal strength and uneven signal distribution, thereby reducing the bit error rate and improving the quality of communication.

[0092] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for integrating recognition and communication using a reflective optical camera, characterized in that, This includes identification methods and communication methods; The identification method includes the following steps: S1. Represent the mixed image of the communication signal and the target background received by the receiving camera using the stripe features formed by the communication signal, the target background features, and the noise. S2. Apply an attention mechanism to perform hybrid image reconstruction, so that the variables most relevant to the target background features of the reconstructed image are given the highest weights; S3. Apply conditional adversarial network structures to further refine the reconstructed images; S4. Input the refined reconstructed image into a convolutional neural network for target recognition and classification; The communication method includes the following steps: T1. Select one line from the received image containing optical communication information as the baseline, and divide the entire image into an upper region and a lower region. T2. Stepwise search for the pixel with the largest gray value in each column of the upper and lower regions of the image, and store the gray values ​​of the pixels with the largest gray values ​​found in the upper and lower regions into sets respectively; T3. Accumulate the grayscale data of the pixels with the largest grayscale values ​​in the same column in the upper and lower regions that have been searched to form a grayscale sequence to be demodulated. T4. Decode the grayscale sequence to be demodulated using a thresholding method to restore the transmitted information.

2. The integrated identification and communication method for a reflective optical camera according to claim 1, characterized in that, In step S1, the images are blended. Represented as: ； in, The stripe feature formed by communication signals; For target background features; It is noise.

3. The integrated identification and communication method for a reflective optical camera according to claim 2, characterized in that, Step S1 also includes the following steps: Creating hidden variables In the hidden space Mixed images Describe it; Create proxy distribution , will hidden variables For mixed images posterior distribution Approximation is performed using the KL divergence formula: ； in, For mixed images Distribution; For mixed images and latent variables The joint distribution; For the encoder-decoder network structure in a variational autoencoder, its variational lower bound is: ； in, and These are the parameterized representations of the encoder and decoder in the network structure, respectively. The mean squared error reconstruction loss; Distributions generated for the encoder and standard isotropic Gaussian distributions KL divergence between them It is the identity matrix; Latent variables The true distribution; Decoder The output result is The loss function of the variational autoencoder is then: ； in, These are hyperparameters that can be manually adjusted; Then hidden variables They can be represented as two statistically independent sets of latent variables, as shown below: ； in, Hidden space The latent variable in the middle represents the characteristics of the light signal stripes; Hidden space The latent variables represent the background features of the target.

4. The integrated identification and communication method for a reflective optical camera according to claim 3, characterized in that, Step S2 is as follows: The attention mechanism is applied to select relevant features for image reconstruction while masking irrelevant features, by learning latent variables. The weighted combination of all variables in the model gives the highest weight to the variable most relevant to the target background features of the reconstructed image.

5. The integrated identification and communication method for a reflective optical camera according to claim 4, characterized in that, Step S3 is as follows: The conditional adversarial network architecture includes a generator. Markov discriminator Markov discriminant By generator The generated images are distinguishable from the real data distribution, encouraging the generator. Fitting Prior The provided real data distribution has a loss function defined as: ； in, These are the latent variables obtained after steps S1 and S2; Combining the loss function of the variational autoencoder, the overall loss function is: ； ； 。 6. The integrated identification and communication method for a reflective optical camera according to claim 5, characterized in that, In step T2, the gray values ​​of the pixels with the largest gray values ​​found in the upper and lower regions are stored in sets respectively. and middle, and They are represented as follows: ； ； in, This refers to the grayscale value data of the pixel with the largest grayscale value in each column of the upper region of the image. This refers to the grayscale value data of the pixel with the largest grayscale value in each column in the lower region of the image. and These represent the horizontal and vertical pixel counts in the received full image containing optical communication information, respectively.

Images