A rolling shutter image processing method based on visible light communication

Abstract

The application discloses a kind of based on visible light communication's rolling shutter image processing method, belong to image processing technical field, including the following steps: S1, obtain the gray value image of rolling shutter based on visible light communication, and carry out noise reduction processing;S2, edge detection is carried out to the image after noise reduction processing, and the real boundary of stripe is detected;S3, establish spatial correspondence, determine boundary, and calculate all boundaries of bright and dark stripe according to the boundary that has been detected;S4, grouping threshold value, extract the gray value of a column of image and carry out grouping fitting to column gray value;S5, carry out logical value judgment to extract information to the gray value, obtain binary gray value data.The application greatly improves the reliability of gray value data in system signal demodulation process, and simplifies the calculation of receiving end image processing.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a rolling shutter image processing method based on visible light communication. BACKGROUND

[0002] In recent years, the Internet technology has developed rapidly, and people's demand for wireless communication technology has increased. The existing 4G, 5G and WIFI electromagnetic signal communication technologies are facing the problem of spectrum resource shortage. Visible light communication combined with mobile communication technology has the advantages of rich spectrum resources, anti-electromagnetic interference and green energy saving, and has high market research value and wide application prospect.

[0003] In the current visible light communication system based on a smartphone camera, the complexity of the received grayscale image processing is high, and the data conversion error rate is high during signal demodulation. When the camera receives information to form a bright and dark stripe pattern, the charge overflows from the saturated pixels to the adjacent pixels, causing the halo effect of the stripes, and the contrast between the bright and dark stripes is still low. In the process of signal demodulation, the grayscale value of each stripe needs to be binarized by threshold value. The current second-order polynomial processing grayscale value algorithm has certain limitations due to the introduction of random clipping noise and high computational complexity, and it is difficult to ensure that the grayscale value data used for signal demodulation has high reliability. SUMMARY

[0004] In view of the above-mentioned deficiencies in the existing encryption technology, the application provides a rolling shutter image processing method based on visible light communication, which solves the problems of low reliability of grayscale value data in the signal demodulation process and high complexity of image processing at the receiving end in the prior art by edge detection and grouping threshold value technology.

[0005] The technical scheme of the application is as follows:

[0006] A rolling shutter image processing method based on visible light communication, comprising the following steps:

[0007] S1, obtaining a rolling shutter grayscale value image based on visible light communication and performing noise reduction processing;

[0008] S2, performing edge detection on the image after noise reduction processing to detect the real boundary of the stripes;

[0009] S3, establishing a spatial correspondence relationship to determine the boundary and calculating all boundaries of the bright and dark stripes according to the detected boundary;

[0010] S4, grouping threshold value, extracting a column of grayscale values of the image and grouping the column of grayscale values;

[0011] S5, performing logical value judgment on the grayscale values to extract information and obtaining binarized grayscale value data.

[0012] Further, the specific process of step S1 is as follows:

[0013] S1.1, obtain the gray value image of recording different width of bright and dark stripes of LED stroboscopic; the gray value stripe image is obtained by the smart phone camera using the rolling shutter mode facing the LED stroboscopic;

[0014] S1.2, the gray value image is processed by Gaussian smoothing using Gaussian filtering technology, and the noise of the gray value image of different width of bright and dark stripes shot by the smart phone using the rolling shutter mode is removed; specifically, the pixel points of the whole gray value image are convolved with the Gaussian kernel to obtain the smoothed gray value image; the convolution process is that all pixel points in the convolution image are weighted with all pixel points in the neighborhood, instead of the value of the center pixel point in the neighborhood.

[0015] Further, the specific process of step S2 is as follows:

[0016] S2.1, the whole gray value image pixel points are respectively convolved with the x direction and y direction of Sobel operator, and the gradient of each pixel point is calculated; the gradient includes amplitude and direction, assuming that the current pixel point is C, the specific calculation formula is:

[0017]

[0018] θ=arctan(GyGx) (2)

[0019] Where, G C(i,j) represents the gradient amplitude of pixel point C, Gx represents the x direction gradient, Gy represents the y direction gradient; θ represents the gradient direction of pixel point C;

[0020] Step S2.2, the non-maximum suppression processing is performed on the edge pixel points, only the pixel point with the largest gray value change in the gradient direction of the pixel is retained, and the remaining pixel points are removed, and then the real boundary of the high-definition but discontinuous stripe is obtained.

[0021] Further, the specific process of step S3 is as follows:

[0022] S3.1, the discontinuous boundary is judged by double threshold value, and the strong edge point, the weak edge point and the non-edge point are determined;

[0023] The double threshold value processing process is to set high and low two threshold values, which are higher than the high threshold value, which are strong edge points, which are between the high threshold value and the low threshold value, which are weak edge points, and which are lower than the low threshold value, which are non-edge points;

[0024] S3.2, the accurate boundary is obtained by using the lag technique, but there are missing pixels; the specific process of obtaining the real boundary by using the lag technique is as follows: all strong edge points are directly reserved as the boundary; all weak edge points are screened, and it is checked whether there is at least one strong pixel point around each weak edge point; if the condition is met, the point is the boundary pixel point of the image, and the pixel point is reserved; otherwise, the weak edge point is removed;

[0025] S3.3, the corresponding relationship between the image space and the parameter space is established, the figure in the image space is determined through the parameter space, and then the accurate stripe boundary in the image space is determined;

[0026] Specifically, the established corresponding relationship between the original image space and the parameter space is the line-point duality of the image space and the parameter space, the point of the original image space corresponds to the straight line of the parameter space, and the intersection of multiple straight lines on the parameter space is equivalent to multiple corresponding points in the original space on the same straight line. The parameter corresponding to the intersection point with the maximum value is calculated. The parameter of the intersection point with the maximum value corresponds to the straight line of the original image space.

[0027] Further, the specific process of step S4 is as follows:

[0028] S4.1, selecting a column of gray value for threshold fitting, grouping the column gray value data according to the change trend of the stripe, and obtaining a sub-gray value column matrix; the specific process is as follows:

[0029] S4.1.1, selecting a column of gray value, and constructing an initial gray value column matrix X;

[0030] X = [x1, x2, …, xm] (3) m ] P (3)

[0031] Wherein, x i (i = 1, 2, …, m) represents the i-th column gray value of the whole group; m represents the number of pixel rows of a frame of image;

[0032] S4.1.2, the column matrix is divided into L sub-column matrices x n by using matrix operation, each sub-column matrix has t elements, and the n-th sub-column matrix is represented by x n ; At this time, the gray value column matrix X is represented as (4),

[0033] X = [x0, x1, …, x L-1 ] P (4)

[0034] S4.1.3, the sub-gray value column matrix X n is obtained by using formula (5),

[0035] X n = [x1+n×t x 2+n×t ,…,x t+n×t ] P (5)

[0036] Wherein, x t+n×t Indicates the full group t+n×t column gray value, satisfy L×t=m;

[0037] S4.2, again to the sub gray value column matrix is fitted with polynomial processing, obtain grouping fitting gray value threshold curve;Fitting process is: the sub gray value column matrix is used as observation set and generates target function, uses polynomial function to fit the generated data, uses mean square error as error function to evaluate the polynomial fitted, minimize the error function through fitting data, and then determine the fitting result, get the fitting curve.

[0038] Further, in step S5, the specific judgment criteria are: if the current pixel gray value is higher than the grouping fitting gray value threshold curve, it is determined as logic value "1", and if it is lower than the threshold, it is determined as logic value "0", and the image information extraction work is completed.

[0039] The beneficial technical effects brought by the present application are:

[0040] Gaussian filter technology is used to remove noise, obtain a smooth gray value image, prevent interference on subsequent edge detection, Sobel operator is used to process the image, obtain the position of the image gray value change, obtain the intensity gradient, the image is processed by non-maximum suppression technology, realize the elimination processing of boundary false detection;The possible boundary is judged by the double threshold method, the image boundary is tracked by using the hysteresis technology, the point-line dual relationship of the rectangular coordinate space and the parameter space of the image is determined, the straight line in the figure is converted into the point in the detection parameter space, and the real edge of the original figure is judged, a group of column gray value matrix is selected from the processed image, the column gray value is grouped, the sub gray value column matrix is obtained, the threshold curve is obtained by polynomial fitting, the logic value corresponding to the selected column gray value is judged according to the threshold curve, the image information extraction operation is completed, and the complexity of image processing is reduced.The present application greatly improves the reliability of gray value data in the system signal demodulation process, and simplifies the calculation of the receiving end image processing. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The flow chart of the present application based on visible light communication rolling shutter image processing method;

[0042] Figure 2 The contrast chart before and after the gray value image preprocessing in the experiment of the present application, wherein (a) is the original gray value image, and (b) is the processed image. DETAILED DESCRIPTION

[0043] The application will be described in further detail below with reference to the drawings and specific embodiments:

[0044] The application provides a rolling shutter image processing method based on visible light communication. The method mainly processes the gray value image of bright and dark stripes obtained by a visible light communication system based on a smartphone camera in a rolling shutter mode. First, the image is preprocessed, including noise reduction processing, edge detection, and determination of the edge. Then, the image is further processed, including grouping threshold fitting. Finally, information is extracted by using a logical value judgment method to obtain binary gray value data with high reliability. The method solves the problems of poor accuracy in information transmission and high error rate in image processing, and realizes efficient transmission of information.

[0045] As shown in Figure 1 , the method specifically includes the following steps:

[0046] Step S1, obtaining a rolling shutter gray value image based on visible light communication and performing noise reduction processing. The specific process is as follows:

[0047] Step S1.1, obtaining a gray value image of bright and dark stripes of different widths recorded by LED strobe. The gray value stripe image is an image obtained by a smartphone camera in a rolling shutter mode facing the LED strobe.

[0048] Step S1.2, to improve the reliability of the gray value in signal processing, Gaussian smoothing processing is performed on the gray value image by using Gaussian filtering technology to eliminate the noise of the gray value image of bright and dark stripes of different widths taken by the smartphone in the rolling shutter mode. Specifically, the pixel points of the entire gray value image are convolved with the Gaussian kernel to obtain a smooth gray value image. In this convolution process, the values of all pixel points in the convolution image are weighted with all pixel points in the neighborhood to replace the value of the center pixel point in the neighborhood.

[0049] Step S2, performing edge detection on the image after noise reduction processing to detect as many real boundaries of the stripes as possible. The specific process is as follows:

[0050] Step S2.1, convolving the pixel points of the entire gray value image with the x direction and y direction of the Sobel operator respectively to calculate the gradient of each pixel point; the gradient includes the amplitude and direction. Assuming that the current pixel point is C, the specific calculation formula is:

[0051]

[0052] θ=arctan(GyGx) (2)

[0053] wherein, G C(i,j)G represents the gradient amplitude of the pixel point C, Gx represents the x direction gradient, Gy represents the y direction gradient, and θ represents the gradient direction of the pixel point C;

[0054] Step S2.2, performing non-maximum suppression processing on the edge pixel points, only keeping the pixel point with the largest gray value change in the gradient direction of the pixel point, and eliminating the remaining pixel points, to obtain a high-definition but discontinuous stripe real boundary.

[0055] Step S3, determining the boundary, and calculating all boundaries of the bright-dark stripes according to the detected boundary. The specific process is as follows:

[0056] Step S3.1, performing double-threshold judgment on the discontinuous boundary to determine strong edge points, weak edge points and non-edge points.

[0057] The double-threshold processing process is to set high and low thresholds, and the strong edge points are higher than the high threshold, the weak edge points are between the high threshold and the low threshold, and the non-edge points are lower than the low threshold.

[0058] Step S3.2, using the hysteresis technique to obtain a boundary with high accuracy but with missing pixel points. The specific process of obtaining the real boundary by using the hysteresis technique is as follows: directly keeping all strong edge points as boundaries; screening all weak edge points, checking whether there is at least one strong pixel point around each weak edge point, if the condition is met, the point is a boundary pixel point of the image, and the pixel point is kept; otherwise, the weak edge point is eliminated. Through the hysteresis technique, part of the weak edge points with high possibility can be screened and kept, and then the pixel points with higher authenticity are left.

[0059] Step S3.3, establishing the correspondence between the image space and the parameter space, determining the graph of the image space through the parameter space, and then determining the accurate stripe boundary of the image space.

[0060] Specifically, the established correspondence between the original image space and the parameter space is the line-point duality of the image space and the parameter space, and the points of the original image space correspond to the straight lines of the parameter space. Multiple straight lines on the parameter space have intersection points, which are equivalent to multiple corresponding points in the original space on the same straight line. The parameters of the intersection point with the largest cumulative number in the parameter space are the parameters of the straight line in the original image space.

[0061] Step S4, establishing the space correspondence, grouping the threshold, extracting a column of gray values of the image, and grouping and fitting the column of gray values. The specific process is as follows:

[0062] Step S4.1, selecting a column of gray values for threshold fitting, grouping the column of gray values according to the stripe change trend, and obtaining a sub-gray value column matrix. The specific process is as follows:

[0063] Step S4.1.1, selecting column gray value, constructing initial gray value column matrix X;

[0064] X = [x1, x2, …, xm] (1) m ] P (3)

[0065] Wherein, x i (i = 1, 2, …, m) represents the i-th column gray value of the whole group; m represents the row number of pixel points of a frame of image.

[0066] Step S4.2.2, the column matrix is divided into L sub-column matrices x n by using matrix operation, each sub-column matrix has t elements, and the n-th sub-column matrix is represented by x n . At this time, the gray value column matrix X can be represented as (4),

[0067] X = [x0, x1, …, xm] (4) L-1 ] P (4)

[0068] Step S4.1.3, obtaining the sub-gray value column matrix X n by using formula (5),

[0069] X n = [x 1+n×t , x 2+n×t , …, x t+n×t ]P (5)

[0070] Wherein, x t+n×t represents the t+n×t-th column gray value of the whole group, and L×t = m is satisfied.

[0071] Step S4.2, the sub-gray value column matrix is further processed by using a polynomial to obtain a grouping fitting gray value threshold curve. The fitting process is as follows: the sub-gray value column matrix is taken as an observation set and a target function is generated, a polynomial function is used to fit the generated data, the mean square error is used as an error function to evaluate the fitted polynomial, the error function is minimized by fitting data, and the fitting result is determined, and the fitting curve is obtained.

[0072] Step S5, the gray value is judged by a logical value to extract information, and the binary gray value data is obtained. The specific judgment criterion is: if the current pixel point gray value is higher than the grouping fitting gray value threshold curve, it is determined as logical value "1"; if it is lower than the threshold, it is determined as logical value "0", and the image information extraction work is completed.

[0073] In order to prove the feasibility and superiority of the present application, relevant experiments are carried out.

[0074] Figure 2(a) for the smartphone camera to take the strobe situation of the LED directly by using the rolling shutter mode, so as to obtain the gray value image of the different width of the bright and dark stripes recording the LED strobe, and it can be seen from the gray value image that the stripe image is very fuzzy. Figure 2 (b) for the image preprocessed by steps 1-3 of the present application, from Figure 2 (b) it can be seen that the bright and dark stripes of the processed image are clearer, thus proving that the method of the present application is feasible and has certain advantages.

[0075] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A rolling shutter image processing method based on visible light communication, characterized by, It comprises the following steps: S1, obtaining a rolling shutter gray value image based on visible light communication, and carrying out noise reduction processing; S2, edge detection is carried out on the image after noise reduction processing, and the real boundary of the stripe is detected; S3, the space correspondence is established, the boundary is determined, and all the boundaries of the bright and dark stripes are calculated according to the detected boundary; S4, grouping threshold, extracting a column of gray value of the image and fitting the column of gray value; S5, logical value judgment is carried out on the gray value to extract information, and binary gray value data is obtained; The specific process of step S1 is as follows: S1.1, obtain the gray value image of the bright and dark stripes with different widths recorded by LED stroboscopic; the gray value stripe image is obtained by the camera of the smart phone in the rolling shutter mode and directly facing the LED stroboscopic; S1.2, Gaussian filtering technology is adopted to perform Gaussian smoothing processing on the gray value image, and the noise of the different width bright and dark stripe gray value images captured by the smartphone using the rolling shutter mode is eliminated; specifically, the pixel points of the entire gray value image are convolved with the Gaussian kernel to obtain a smoothed gray value image; in the convolution process, the values of all pixel points in the convolution image are weighted with all pixel points in the neighborhood, replacing the value of the central pixel point in the neighborhood; The specific process of step S2 is as follows: S2.1, the pixels of the whole gray value image are respectively convolved with the x direction and y direction of Sobel operator, and the gradient of each pixel point is calculated; the gradient includes amplitude and direction, assuming that the current pixel point is C, the specific calculation formula is: (1) (2) wherein, represents the gradient magnitude of the pixel point C, represents the x-direction gradient, represents the y-direction gradient; represents the gradient direction of the pixel point C; Step S2.2, non-maximum suppression processing is carried out on the edge pixel points, only the pixel point with the largest gray value change in the gradient direction of the pixel is retained, and the remaining pixel points are removed, and then the real boundary of the stripe with high definition but not continuous is obtained; The specific process of step S3 is as follows: S3.1, double threshold judgment is carried out on the discontinuous boundary, and strong edge points, weak edge points and non-edge points are determined; The double threshold processing process is to set high and low thresholds, the points higher than the high threshold are strong edge points, the points between the high threshold and the low threshold are weak edge points, and the points lower than the low threshold are non-edge points; S3.2, the lag technique is used to obtain the boundary with high accuracy but with missing pixel points; the specific process of obtaining the real boundary by using the lag technique is: all strong edge points are directly retained as the boundary; all weak edge points are screened, and it is checked whether there is at least one strong pixel point around the eight pixel points of each weak edge point, if the condition is met, the point is the boundary pixel point of the image, and the pixel point is retained; otherwise, the weak edge point is removed; S3.3, the correspondence between the image space and the parameter space is established, the figure of the image space is determined through the parameter space, and the accurate stripe boundary of the image space is determined; Specifically, the established correspondence between the original image space and the parameter space is the line-point duality of the image space and the parameter space, the point of the original image space corresponds to the straight line of the parameter space, and the intersection of multiple straight lines on the parameter space is equivalent to multiple corresponding points on the same straight line in the original space, the cumulative number of the same intersection point in the parameter space is calculated, and the parameter of the intersection point with the maximum value corresponds to the straight line of the original image space; The specific process of step S4 is as follows: S4.1, a column of gray value is selected for threshold fitting, the column of gray value data is grouped according to the change trend of the stripe, and a sub gray value column matrix is obtained; the specific process is as follows: S4.1.1, select column gray value, construct initial gray value column matrix ; (3) wherein, (i = 1, 2,..., m) represents the column gray value of the whole group ; represents the number of pixel rows of a frame of image; S4.1.2, dividing the column matrix into is represented as (4),​​​​​ (4) S4.1.3, obtain the column matrix of sub-gray values by using formula (5) , (5) wherein, represents the full set of the column gray scale values, satisfying ; S4.2, the sub-gray value column matrix is fitted by using a polynomial to obtain a grouped fitting gray value threshold curve; the fitting process is: taking the sub-gray value column matrix as an observation set and generating a target function, using a polynomial function to fit the generated data, using mean square error as an error function to evaluate the fitted polynomial, minimizing the error function by fitting the data, and then determining the fitting result to obtain a fitting curve. 2.The method of claim 1, wherein, In the step S5, the specific judgment criterion is: if the current pixel point gray value is higher than the grouped fitting gray value threshold curve, it is determined as a logic value "1"; if it is lower than the threshold, it is determined as a logic value "0", and the image information extraction work is completed.

Images