An adaptive camera offset recognition method

Abstract

The present invention discloses an adaptive camera offset recognition method, which belongs to the field of image recognition and comprises the following steps: S1: using a Gaussian filter to reduce noise; S2: performing average grayscale values ​​of pixel points to obtain a mean grayscale image; S3: using a Hough transform method to extract straight line features; S4: obtaining intersecting straight line features; S5: performing target detection to obtain intersecting feature regions; S6: performing distribution statistics to obtain respective distribution parameters; S7: using a cumulative distribution method to test the distribution; S8: determining whether the intersecting feature regions contain intersecting straight line features; S9: calculating the slope and intercept of the intersecting straight line features, and determining whether the camera is offset and the corresponding offset by determining the relationship between the slope and the intercept, so as to provide a camera offset warning. The present invention can reduce the influence of a changing environment on recognition and achieve accurate detection of camera offset in a complex and changing environment.

Description

Technical Field

[0001] The present invention relates to an adaptive camera offset recognition method, which belongs to the fields of image recognition and edge computing, and is particularly suitable for camera offset detection and recognition in complex environments. Background Art

[0002] Video surveillance systems are widely used in various fields, including urban safety, smart transportation, smart environmental protection, and border security. For a surveillance system to function effectively, it must first ensure that the scene it monitors is accurate. If the camera's shooting angle shifts due to external forces, causing the scene to shift, the system must promptly detect the anomaly and issue an alarm, alerting maintenance personnel to perform appropriate maintenance.

[0003] There are three main methods for detecting scene offset: pixel difference, histogram matching, and image feature point matching or feature line matching. However, pixel difference and histogram matching are very sensitive to changes in lighting and cannot accurately determine the offset coordinates of scene offset.

[0004] Methods that use feature point matching or feature line matching to assist in identifying camera offsets are mainly represented by the "Scene Change Detection Method, Device, Equipment, and Readable Storage Medium" proposed in CN201910944546.5, and the "Deviation Detection Method, Device, Camera Module, Terminal Equipment, and Storage Medium" proposed in CN202110413657.0. Although these methods can overcome the effects of illumination changes to a certain extent, they are unable to effectively monitor surveillance scenes with relatively clean textures. Furthermore, when there are many moving objects in the image, effective monitoring is also impossible due to the drastic changes in the image feature points. In summary, the image feature point matching algorithm is highly dependent on the image content, which reduces its wide applicability to a certain extent.

[0005] CN202110053803.3 proposed "a scene offset detection method and system based on phase correlation", which uses Fourier transform to judge the cross-power spectrum between the image and the reference image to determine the degree of camera offset. To a certain extent, it can reduce the impact of complex and changeable environments. However, due to the complex and changeable environment, the spectral centers of the two images will shift, resulting in inaccurate calculation results, especially when the camera is obstructed, it will cause recognition errors.

[0006] Based on this, there is an urgent need for a detection and recognition method that can handle camera offset in complex and changing environments. Summary of the Invention

[0007] To solve the technical problem of detecting and identifying camera offsets in the above-mentioned complex and changing environments, the present invention provides an adaptive camera offset identification method. First, continuous images of a period are averaged to reduce the impact of the changing environment on identification. Then, a histogram matching method and an image feature matching method are combined to achieve more accurate identification of the intersection area that changes within the camera period, further reducing the impact of environmental changes on identification. Finally, the camera offset is directly calculated through the slope and intercept of the characteristic line.

[0008] An adaptive camera offset recognition method, characterized by comprising the following steps:

[0009] S1: Use Gaussian filter to reduce noise on the HSV image collected by the camera;

[0010] S2: Convert the HSV image collected by the camera within a recognition cycle into a grayscale image, and average the grayscale values ​​of the pixels to obtain a mean grayscale image;

[0011] S3: Use the Hough transform method to extract the straight line features in the mean grayscale image to obtain the feature map within this recognition cycle;

[0012] S4: Intersect the feature map in the current recognition cycle with the feature map in the initial recognition cycle to obtain a set of intersecting straight line features;

[0013] S5: Use deep learning technology to detect the intersecting straight line features and define the detection area, that is, obtain the intersecting feature area;

[0014] S6: extracting the corresponding intersecting feature areas in the mean grayscale images of the current recognition cycle and the initial recognition cycle, performing distribution statistics on each area, and obtaining their respective distribution parameters;

[0015] S7: Using the cumulative distribution method to check whether the distribution of each intersecting feature area in the current recognition cycle and the initial recognition cycle is the same, and deleting the intersecting feature areas that are not of the same distribution;

[0016] S8: If the intersecting feature area does not contain intersecting straight line features, return to step S2 and perform recognition for the next recognition cycle; when the intersecting feature areas of multiple consecutive recognition cycles do not contain intersecting straight line features, a fault alarm is issued; if the intersecting feature areas contain intersecting straight line features, the process proceeds to step S9;

[0017] S9: Establish an image coordinate system, and obtain the slope and intercept of the intersecting straight line features of the current recognition cycle and the initial recognition cycle respectively. By judging the relationship between the slope and intercept, determine whether the camera is offset and the corresponding offset amount, so as to provide a camera offset warning.

[0018] Furthermore, the length of the recognition cycle can be adjusted according to specific needs using an adaptive method. In particular, the length of the cycle can be determined based on the number of intersecting feature regions calculated in step S6, which can improve recognition accuracy and control the amount of calculation.

[0019] Furthermore, in order to reduce the amount of calculation and specifically extract obvious straight line features, the mean grayscale image in step S3 needs to be compressed before the Hough transform is performed.

[0020] Furthermore, the Hough transform described in step S3 is a method for extracting lines and circles. Compared with other detection methods, it can better reduce noise interference and is beneficial for extracting contour information. The implementation of the Hough transform in Matlab requires the following three steps:

[0021] (1) Use hough() function to perform Hough transform and obtain Hough matrix;

[0022] (2) Use the houghpeaks() function to find the peak point in the Hough matrix;

[0023] (3) Use the houghlines() function to obtain the contour information of the original image based on the results of the previous two steps.

[0024] In the application, you can first use Matlab to write the Hough transform program, and then call the Matlab file through Python.

[0025] Furthermore, the step S4 is specifically as follows:

[0026] S401: searching and matching the feature points included in the feature line of the feature map in the current recognition cycle with the feature points included in the feature line of the feature map in the initial recognition cycle;

[0027] S402: Extracting successfully matched feature points from the feature lines to form a new set of feature lines, that is, a set of intersecting line features.

[0028] In particular, since there may be errors in the successfully matched feature lines, Hamming distance can be used for filtering.

[0029] Furthermore, the deep learning technology described in step S5 is one of Faster R-CNN, SSD and YOLO target recognition models.

[0030] Furthermore, in order to further reduce the interference of changes in the external environment on the intersecting straight line features, the mean grayscale image of the current recognition cycle and the initial recognition cycle described in step S6 can be an HSV image of the mean of the current recognition cycle and the initial recognition cycle, and three distributions corresponding to the three primary colors are established for each pair of intersecting feature areas.

[0031] Furthermore, the specific process of step S6 is: with grayscale or color level as the horizontal axis and frequency as the vertical axis, the relationship between frequency and grayscale or color level is plotted. The image is the histogram of an image, and the distribution of the texture feature information of the feature map can be viewed based on the histogram. The reference for grayscale level is:

[0032] Liu Jianzhuang, Li Wenqing. Two-dimensional Otsu automatic threshold segmentation method for grayscale images[J]. Acta Automatica Sinica, 1993(01):101-105.

[0033] Further, the step S9 is:

[0034] S901: Establish an image coordinate system based on the maximum monitoring angle of the camera as the boundary and the pixel size as the minimum scale;

[0035] S902: Fitting the coordinates corresponding to the feature points of each pair of intersecting straight line features using the least squares method to obtain the slope and intercept;

[0036] S903: Calculate the slope difference and intercept difference of each pair of intersecting straight line features;

[0037] S094: Calculate the mean of the slope differences and the mean of the intercept differences of all intersecting straight line features, which are the image offset angle and offset distance;

[0038] S095: When the offset angle and the offset distance are greater than the set threshold, an alarm indicating that the camera is offset is output; otherwise, a result indicating that the camera is not offset is output.

[0039] Furthermore, in order to better eliminate interference and improve recognition accuracy, step S904 can use the K-means method to perform two-center clustering, clustering the intersecting straight line features of the current recognition cycle and the initial recognition cycle respectively, and directly calculate the image offset angle and offset distance through the cluster center.

[0040] The beneficial effects of the present invention are as follows: the present invention provides an adaptive camera offset recognition method, which first averages the continuous images of the recognition period to reduce the impact of the changing environment on the recognition; then, the image straight line feature matching method is combined with the distribution statistics of the intersection area to achieve more accurate recognition of the common straight line features of the image within the camera recognition period, further reducing the impact of environmental changes on the recognition; finally, the slope and intercept of the straight line feature are used to directly realize the accurate detection of the camera offset in a complex changing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A structural diagram of an adaptive camera offset recognition method;

[0042] Figure 2 It is a matching diagram of intersecting characteristic lines in the embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0044] Example: For a camera installed at an intersection in a smart transportation system, due to long-term external forces, the camera may be offset. It is now necessary to automatically detect and warn of camera offset. Considering that there are many vehicles passing through the intersection and it is prone to congestion, the image at two moments may change greatly or the common area may be blocked. This example provides "an adaptive camera offset recognition method."

[0045] Combine Figure 1 , which includes the following steps:

[0046] Step 1: Preprocessing: Use Gaussian filter to reduce noise on the HSV image captured by the camera; use the histogram averaging of the cumulative distribution function to uniformly adjust the brightness of the image to make the recognition effect more accurate.

[0047] Step 2: Set the length of the recognition cycle, convert the HSV image collected by the camera within one recognition cycle into a grayscale image, and average the grayscale values ​​of the pixels to obtain a mean grayscale image.

[0048] The length of the recognition period can be adjusted according to specific needs using an adaptive method, which is the number of frames of continuous images.

[0049] Step 3: Use the Hough transform method to extract the straight line features in the mean grayscale image after the grayscale level is compressed and mapped to the integer level [0~255] to obtain the feature map within this recognition cycle.

[0050] The Hough transform is a method used to extract lines and circles. Compared with other detection methods, it can better reduce noise interference and is beneficial for extracting contour information. The implementation of the Hough transform in Matlab requires the following three steps:

[0051] (1) Use hough() function to perform Hough transform and obtain Hough matrix;

[0052] (2) Use the houghpeaks() function to find the peak point in the Hough matrix;

[0053] (3) Use the houghlines() function to obtain the contour information of the original image based on the results of the previous two steps.

[0054] Step 4: Intersect the feature map of this recognition cycle with the feature map of the initial recognition cycle to obtain a set of intersecting straight line features. Specifically:

[0055] 1) Search and match the feature points contained in the feature line of the feature map in the current recognition cycle with those in the feature map of the initial recognition cycle;

[0056] 2) The successfully matched feature points in the feature lines are extracted to form a new set of feature lines, that is, a set of intersecting line features.

[0057] The format of the feature line is: feature line = {{feature line 1 = {feature point 11, feature point 12, ...}, {feature line 2 = {feature point 21, feature point 22, ...}}, ...}.

[0058] Step 5: Use the YOLOv3 network to classify and locate the intersecting straight line features for target detection. The anchor box (Anchor Box) delineates the detection area, that is, the intersecting feature area is obtained.

[0059] Step 6: Extract the corresponding intersecting feature areas in the mean HSV images of the current recognition cycle and the initial recognition cycle, use the color levels of the three primary colors as the horizontal coordinate and the frequency as the vertical coordinate, and plot the relationship between the frequency and the color levels of the three primary colors. The image is the histogram of an image. The distribution status of the texture feature information of the feature map can be viewed based on the histogram, and distribution statistics can be performed separately to obtain their respective distribution parameters.

[0060] Step 7: Use the cumulative distribution method to check whether the distribution of each intersecting feature area in the current recognition cycle and the initial recognition cycle is the same, and delete the intersecting feature areas that are not of the same distribution.

[0061] Step 8: If the intersecting feature area does not contain intersecting straight line features, return to step S2 and perform recognition for the next recognition cycle; when the intersecting feature area does not contain intersecting straight line features for 5 consecutive recognition cycles, a fault alarm is issued; if the intersecting feature area contains intersecting straight line features, proceed to step S9;

[0062] Step 9: Establish an image coordinate system, calculate the slope and intercept of the intersecting straight line features of the current recognition cycle and the initial recognition cycle, and determine whether the camera is offset and the corresponding offset by judging the relationship between the slope and intercept, so as to provide a camera offset warning. Specifically,

[0063] a) Establish an image coordinate system with the camera's maximum monitoring angle as the boundary and the pixel size as the minimum scale;

[0064] b) fitting the coordinates corresponding to the feature points of each pair of intersecting straight line features using the least squares method to obtain the slope and intercept;

[0065] c) Calculate the slope difference and intercept difference of each pair of intersecting straight line features; at this time, if there is an abnormal difference, you can adjust the recognition cycle length or compare multiple recognition cycles to determine whether the characteristic line corresponding to the difference is misjudged;

[0066] d) Calculate the mean of the slope differences and the mean of the intercept differences of all intersecting straight line features, which are the image offset angle and offset distance;

[0067] e) When the offset angle and offset distance are greater than the set threshold of 5°, an alarm indicating camera offset is output; otherwise, a result indicating that the camera is not offset is output.

[0068] The present invention is not limited to the above-mentioned embodiments. For those skilled in the art, any improvements or changes made to the above-mentioned embodiments of the present invention will not exceed the scope of protection of the embodiments of the present invention and the appended claims, which are shown only by way of example. The described embodiments are only intended to facilitate the understanding of the present invention and do not have any limiting effect on it.

Claims

1. An adaptive camera offset recognition method, characterized in that: The following steps are involved: S1: Use Gaussian filter to reduce noise on the HSV image collected by the camera; S2: Convert the HSV image collected by the camera within a recognition cycle into a grayscale image, and average the grayscale values ​​of the pixels to obtain a mean grayscale image; S3: Use the Hough transform method to extract the straight line features in the mean grayscale image to obtain the feature map within this recognition cycle; S4: Intersect the feature map in the current recognition cycle with the feature map in the initial recognition cycle to obtain a set of intersecting straight line features; S5: Use deep learning technology to detect the intersecting straight line features and define the detection area, that is, obtain the intersecting feature area; S6: extracting the corresponding intersecting feature areas in the mean grayscale images of the current recognition cycle and the initial recognition cycle, performing distribution statistics on each area, and obtaining their respective distribution parameters; S7: For each intersecting feature area, use the cumulative distribution method to check whether the distribution of the current recognition cycle and the initial recognition cycle is the same, and delete the intersecting feature areas that are not the same distribution; S8: If the intersecting feature area does not contain intersecting straight line features, return to step S2 and perform recognition for the next recognition cycle; when the feature area that intersects for multiple consecutive recognition cycles does not contain intersecting straight line features, a fault alarm is issued; if the intersecting feature area contains intersecting straight line features, the process proceeds to step S9; S9: Establish an image coordinate system, and obtain the slope and intercept of the intersecting straight line features of the current recognition cycle and the initial recognition cycle respectively. By judging the relationship between the slope and intercept, determine whether the camera is offset and the corresponding offset amount, so as to provide a camera offset warning.

2. The method for identifying an adaptive camera offset according to claim 1, wherein: The length of the identification period is adjusted according to specific needs using an adaptive method.

3. The method for identifying an adaptive camera offset according to claim 1, wherein: The mean grayscale image in step S3 needs to be compressed before the Hough transform is performed.

4. The method for identifying an adaptive camera offset according to claim 1, wherein: The step S4 is specifically as follows: S401: searching and matching the feature points included in the feature line of the feature map in the current recognition cycle with the feature points included in the feature line of the feature map in the initial recognition cycle; S402: Extracting successfully matched feature points from the feature lines to form a new set of feature lines, that is, a set of intersecting line features.

5. The method for identifying an adaptive camera offset according to claim 1, wherein: The deep learning technology described in step S5 is one of Faster R-CNN, SSD and YOLO target recognition models.

6. The method for identifying an adaptive camera offset according to claim 1, wherein: The mean grayscale image of the current recognition cycle and the initial recognition cycle described in step S6 is the HSV image of the mean of the current recognition cycle and the initial recognition cycle, and three distributions corresponding to the three primary colors are established for each pair of intersecting feature areas.

7. The method for identifying an adaptive camera offset according to claim 1, wherein: The step S9 is as follows: S901: Establish an image coordinate system based on the maximum monitoring angle of the camera as the boundary and the pixel size as the minimum scale; S902: Fitting the coordinates corresponding to the feature points of each pair of intersecting straight line features using the least squares method to obtain the slope and intercept; S903: Calculate the slope difference and intercept difference of each pair of intersecting straight line features; S904: Calculate the mean of the slope differences and the mean of the intercept differences of all intersecting straight line features, which are the image offset angle and offset distance; S905: When the offset angle and the offset distance are greater than the set threshold, an alarm indicating that the camera is offset is output; otherwise, a result indicating that the camera is not offset is output.

8. The adaptive camera offset recognition method according to claim 7, characterized in that: The step S904 adopts the K-means method to perform two-center clustering, clustering the intersecting straight line features of the current recognition cycle and the initial recognition cycle respectively, and directly calculating the image offset angle and offset distance through the cluster centers.

Images