Method, apparatus and storage medium for setting global sensitive area of variable camera

Abstract

The present invention relates to the technical field of video surveillance, and discloses a method, device and storage medium for setting a globally sensitive area of a variable camera. The method includes determining whether polygons formed by elements of vertex coordinate sets of sensitive areas in a detected image and a reference image are similar polygons, removing parts of the sensitive areas outside the image frame area according to the relationship between each vertex of the sensitive area and the boundary of the detected image frame, and determining the positions of each vertex of the sensitive area within the range of the detected image, so as to determine the sensitive area. Compared with the prior art, the present invention obtains the projection transformation relationship between the detected image and the reference image through comparison of image feature points, determines the position of the sensitive area on the detected screen, without the need to obtain the shooting direction parameter and lens zoom parameter from the camera device, and also avoids the positioning deviation caused by motion accumulation error or fixed device structure deformation, improving the accuracy of the sensitive area monitored by the camera and the use efficiency of the monitoring system.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and particularly relates to a method, device, and storage medium for setting global sensitive areas of a variable camera. Background Art

[0002] With the increasingly widespread application of video surveillance technology in production, life, and social management activities, many applications require setting some special areas in the camera's field of view, and these special areas need to be either key monitored, not allowed to be monitored, or require special image processing, etc.

[0003] Currently, the cameras used in video surveillance technology mainly include different structural and functional types such as bullet cameras, small dome cameras, large dome cameras, integrated cameras, and PTZ cameras. Some of them are fixed cameras with unchanging direction and focal length, and some are cameras with variable direction or focal length. When a fixed camera is equipped with a pan-tilt device, the change of the monitoring direction can also be achieved.

[0004] For a variable camera whose direction and focal length can be changed separately or simultaneously, the special areas set in the monitoring screen need to have global common characteristics. As the monitoring direction and the camera's focal length change, the positions of the special areas need to be adjusted accordingly to ensure that the relevant areas can be accurately covered. For a camera that can accurately determine the current pointing direction and lens focal length, under the condition that the initial position and initial focal length of the camera remain fixed, the position of the sensitive area in the image after the change of direction and focal length can be calculated through the mathematical relationship between the deflection angle and the zoom ratio. If the initial position of the camera changes due to long-term rotation of the camera and telescoping of the lens, or if the installation and fixing structure deforms due to external force factors, the situation of deviation in area positioning will occur. Summary of the Invention

[0005] Aiming at the above defects in the prior art, the present invention proposes a method, device, and storage medium for setting global sensitive areas of a variable camera.

[0006] The technical solution of the present invention is realized as follows:

[0007] A method for setting global sensitive areas of a variable camera includes:

[0008] 1) Obtain an image containing the sensitive area to be set as a reference image P, and calibrate the positions of the vertices of the polygon formed by the sensitive area on the reference image P, record the coordinates of each vertex in the reference image P, and obtain a vertex coordinate set {S};

[0009] 2) Detect image feature points based on the reference image P to obtain a feature point coordinate and feature value set {F} of the reference image P;

[0010] 3) Detect whether the screen of the reference image P has moved. If the screen has not moved, loop and execute this step. If the screen has moved, extract a detection image Pj, and based on this detection image Pj, perform image feature point detection using the same method as the feature point detection of the reference image P to obtain the feature point coordinates and eigenvalue set {Fj} of the detection image Pj;

[0011] 4) Perform feature matching on the elements in the sets {F} and {Fj} to obtain a set of matching feature point pairs {MP}. Check the spatial geometric relationships of the elements in the set of matching feature point pairs {MP}, and remove the elements of the mismatched feature point pairs from the set of matching feature point pairs {MP};

[0012] 5) Use the set of matching feature point pairs {MP} to calculate the projective transformation matrix between the detection image Pj and the reference image P, and then calculate the coordinate position set {Sj} of the elements in the set of sensitive area vertex coordinates {S} in the plane where the detection screen is located;

[0013] 6) Determine whether the polygons formed by the elements of the sets {Sj} and {S} are similar polygons;

[0014] 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, remove the part of the sensitive area outside the image screen area, and determine the positions of each vertex of the sensitive area within the detection screen range.

[0015] Preferably, in step 3), at a fixed inspection time interval t, decode a single-frame picture from the video image captured by the surveillance camera as the detection image, and continuously detect whether the camera screen has moved.

[0016] Preferably, in step 2), the types of detection image feature points include one or more of SIFT feature points, SURT feature points, ORB feature points, BRISK feature points, and AKAZE feature points.

[0017] Preferably, in step 4), the methods of feature matching include BF brute-force matching and FLANN matching.

[0018] Preferably, in step 4), the methods of spatial geometric relationship verification include RANSAC, PROSAC, and LMedS.

[0019] Preferably, in step 6), the methods of judging polygon similarity include but are not limited to Hu moment comparison, Fréchet distance comparison, and angle difference comparison.

[0020] Preferably, in step 1), multiple sensitive areas are set in the same reference image, and these sensitive areas may not cross, partially cross, or contain each other.

[0021] Preferably, in step 1), multiple reference images are obtained, and multiple sensitive regions are set in the multiple reference images. These reference images may not cross, partially cross, or contain each other, and these sensitive regions may also not cross, partially cross, or contain each other.

[0022] Preferably, image preprocessing is performed on the detection image, and the processing methods include one or more of image scaling, image sharpening or blurring, brightness equalization, and image defogging and denoising.

[0023] The present invention also discloses a device for setting global sensitive regions of a variable camera, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method for setting global sensitive regions of the variable camera is implemented.

[0024] The present invention also discloses another device for setting global sensitive regions of a variable camera, including:

[0025] A data communication module, configured to obtain the image data of the reference image and the detection image, obtain the position data of the sensitive regions in the reference image, and send the position data of the sensitive regions in the detection image;

[0026] A data processing module, configured to determine the similarity matching, accuracy verification of the feature point sets and feature points in the reference image and the detection image, and calculate the projection matrix and the vertex positions of the sensitive regions; and is also configured to compare the similarity of the sensitive region polygons, and determine the actual region position of the sensitive region in the detection image according to the positional relationship between the current detection screen boundary and the sensitive region.

[0027] The present invention discloses a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute the method for setting global sensitive regions of the variable camera.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention obtains the projection transformation relationship between the detection image and the reference image through the comparison of image feature points, and then determines the position of the sensitive region on the detection screen, without the need to obtain the shooting direction parameter and the lens zoom parameter from the camera device, and also avoids the positioning deviation caused by the motion accumulation error or the deformation of the fixed device structure, which is beneficial to improving the accuracy of the sensitive region monitored by the camera and improving the use efficiency and safety of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of the method for setting global sensitive regions of the variable camera of the present invention;

[0031] Figure 2 Functional block diagram of the device for setting the global sensitive area of the variable camera according to the present invention. Specific embodiments

[0032] The present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The method for setting the global sensitive area of the variable camera provided by the embodiment of the present invention determines the position of the sensitive area on the detection screen by obtaining the projective transformation relationship between the detection image and the reference image, without the need to obtain the shooting direction parameter and the lens zoom parameter from the camera device, and is not affected by the motion accumulation error or the deformation of the fixed device structure, realizing the setting of the sensitive area of the monitoring screen in the global range.

[0034] As Figure 1 shown, the flowchart of the method for setting the global sensitive area of the variable camera according to the present invention includes:

[0035] 1) Obtain an image containing the sensitive area to be set as the reference image P, calibrate the positions of the vertices of the polygon formed by the sensitive area on the reference image P, record the coordinates of each vertex in the reference image P, and obtain the vertex coordinate set {S};

[0036] 2) Detect the feature points of the reference image P to obtain the feature point coordinates and the eigenvalue set {F} of the reference image P;

[0037] 3) Detect whether the screen of the reference image P has moved. If the screen has not moved, loop to execute this step. If the screen has moved, extract a detection image Pj, and based on this detection image Pj, perform image feature point detection using the same method as the feature point detection of the reference image P to obtain the feature point coordinates and the eigenvalue set {Fj} of the detection image Pj;

[0038] 4) Perform feature matching on the elements in the sets {F} and {Fj} to obtain the set of matching feature point pairs {MP}, perform spatial geometric relationship verification on the elements in the set of matching feature point pairs {MP}, and remove the mis-matched feature point pair elements from the set of matching feature point pairs {MP};

[0039] 5) Calculate the projective transformation matrix between the detection image Pj and the reference image P using the set of matching feature point pairs {MP}, and further calculate the coordinate position set {Sj} of the elements in the sensitive area vertex coordinate set {S} in the plane where the detection screen is located;

[0040] 6) Determine whether the polygons formed by the elements of the sets {Sj} and {S} are similar polygons;

[0041] 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, remove the part of the sensitive area outside the image screen area, and determine the positions of each vertex of the sensitive area within the detection screen range.

[0042] In the embodiment of the present invention, the method specifically includes:

[0043] S1, obtain an image containing the sensitive area to be set as the reference image P, and calibrate the positions of each vertex of the polygon formed by the sensitive area on the reference image P, record the coordinates of each vertex in the reference image P, and obtain the vertex coordinate set {S};

[0044] S2, detect image feature points based on the reference image P. The types of detected image feature points include but are not limited to SIFT feature points, SURT feature points, ORB feature points, BRISK feature points, and AKAZE feature points, etc., and obtain the feature point coordinates and eigenvalue set {F} of the reference image P;

[0045] Specifically, to detect the ORB feature points of the image, it is necessary to calculate the feature points in the reference image P. Calculating the feature points means finding the corner points in the reference image P. The method is to use the downsampling method for the reference image P to obtain an image pyramid; for each pixel point p in each layer of the image pyramid, according to the brightness Ip of the p point; set an interval threshold T (exemplarily, T = Ip * 20%); in the same-level image, for the 16 pixel points on the circumference with a radius of 3 pixels centered on the pixel p, determine whether there are consecutive N points whose brightness is outside the interval [Ip - T, Ip + T]. Exemplarily, N = 12, 9, or 11, etc. If so, determine that the p point is a feature point. The direction of the feature point is determined by the gray centroid method, that is, for the surrounding small piece of image area with the pixel point p as the geometric center, find the average brightness in the horizontal and vertical directions, and use the arctangent function of the vertical direction brightness average value and the horizontal direction brightness average value to represent the feature direction of the feature point p.

[0046] Exemplarily, to detect the ORB feature points of the image, it is necessary to calculate the BRIEF descriptor of the feature values of the feature points in the reference image P. The method is: use a Gaussian filter with a variance of σ and a size of N x N to perform Gaussian smoothing on the area Z with a size of S * S centered on the feature point. Exemplarily, S = 31, N = 9, σ = 2; randomly select d pairs of pixel point pairs within the area Z according to the Gaussian distribution law, compare the brightness values of the two points in each point pair in sequence, and assign 1 or 0 to the corresponding bits of the eigenvalue in sequence, so as to obtain a d-bit eigenvalue. Exemplarily, d = 128, 256, or 512, etc.

[0047] S3. At regular intervals of time t, detect whether the screen has moved. If the screen has not moved, continue to execute this step S2. If the screen has moved, extract a detection image Pj, and based on this detection image Pj, perform image feature point detection using the same method as the reference image feature point detection to obtain the feature point coordinates and eigenvalue set {Fj} of the detection image Pj;

[0048] S4. Perform feature matching on the elements in the sets {F} and {Fj} to obtain a set of matching feature point pairs {MP}. The feature matching methods include but are not limited to methods such as BF brute-force matching and FLANN matching. Perform spatial geometric relationship verification on the elements in the set of matching feature point pairs {MP}, and remove the mis-matched feature point pair elements from the set {MP}. The spatial geometric relationship verification methods include but are not limited to methods such as RANSAC, PROSAC, and LMedS;

[0049] Specifically, using the BF brute-force matching method, for the feature point coordinates and eigenvalue set {F} of the reference image P, find the eigenvalue with the smallest distance from the feature point coordinates and eigenvalue set {Fj} of the detection image Pj as the matching point.

[0050] Exemplarily, a cross-matching method can be adopted to optimize the matching accuracy, that is, when the eigenvalue in the reference image and the eigenvalue in the detection image are in the best match, they are confirmed as a matching point pair, so as to avoid the matching error of one point matching multiple points.

[0051] Exemplarily, a method of setting a comparison threshold can be adopted to optimize the matching accuracy. Exemplarily, the threshold is set to [0.5, 0.9]. When the ratio of the distance between the optimal match and the sub-optimal match satisfies the threshold condition, it is confirmed as a matching point pair.

[0052] S5. Use the set of matching feature point pairs {MP} to calculate the projection transformation matrix M between the detection image Pj and the reference image P, and then calculate the coordinate position set {Sj} of the elements in the sensitive area vertex coordinate set {S} in the plane where the detection image screen is located. The plane where the detection image screen is located refers to the detection screen and the entire plane extending along the periphery of the detection screen;

[0053] S6. Determine whether the polygons formed by the elements in the sets {Sj} and {S} are polygons with similar shapes. If the shapes are similar, continue to execute step S6, otherwise execute step S2. The methods for determining similar shapes include but are not limited to Hu moment comparison, Fréchet distance comparison, and angle difference comparison, etc.;

[0054] Specifically, the above Hu moment is a set of invariant features used to describe the contour shape, which are 7 invariant moments calculated based on the normalized central moments and can maintain invariant features under translation, rotation, and scaling.

[0055] Specifically, the above Fréchet distance is a measure of the similarity between curves. It takes into account the position and order relationship of points along the curves and is the infimum of the maximum distance between two curves.

[0056] S7. According to the relationship between the vertices of the sensitive area and the boundary of the detection screen, remove the part of the sensitive area outside the image screen area, and determine the positions of the vertices of the sensitive area within the detection screen range and on the detection screen boundary.

[0057] In an actual video surveillance system, there are many ways to form a camera with variable direction and / or variable focal length. There are intelligent surveillance cameras with a very high level of intelligence, as well as variable direction surveillance cameras composed of a simple fixed focal length gun-type camera configured with a pan-tilt device, or integrated cameras without a steering device but with a zoom lens.

[0058] The method for setting the global sensitive area of the variable camera according to the embodiments of the present invention does not require relying on the camera rotation and focal length control system to provide direction and focal length parameters under the application conditions of an actual video surveillance system. It completely relies on image correlation analysis and multiple verifications to determine the position of the sensitive area, and has the characteristics of simple application configuration, high reliability in long-term use, and stable and accurate positioning.

[0059] Optionally, in some embodiments, multiple sensitive areas can be set in the same reference image, and these sensitive areas may not cross, partially cross, or contain each other.

[0060] Optionally, in some embodiments, multiple sensitive areas can be set in multiple reference images. These reference images may not cross, partially cross, or contain each other, and these sensitive areas may also not cross, partially cross, or contain each other.

[0061] Optionally, in some embodiments, image preprocessing can be performed on the detected image, including but not limited to image scaling, image sharpening or blurring processing, brightness equalization, and image dehazing and denoising processing.

[0062] Optionally, reference images can be obtained respectively at different time periods, such as during the day and at night, to set the same sensitive area.

[0063] Optionally, reference images can be obtained respectively in different seasons, such as in summer and winter, to set the same sensitive area.

[0064] Exemplarily, in step S2, t = 1 second can be selected, that is, a current image is obtained at a fixed time interval of once per second for relevant detection.

[0065] Such as Figure 2As shown in the figure, an embodiment of the present invention further provides a device 20 for setting the globally sensitive area of a variable camera. The device consists of a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the method for setting the globally sensitive area of a variable camera provided in the above embodiment. The functional modules of the device include:

[0066] A data communication module 21, configured to obtain the image data of a reference image and a detection image, obtain the position data of the sensitive area in the reference image, and send the position data of the sensitive area in the detection image.

[0067] A data processing module 22, configured to determine the feature point sets, the similarity matching of feature points, the accuracy verification, and the calculation of the projection matrix and the vertex positions of the sensitive area in the reference image and the detection image; and also configured to compare the similarity of the sensitive area polygons, and determine the actual area position of the sensitive area in the detection image according to the positional relationship between the current detection screen boundary and the sensitive area.

[0068] An embodiment of the present invention further provides a computer-readable storage medium, including instructions that, when running on a computer, cause the computer to execute the method for setting the globally sensitive area of a variable camera provided in the above embodiment.

[0069] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described device and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0070] In the embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of functional modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0071] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention.

[0072] In addition, in each embodiment of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0073] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0074] Based on the embodiments of the present invention, the present invention provides a method, device, and storage medium for setting the global sensitive area of a variable camera. By comparing image feature points, the projective transformation relationship between the detected image and the reference image is obtained, and then the position of the sensitive area on the detected screen is determined. There is no need to obtain the shooting direction parameter and the lens zoom parameter from the camera device, and at the same time, it also avoids the positioning deviation caused by the motion accumulation error or the deformation of the fixed device structure, which is beneficial to improving the accuracy of the sensitive area of the camera monitoring, and improving the use efficiency and security of the monitoring system.

Claims

1. A method for setting a global sensitive area of a variable camera, characterized in that Including: 1) Obtain an image containing the sensitive area to be set as the reference image, calibrate the positions of each vertex of the polygon formed by the sensitive area on the reference image, record the coordinates of each vertex in the reference image, and obtain a set of vertex coordinates; 2) Detect the feature points of the image based on the reference image to obtain the set of feature point coordinates and eigenvalues of the reference image; 3) Detect whether the image of the reference image has moved. If the image has not moved, loop through this step. If the image has moved, extract a detection image, and based on this detection image, perform image feature point detection using the same method as the reference image feature point detection to obtain the set of feature point coordinates and eigenvalues of the detection image; 4) Perform feature matching on the elements in the eigenvalue sets of the reference image and the detection image to obtain a set of matching feature point pairs. Perform spatial geometric relationship verification on the elements in the set of matching feature point pairs, and remove the mis-matched feature point pair elements from the set of matching feature point pairs; 5) Calculate the projection transformation matrix between the detection image and the reference image using the set of matching feature point pairs, and then calculate the set of coordinate positions of the elements in the set of sensitive area vertex coordinates in the plane where the detection screen is located; 6) Determine whether the polygons formed by the elements in the set of vertex coordinates corresponding to the sensitive area in the detection image and the reference image and the set of coordinate positions are similar polygons; 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, remove the part of the sensitive area outside the image area, and determine the positions of each vertex of the sensitive area within the range of the detection screen; In step 2), the types of detection image feature points include one or more of SIFT feature points, SURT feature points, ORB feature points, BRISK feature points, and AKAZE feature points; For the ORB feature points of the detection image, calculate the feature points in the reference image P, find the corner points in the reference image P. The finding method is to obtain an image pyramid for the reference image P by using the downsampling method; for the pixel point p in each layer of the image pyramid, according to the brightness Ip of the pixel point p; set an interval threshold T = Ip * 20%; in the same-level image, for the 16 pixel points on the circumference with a radius of 3 pixels centered on the pixel point p, determine whether there are consecutive N points whose brightness is outside the interval [Ip - T, Ip + T], where N = 12, 9, or 11. If so, determine that the pixel point p is a feature point, and the direction of the feature point is determined by the gray centroid method. For the small patch image area centered on the pixel point p geometrically, calculate the average brightness in the horizontal and vertical directions, and represent the feature direction of the feature point by the arctangent function of the vertical direction brightness average value and the horizontal direction brightness average value.

2. The method for setting the global sensitive area of a variable camera according to claim 1, wherein In step 3), according to a fixed inspection time interval t, decode a single-frame picture from the video image captured by the monitoring camera as the detection image, and continuously detect whether the camera image has moved.

3. The method for setting the global sensitive area of a variable camera according to claim 1, characterized in that, In step 4), the feature matching methods include BF brute-force matching and FLANN matching.

4. The method for setting the global sensitive area of a variable camera according to claim 1, wherein In step 4), the spatial geometric relationship verification methods include RANSAC, PROSAC, and LMedS.

5. The method for setting the global sensitive area of a variable camera according to claim 1, wherein, In step 6), the methods for judging polygon similarity include Hu moment comparison, Fréchet distance comparison, and comparison of angular differences.

6. The method for setting the global sensitive area of a variable camera according to claim 1, wherein In step 1), multiple sensitive regions are set in the same reference image, and these sensitive regions do not cross, partially cross, or contain each other.

7. The method for setting the global sensitive area of a variable camera according to claim 1, wherein In step 1), multiple reference images are obtained, and multiple sensitive regions are set in the multiple reference images. These reference images do not cross, partially cross, or contain each other, and these sensitive regions do not cross, partially cross, or contain each other.

8. The method for setting the global sensitive area of a variable camera according to claim 1, characterized in that Image preprocessing is performed on the detected image, and the processing methods include one or more of image scaling, image sharpening or blurring, brightness equalization, and image defogging and denoising.

9. An apparatus for setting a global sensitive area of a variable camera, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, it implements the method for setting the variable camera global sensitive region as described in any one of claims 1 to 8.

10. An apparatus for setting a global sensitive area of a variable camera, characterized in that, A method for implementing the variable camera global sensitive region setting as described in any one of claims 1 to 8 includes: A data communication module for obtaining the image data of the reference image and the detected image, obtaining the position data of the sensitive region in the reference image, and for sending the position data of the sensitive region in the detected image; A data processing module for determining the similarity matching, accuracy verification of the feature point sets and feature points in the reference image and the detected image, and calculating the projection matrix and the vertex positions of the sensitive region; and also for comparing the similarity of the sensitive region polygons, and determining the actual region position of the sensitive region in the detected image according to the position relationship between the current detected image boundary and the sensitive region.

11. A computer-readable storage medium, comprising instructions, characterized in that, When the instruction runs on the computer, it causes the computer to execute the method for setting the variable camera global sensitive region as described in any one of claims 1 to 8.

Images