Method and device for setting global sensitive area of variable camera and storage medium

Abstract

The invention relates to the technical field of video monitoring, and discloses a variable camera global sensitive area setting method and device and a storage medium, and the method comprises the steps: judging whether polygons formed by sensitive area vertex coordinate set elements in a detection image and a reference image are similar polygons or not; according to the relation between each vertex of the sensitive area and the boundary of the detection image, the sensitive area part outside the image area is removed, and the position of each vertex of the sensitive area in the detection image range is determined, so that the sensitive area is determined. The projection transformation relation between the detection image and the reference image is obtained through comparison of the image feature points, the position of the sensitive area on the detection image is determined, and shooting direction parameters and lens zooming parameters do not need to be obtained from a camera device. Meanwhile, positioning deviation caused by motion accumulated errors or structural deformation of the fixing device is avoided, and the accuracy of the sensitive area monitored by the camera and the use efficiency of the monitoring system are improved.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a method, a device and a storage medium for setting a global sensitive area of ​​a variable camera. Background Art

[0002] As video surveillance technology is increasingly used in production, life and social management activities, many applications require setting up some special areas in the camera's field of view. These special areas either need to be monitored intensively, or not allowed to be monitored, or require special image processing, etc.

[0003] The cameras used in current video surveillance technology mainly include gun cameras, small hemisphere cameras, large hemisphere cameras, all-in-one cameras, ball cameras and other different structures and functional types. Some of them are fixed cameras whose direction and focal length will not change, and some are cameras with variable direction or focal length. When the fixed camera is equipped with a pan / tilt device, the monitoring direction can also be changed.

[0004] For variable cameras whose direction and focal length can be changed individually or simultaneously, the special areas set for the monitoring screen need to have global universal characteristics. The position of the special area needs to be adjusted accordingly as the monitoring direction and camera focal length change to ensure that the relevant area can be accurately covered. For cameras that can accurately determine the current pointing direction and lens focal length, under the condition that the initial position and initial focal length of the camera are fixed, the position of the sensitive area in the screen after the direction and focal length change can be calculated through the mathematical relationship between the deflection angle and the zoom ratio. If the initial position changes due to long-term camera rotation and lens telescoping, or the mounting fixed structure is deformed due to external factors, it will cause deviations in regional positioning. Summary of the invention

[0005] In view of the above defects in the prior art, the present invention proposes a method, device and storage medium for setting a variable camera global sensitive area.

[0006] The technical solution of the present invention is achieved in this way: A method for setting a variable camera global sensitive area, comprising: 1) Obtain an image containing the sensitive area to be set as a reference image P, calibrate the positions of the vertices of the polygon formed by the sensitive area on the reference image P, record the coordinates of each vertex in the reference image P, and obtain a vertex coordinate set {S}; 2) Detect image feature points based on the reference image P, and obtain the feature point coordinates and feature value set {F} of the reference image P; 3) Detect whether the reference image P moves. If not, execute this step repeatedly. If not, extract a detection image Pj, and perform image feature point detection based on the detection image Pj using the same method as the reference image P feature point detection to obtain the feature point coordinates and feature value set {Fj} of the detection image Pj. 4) Perform feature matching on the elements in the sets {F} and {Fj} to obtain a matching feature point pair set {MP}, perform spatial geometric relationship verification on the elements in the matching feature point pair set {MP}, and remove the incorrectly matched feature point pair elements from the matching feature point pair set {MP}; 5) The projection transformation matrix between the detection image Pj and the reference image P is calculated using the matching feature point pair set {MP}, and then the coordinate position set {Sj} of the elements in the sensitive area vertex coordinate set {S} in the plane where the detection screen is located is calculated; 6) Determine whether the polygons formed by the set elements of {Sj} and {S} are similar polygons; 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, the part of the sensitive area outside the image screen area is removed, and the position of each vertex of the sensitive area within the detection screen range is determined.

[0007] Preferably, in step 3), according to a fixed inspection time interval t, a single frame of picture is decoded from the video image captured by the monitoring camera as a detection image, and whether the camera image moves is continuously detected.

[0008] Preferably, in step 2), the types of detected image feature points include one or more of SIFT feature points, SURT feature points, ORB feature points, BRISK feature points and AKAZE feature points.

[0009] Preferably, in step 4), the feature matching method includes BF brute force matching and FLANN matching.

[0010] Preferably, in step 4), the spatial geometric relationship verification method includes RANSAC, PROSAC and LMedS.

[0011] Preferably, in step 6), the method for determining the similarity of polygons includes comparison of Hu moments, comparison of Fréchet distances and comparison of angle differences.

[0012] Preferably, in step 1), multiple sensitive areas are set in the same reference image, and these sensitive areas may not overlap, partially overlap, or include each other.

[0013] Preferably, in step 1), multiple reference images are acquired, and multiple sensitive areas are set in the multiple reference images. The reference images may not intersect, partially intersect, or contain each other, and the sensitive areas may not intersect, partially intersect, or contain each other.

[0014] Preferably, the detection image is subjected to image preprocessing, and the processing method includes one or more of image scaling, image sharpening or blunting processing, brightness equalization, and image defogging and denoising processing.

[0015] The present invention also discloses a device for setting the global sensitive area of ​​a variable camera. The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for setting the global sensitive area of ​​a variable camera is implemented.

[0016] The present invention also discloses another device for setting a variable global sensitive area of ​​a camera, comprising: A data communication module, used to obtain image data of a reference image and a detection image, obtain position data of a sensitive area in the reference image, and to send position data of a sensitive area in the detection image; The data processing module is used to determine the feature point set in the reference image and the detection image, the similarity matching of the feature points, the accuracy verification, and the calculation of the projection matrix and the vertex positions of the sensitive area; it is also used to compare the similarity of the polygons of the sensitive area, and determine the actual area position of the sensitive area in the detection image based on the positional relationship between the boundary of the current detection image and the sensitive area.

[0017] The invention discloses a computer-readable storage medium, comprising instructions. When the instructions are run on a computer, the computer is enabled to execute the method for setting the global sensitive area of ​​a variable camera.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention obtains the projection transformation relationship between the detection image and the reference image by comparing image feature points, and then determines the position of the sensitive area on the detection screen. There is no need to obtain shooting direction parameters and lens magnification parameters from the camera device. At the same time, it also avoids positioning deviations caused by motion accumulation errors or structural deformation of the fixing device, which is beneficial to improving the accuracy of sensitive areas monitored by the camera and improving the use efficiency and safety of the monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of a method for setting a global sensitive area of ​​a variable camera according to the present invention; Figure 2 Functional block diagram of a device for setting the global sensitive area of ​​a variable camera of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] The method for setting the global sensitive area of ​​a variable camera provided in an embodiment of the present invention determines the position of the sensitive area on the detection screen by obtaining the projection transformation relationship between the detection image and the reference image. There is no need to obtain shooting direction parameters and lens magnification parameters from the camera device, and it will not be affected by motion accumulation errors or structural deformation of the fixed device, thereby achieving the setting of sensitive areas of the monitoring screen on a global scale.

[0022] like Figure 1 As shown, the flow chart of the method for setting the global sensitive area of ​​a variable camera of the present invention comprises: 1) Obtain an image containing the sensitive area to be set as a reference image P, calibrate the positions of the vertices of the polygon formed by the sensitive area on the reference image P, record the coordinates of each vertex in the reference image P, and obtain a vertex coordinate set {S}; 2) Detect image feature points based on the reference image P, and obtain the feature point coordinates and feature value set {F} of the reference image P; 3) Detect whether the reference image P moves. If not, execute this step repeatedly. If not, extract a detection image Pj, and perform image feature point detection based on the detection image Pj using the same method as the reference image P feature point detection to obtain the feature point coordinates and feature value set {Fj} of the detection image Pj. 4) Perform feature matching on the elements in the sets {F} and {Fj} to obtain a matching feature point pair set {MP}, perform spatial geometric relationship verification on the elements in the matching feature point pair set {MP}, and remove the incorrectly matched feature point pair elements from the matching feature point pair set {MP}; 5) The projection transformation matrix between the detection image Pj and the reference image P is calculated using the matching feature point pair set {MP}, and then the coordinate position set {Sj} of the elements in the sensitive area vertex coordinate set {S} in the plane where the detection screen is located is calculated; 6) Determine whether the polygons formed by the set elements of {Sj} and {S} are similar polygons; 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, the part of the sensitive area outside the image screen area is removed, and the position of each vertex of the sensitive area within the detection screen range is determined.

[0023] In an embodiment of the present invention, the method specifically includes: S1, obtaining an image containing the sensitive area to be set as a reference image P, and calibrating the positions of the vertices of the polygon formed by the sensitive area on the reference image P, recording the coordinates of each vertex in the reference image P, and obtaining a vertex coordinate set {S}; S2, detecting image feature points based on the reference image P, where the types of the detected image feature points include but are not limited to SIFT feature points, SURT feature points, ORB feature points, BRISK feature points, and AKAZE feature points, etc., to obtain feature point coordinates and feature value set {F} of the reference image P; Specifically, to detect the ORB feature points of an image, it is necessary to calculate the feature points in the reference image P. Calculating the feature points is to find the corner points in the reference image P. The method is to use the downsampling method for the reference image P to obtain an image pyramid; for the pixel point p in each level of the image pyramid, according to the brightness Ip of the point p; set an interval threshold T (exemplarily, T=Ip*20%); in the same level image, for the 16 pixel points on the circle with a radius of 3 pixels and centered on the pixel p, determine whether there are N consecutive points whose brightness is outside the interval [Ip-T, Ip+T], exemplarily, N=12, 9 or 11, etc. If so, point p is determined to be a feature point. The direction of the feature point is determined by the grayscale centroid method, that is, for the surrounding small image area with the pixel point p as the geometric center, the horizontal and vertical brightness mean is calculated, and the feature direction of the feature point p is represented by the inverse tangent function of the vertical brightness mean and the horizontal brightness mean.

[0024] Exemplarily, to detect the ORB feature points of an image, it is necessary to calculate the eigenvalue BRIEF descriptor of the feature point in the reference image P, and the method is: use a Gaussian filter with a variance of σ and a size of N x N to perform Gaussian smoothing on an area Z with a size of S*S centered on the feature point, and exemplary, S=31, N=9, σ=2; randomly select d pairs of pixel points in the area Z according to the Gaussian distribution law, compare the brightness values ​​of the two points in each point pair in sequence, and assign 1 or 0 to the corresponding bits of the eigenvalue in sequence, thereby obtaining a d-bit eigenvalue, and exemplary, d=128, 256 or 512, etc.

[0025] S3, at a certain interval t, detect whether the picture moves. If the picture does not move, continue to execute this step S2. If the picture moves, extract a detection image Pj, and based on this detection image Pj, use the same method as the reference image feature point detection to perform image feature point detection, and obtain the feature point coordinates and feature value set {Fj} of the detection image Pj; S4, feature matching is performed on the elements in the sets {F} and {Fj} to obtain a set of matching feature point pairs {MP}. Feature matching methods include but are not limited to BF brute force matching and FLANN matching. Spatial geometric relationship verification is performed on the elements in the set of matching feature point pairs {MP}, and incorrectly matched feature point pair elements are removed from the set {MP}. Spatial geometric relationship verification methods include but are not limited to RANSAC, PROSAC, LMedS and other methods; Specifically, the BF brute force matching method is used to find the eigenvalue with the smallest distance from the feature point coordinates and eigenvalue set {F} of the reference image P to the feature point coordinates and eigenvalue set {Fj} of the detection image Pj as the matching point.

[0026] Exemplarily, a cross-matching method may be used to optimize matching accuracy, that is, when the eigenvalues ​​in the reference image and the eigenvalues ​​in the detection image are optimally matched, they are confirmed as matching point pairs, thereby avoiding matching errors of one point matching multiple points.

[0027] Exemplarily, a comparison threshold may be set to optimize the matching accuracy. Exemplarily, the threshold is set to [0.5, 0.9]. When the ratio of the distance between the best match and the second-best match meets the threshold condition, the point pair is confirmed as a matching point.

[0028] S5, using the matching feature point pair set {MP} to calculate the projection transformation matrix M between the detection image Pj and the reference image P, and then calculate the coordinate position set {Sj} of the elements in the sensitive area vertex coordinate set {S} in the plane where the detection image screen is located, where the detection image screen is located refers to the detection screen and the entire plane extending around the detection screen; S6, determine whether the polygons formed by the set elements of {Sj} and {S} are polygons with similar shapes. If the shapes are similar, continue to execute step S6, otherwise execute step S2. Methods for determining shape similarity include but are not limited to Hu moment comparison, Fréchet distance comparison, and angle difference comparison; Specifically, the above Hu moments are a set of invariant features for describing contour shapes, which are 7 invariant moments calculated based on the normalized central moment, and are features that can remain unchanged under translation, rotation and scaling.

[0029] Specifically, the above-mentioned Fréchet distance is a measure of similarity between curves, which takes into account the position and order relationship of points along the curves and is the infimum of the maximum distance between two curves.

[0030] S7, according to the relationship between each vertex of the sensitive area and the boundary of the detection screen, remove the part of the sensitive area outside the image screen area, and determine the position of each vertex of the sensitive area within the detection screen range and on the boundary of the detection screen.

[0031] In actual video surveillance systems, there are many ways to compose cameras with variable direction and / or variable focal length. There are intelligent surveillance cameras with a high level of intelligence, variable direction surveillance cameras composed of simple fixed-focus gun-type cameras and pan-tilt devices, or integrated cameras without steering devices but with zoom lenses.

[0032] The method for setting the global sensitive area of ​​a variable camera involved in an embodiment of the present invention does not need to rely on the camera rotation and focal length control system to provide direction and focal length parameters under actual video surveillance system application conditions, but completely relies on image correlation analysis and multiple verifications to determine the position of the sensitive area. It has the characteristics of simple application configuration, high reliability in long-term use, and stable and accurate positioning.

[0033] Optionally, in some embodiments, multiple sensitive areas may be set in the same reference image, and these sensitive areas may not intersect, partially intersect, or contain each other.

[0034] Optionally, in some embodiments, multiple sensitive areas may be set in multiple reference images. The reference images may not intersect, partially intersect, or include each other, and the sensitive areas may not intersect, partially intersect, or include each other.

[0035] Optionally, in some embodiments, the detected image may be subjected to image preprocessing, including but not limited to image scaling, image sharpening or blunting, brightness equalization, image defogging, and denoising.

[0036] Optionally, reference images may be obtained in different time periods, such as daytime and nighttime, respectively, and settings may be made for the same sensitive area.

[0037] Optionally, reference images may be obtained in different seasons, such as summer and winter, respectively, and settings may be performed on the same sensitive area.

[0038] Exemplarily, in step S2, t=1 second may be selected, that is, a current image is obtained at a fixed time interval of once per second to perform relevant detection.

[0039] like Figure 2 As shown, the embodiment of the present invention also provides a device 20 for setting the global sensitive area of ​​a variable camera, the device is composed of a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to implement the method for setting the global sensitive area of ​​a variable camera provided in the above embodiment. The functional modules of the device include: A data communication module 21, used to obtain image data of a reference image and a detection image, obtain position data of a sensitive area in the reference image, and to send position data of a sensitive area in the detection image; The data processing module 22 is used to determine the feature point set in the reference image and the detection image, the similarity matching of the feature points, the accuracy verification, and the calculation of the projection matrix and the vertex positions of the sensitive area; it is also used to compare the similarity of the polygons of the sensitive area, and determine the actual area position of the sensitive area in the detection picture based on the positional relationship between the boundary of the current detection picture and the sensitive area.

[0040] An embodiment of the present invention further provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method for setting a variable global sensitive area of ​​a camera as provided in the above embodiment.

[0041] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0042] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0043] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0044] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0045] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0046] Based on the embodiments of the present invention, the present invention proposes a method, device and storage medium for setting the global sensitive area of ​​a variable camera. The projection transformation relationship between the detection image and the reference image is obtained by comparing the image feature points, and then the position of the sensitive area on the detection screen is determined. There is no need to obtain the shooting direction parameters and the lens magnification parameters from the camera device. At the same time, it also avoids the positioning deviation caused by the accumulated motion error or the deformation of the fixed device structure, which is beneficial to improving the accuracy of the sensitive area monitored by the camera and improving the use efficiency and safety of the monitoring system.

Claims

1. A method for setting a variable camera global sensitive area, characterized in that: include: 1) Obtain an image containing the sensitive area to be set as a reference image, calibrate the positions of each vertex of the polygon formed by the sensitive area on the reference image, record the coordinates of each vertex in the reference image, and obtain a vertex coordinate set; 2) Detect image feature points based on the reference image to obtain the feature point coordinates and feature value set of the reference image; 3) Detect whether the reference image moves. If the image does not move, execute this step repeatedly. If the image moves, extract a detection image, and perform image feature point detection based on the detection image using the same method as the reference image feature point detection to obtain the feature point coordinates and feature value set of the detection image; 4) Perform feature matching on the elements in the feature value sets of the reference image and the detection image to obtain a set of matching feature point pairs, perform spatial geometric relationship verification on the elements in the set of matching feature point pairs, and remove the feature point pair elements that are incorrectly matched from the set of matching feature point pairs; 5) The projection transformation matrix between the detection image and the reference image is calculated using the set of matching feature point pairs, and then the coordinate position set of the elements in the vertex coordinate set of the sensitive area in the plane where the detection image is located is calculated; 6) Determine whether the polygons formed by the vertex coordinate sets and the elements in the coordinate position sets corresponding to the sensitive areas in the detection image and the reference image are similar polygons; 7) According to the relationship between each vertex of the sensitive area and the boundary of the detection screen, the part of the sensitive area outside the image screen area is removed, and the position of each vertex of the sensitive area within the detection screen range is determined.

2. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 3), according to a fixed inspection time interval t, a single frame image is decoded from the video image captured by the monitoring camera as a detection image, and whether the camera image moves is continuously detected.

3. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 2), the types of detected image feature points include one or more of SIFT feature points, SURT feature points, ORB feature points, BRISK feature points and AKAZE feature points.

4. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 4), the feature matching method includes BF brute force matching and FLANN matching.

5. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 4), the spatial geometric relationship verification methods include RANSAC, PROSAC and LMedS.

6. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 6), the methods for determining polygon similarity include Hu moment comparison, Fréchet distance comparison and angle difference comparison.

7. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 1), multiple sensitive areas are set in the same reference image. These sensitive areas may not overlap, partially overlap, or include each other.

8. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: In step 1), multiple reference images are acquired, and multiple sensitive areas are set in the multiple reference images. The reference images may not intersect, partially intersect, or include each other, and the sensitive areas may not intersect, partially intersect, or include each other.

9. The method for setting a variable camera global sensitive area according to claim 1, characterized in that: The detected image is preprocessed, and the processing method includes one or more of image scaling, image sharpening or blunting, brightness balancing, and image defogging and denoising.

10. A device for setting a variable global sensitive area of ​​a camera, characterized in that: The device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the method for setting a variable global sensitive area of ​​a camera as claimed in any one of claims 1 to 9 is implemented.

11. A device for setting a variable global sensitive area of ​​a camera, characterized in that: include: A data communication module, used to obtain image data of a reference image and a detection image, obtain position data of a sensitive area in the reference image, and to send position data of a sensitive area in the detection image; The data processing module is used to determine the feature point set in the reference image and the detection image, the similarity matching of the feature points, the accuracy verification, and the calculation of the projection matrix and the vertex positions of the sensitive area; it is also used to compare the similarity of the polygons of the sensitive area, and determine the actual area position of the sensitive area in the detection image based on the positional relationship between the boundary of the current detection image and the sensitive area.

12. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a computer, the computer is enabled to execute the method for setting a variable global sensitive area of ​​a camera according to any one of claims 1 to 9.

Images