A method and device for determining the position of a camera, a security system, and a storage medium

Abstract

The present invention discloses a method, device, security system and storage medium for determining the position of a camera. The method includes: acquiring a first picture and a second picture captured by the camera at different time periods; detecting key points in the first picture and the second picture, and performing matching of similar key points to obtain a sample set composed of similar key point pairs; based on the sample set, taking the first picture as a reference system, determining an optimal single mapping transformation matrix of the key points in the second picture; according to the original coordinates of the corner points and the center point in the second picture and the optimal single mapping transformation matrix, determining the new coordinates of the corner points and the center point in the second picture, determining the new width and new height of the second picture, and further determining whether the camera has moved to determine whether the position of the camera has changed. With this solution, by judging whether the camera itself has moved, false alarms in the area monitored by the camera can be reduced or even avoided due to the movement of the camera itself.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a method and device for determining the position of a camera, a security system, and a storage medium, and more particularly to a method and device for determining the movement of a camera based on image key points, a security system, and a storage medium. Background Art

[0002] In many application requirements in the field of intelligent security, it is often necessary to detect whether the monitored objects in the picture have changed. For example, in the following application scenarios: during coal mining, gas and carbon monoxide sensors need to be installed at the return air corner of the fully mechanized coal mining face, and it is required that the positions of the sensors cannot be moved randomly.

[0003] For such application scenarios, if the ground vibrates during coal mining, causing the camera to shake, the security system will detect a change in the position of the sensor and issue an alarm. After receiving the alarm information, the monitoring and management personnel will process these alarm information, such as correcting the position of the sensor, but all these processes are in vain. Because, in this process, in fact, the sensor has not moved, but only because the camera itself shakes, which causes the position of the sensor captured by the camera to move relatively. The security system thus detects a change in the position of the sensor and issues a false alarm message, and then the monitoring and management personnel will waste a lot of time and energy in vain to process these false alarm messages, which is time-consuming and laborious.

[0004] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and device for determining the position of a camera, a security system, and a storage medium, so as to solve the problem that when the camera itself moves, it will cause the photographed object in the area monitored by the camera to move relatively, resulting in false alarms in the area monitored by the camera when it is impossible to determine whether the camera itself has moved, and achieve the effect of reducing or even avoiding false alarms in the area monitored by the camera due to the movement of the camera itself by judging whether the camera itself has moved.

[0006] The present invention provides a method for determining the position of a camera, including: obtaining two pictures captured by the camera for the same shooting object at different times, denoted as the first picture and the second picture; detecting key points in the first picture and the second picture, and performing matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture; based on the sample set, taking the first picture as a reference system, determining an optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture; according to the original coordinates of the corner points and the center point in the second picture, and the optimal single mapping transformation matrix, determining the new coordinates of the corner points and the center point in the second picture; according to the new coordinates of the corner points and the center point in the second picture, determining the new width and new height of the second picture; according to at least one of the new coordinates of the corner points and the center point in the second picture, and the new width and new height of the second picture, determining whether the camera has moved to determine whether the position of the camera has changed.

[0007] Matched with the above method, on the other hand, the present invention provides a device for determining the position of a camera, including: an obtaining unit configured to obtain two pictures captured by the camera for the same shooting object at different times, denoted as the first picture and the second picture; a detecting unit configured to detect key points in the first picture and the second picture, and perform matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture; a determining unit configured to, based on the sample set, taking the first picture as a reference system, determine an optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture; the determining unit is further configured to, according to the original coordinates of the corner points and the center point in the second picture, and the optimal single mapping transformation matrix, determine the new coordinates of the corner points and the center point in the second picture; the determining unit is further configured to, according to the new coordinates of the corner points and the center point in the second picture, determine the new width and new height of the second picture; the determining unit is further configured to, according to at least one of the new coordinates of the corner points and the center point in the second picture, and the new width and new height of the second picture, determine whether the camera has moved to determine whether the position of the camera has changed.

[0008] Matched with the above device, on the other hand, the present invention provides a security system, including: the above-mentioned device for determining the position of a camera.

[0009] Matched with the above method, on the other hand, the present invention provides a storage medium, where the storage medium includes a stored program, and when the program runs, it controls the device where the storage medium is located to execute the above-mentioned method for determining the position of a camera.

[0010] Therefore, the solution of the present invention can accurately determine whether the camera has moved by using two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement. Thus, by determining whether the camera itself has moved, false alarms in the area monitored by the camera can be reduced or even avoided due to the movement of the camera itself.

[0011] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention.

[0012] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic flowchart of an embodiment of the method for determining the position of the camera of the present invention;

[0014] Figure 2 It is a schematic flowchart of an embodiment of detecting key points in the first picture and the second picture and matching similar key points in the method of the present invention;

[0015] Figure 3 It is a schematic flowchart of an embodiment of determining the optimal single mapping transformation matrix for mapping key points in the second picture to the first picture in the method of the present invention;

[0016] Figure 4 It is a schematic flowchart of an embodiment of determining the effective set in the sample set in the method of the present invention;

[0017] Figure 5 It is a schematic flowchart of an embodiment of determining the optimal single mapping transformation matrix in the method of the present invention;

[0018] Figure 6 It is a schematic structural diagram of an embodiment of the device for determining the position of the camera of the present invention;

[0019] Figure 7 It is a schematic diagram of Picture A and Picture B taken by the camera in an embodiment of the method for determining camera movement based on image key points of the present invention;

[0020] Figure 8 It is a schematic flowchart of the method for determining camera movement based on image key points of the present invention;

[0021] Figure 9 It is a schematic diagram of the camera translating in the up, down, left, and right directions in an embodiment of the method for determining camera movement based on image key points of the present invention;

[0022] Figure 10 Schematic diagram of the front - rear translation of a camera in an embodiment of a camera movement determination method based on image key points according to the present invention;

[0023] Figure 11 Schematic diagram of the rotation of a camera in an embodiment of a camera movement determination method based on image key points according to the present invention.

[0024] Combined with the accompanying drawings, the reference numerals in the embodiments of the present invention are as follows:

[0025] 102 - acquisition unit; 104 - detection unit; 106 - determination unit. Detailed implementation manners

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0027] According to an embodiment of the present invention, a method for determining the position of a camera is provided, as Figure 1 shown in the flowchart of an embodiment of the method of the present invention. The method for determining the position of the camera may include: step S110 to step S160.

[0028] At step S110, two pictures taken by the camera for the same shooting object at different times are acquired, denoted as the first picture and the second picture.

[0029] At step S120, key points in the first picture and the second picture are detected, and similar key points are matched to obtain a sample set composed of pairs of similar key points in the first picture and the second picture.

[0030] In some implementation manners, for the specific process of detecting key points in the first picture and the second picture and matching similar key points at step S120 to obtain a sample set composed of pairs of similar key points in the first picture and the second picture, refer to the following exemplary description.

[0031] The following combines Figure 2 shown in the flowchart of an embodiment of detecting key points in the first picture and the second picture and matching similar key points in the method of the present invention to further illustrate the specific process of detecting key points in the first picture and the second picture and matching similar key points at step S120, including: step S210 to step S220.

[0032] In step S210, key point detection technology is adopted to detect the key points of the first picture, denoted as the first key point group; and the key points of the second picture are detected, denoted as the second key point group.

[0033] In step S220, key point matching technology is sampled to match the similar key points among the key points in the first key point group and the second key point group, obtaining l groups of key point pairs, where l is a positive integer. The l groups of key point pairs form a sample set.

[0034] Considering that in the actual application scenario, the camera may translate or rotate in any direction. Figure 7 This is a schematic diagram of pictures A and B taken by a camera in an embodiment of a method for determining camera movement based on image key points according to the present invention. As Figure 8 shown, the camera takes the first picture (such as picture A) at the first moment (such as time t0), and takes the second picture (such as picture B) after a set time period (such as time Δt). The width of picture A is w A and the height is h A , and the original width of picture B is w B and the original height is h B . Among them, the size markings of picture A and picture B are unified.

[0035] In the solution of the present invention, the key points in pictures A and B are detected and similar key points are matched. Specifically, key point detection and key point matching technology can be adopted to detect the key points in pictures A and B and match the similar key points. Similar key points refer to: if the distance between the feature vectors of two or more key points in a picture is less than a certain threshold, these key points are similar key points to each other. In actual application in a picture, similar key points are generally the same pixel point.

[0036] Among them, the key point detection and key point matching technology can adopt the key point detection and key point matching technology described in documents such as David G. Lowe. Object Recognition from Local Scale-Invariant Features. 1999 and David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. 2004.

[0037] In the solution of the present invention, it is assumed that after detecting the key points in pictures A and B, it can be obtained that there are m key points Ri in picture A i , i = 1, 2,..., m, and there are n key points Sj in picture B j, where \(j = 1, 2, \ldots, n\), and both \(m\) and \(n\) are positive integers. After performing similar key-point matching on the key points in Picture A and Picture B, a total of \(l\) groups of key-point pairs are matched between Picture A and Picture B. These matched key-point pairs form a sample set. The pixel coordinates of the key points in Picture A and Picture B are respectively

[0038] Among them, is the pixel coordinate of the key point in Picture A, is the pixel coordinate of the key point in Picture B. \(1\leq i\) k \(\leq m\), \(1\leq j\) k \(\leq n\), \(k = 1, 2, \ldots, l\), where \(l\) is the number of groups of key points matched between Picture A and Picture B, and \(l\) is a positive integer.

[0039] Based on the data information obtained by detecting the key points in Picture A and Picture B and performing similar key-point matching, such as the pixel coordinates of the key points in Picture A and Picture B are respectively The solution of the present invention provides a method for determining the movement of a camera based on image key points.

[0040] In step S130, based on the sample set and with the first picture as the reference system, determine the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture.

[0041] In some embodiments, for the specific process of determining the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture in step S130 based on the sample set and with the first picture as the reference system, refer to the following exemplary description.

[0042] Next, in conjunction with Figure 3 the schematic flowchart of an embodiment for determining the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture in the method of the present invention as shown, further illustrate the specific process of determining the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture in step S130, including: steps S310 to S340.

[0043] Step S310: Select a set of key-point pairs that have not been selected from the sample set and calculate the single mapping transformation matrix.

[0044] Figure 8 is the schematic flowchart of a method for determining the movement of a camera based on image key points of the present invention. As Figure 8 shown, the specific steps of a method for determining the movement of a camera based on image key points include:

[0045] Step 1: Taking Picture A as the reference system, calculate the optimal single mapping transformation matrix of the key points in Picture B to Picture A. The specific calculation method is as follows:

[0046] Step 1.1: Calculate the single mapping transformation matrix H 3×3 :

[0047]

[0048] For the convenience of processing and calculation, the elements h 3×3 in the single mapping transformation matrix H 3×3 are normalized here: Let h 3×3 = 1.

[0049] The single mapping transformation matrix H 3×3 contains eight unknown elements. Therefore, randomly select 4 groups of key point pairs that have not been selected from the sample set composed of l groups of key point pairs matched between Picture A and Picture B Suppose these 4 groups of key point pairs are The pixel coordinates of the key points in Picture A and Picture B are respectively where k = k1, k2, k3, k4, which represents four random numbers among k = 1, 2,..., l, and l is the number of groups of key points matched between Picture A and Picture B.

[0050] The following equations can be established for each group of key point pairs:

[0051]

[0052] where k takes k1, k2, k3, k4 respectively. A total of eight equations can be obtained from the 4 groups of key point pairs. By combining the eight equations, the eight unknown elements of the single mapping transformation matrix H 3×3 can be obtained.

[0053] Step S320: Determine the effective set in the sample set according to the single mapping transformation matrix.

[0054] As Figure 8 shown, the specific steps of a method for determining the movement of a camera based on key points of an image further include: Step 1.2: Determine the effective set according to the single mapping transformation matrix.

[0055] In some embodiments, for the specific process of determining the effective set in the sample set according to the single mapping transformation matrix in Step S320, refer to the following exemplary description.

[0056] The following combines with Figure 4Schematic flowchart of an embodiment for determining the valid set in the method of the present invention, further illustrating the specific process of determining the valid set in step S320, including: steps S410 to S430.

[0057] Step S410: According to the single mapping transformation matrix, calculate the back-projection error rate of each key point pair in the remaining key point pairs in the sample set one by one. The remaining key point pairs are the key point pairs that have not been selected in the sample set.

[0058] Step S420: If the back-projection error rate of any key point pair in the remaining key point pairs is less than the set error rate, it is determined that the single mapping transformation matrix is valid for this key point pair, and this key point pair is recorded as a valid key point pair.

[0059] Step S430: Determine the set composed of the set of the set key point pairs and all valid key point pairs in the remaining key point pairs as the valid set.

[0060] As Figure 8 shown, the specific steps of a camera movement determination method based on image key points further include:

[0061] In step 1.2, the specific method for determining the valid set according to the single mapping transformation matrix is as follows:

[0062] Judge whether the single mapping transformation matrix H 3×3 calculated in step 1.1 is valid for the remaining key point pairs in the sample set. Here, the remaining key points refer to the key point pairs that have not been selected in step 1.1 for calculating the single mapping transformation matrix H 3×3 of the key point pairs.

[0063] The single mapping transformation matrix H 3×3 is calculated from 4 groups of key point pairs randomly selected from the sample set. Therefore, the single mapping transformation matrix H 3×3 is necessarily valid for these 4 groups of key points. Whether the single mapping transformation matrix H 3×3 is valid for the remaining key points in the sample set needs to be further verified. The specific verification method is as follows:

[0064] Substitute the single mapping transformation matrix H 3×3 into the following formula, and calculate and judge the back-projection error rate E k of each key point pair in the remaining key point pairs in the sample set one by one:

[0065]

[0066] where E k represents the back-projection error rate, k = 1, 2,..., l, and l is the number of groups of key points that match between picture A and picture B.

[0067] If the back-projection error rate E of a certain key-point pair k <10, it is considered that the single mapping transformation matrix is effective for this key-point pair. After calculating and judging each key-point pair in the sample set one by one, count all the key-point pairs in the sample set that satisfy the back-projection error rate E k <10. The set composed of these key-point pairs is the effective set, and each key-point pair is an element of this effective set.

[0068] Step S330, determine the current cardinality of the effective set. If the current cardinality of the effective set is greater than the historical cardinality, determine that the effective set is the current optimal effective set. Among them, the current cardinality of the effective set is the number of elements in the effective set determined in this loop. The number of elements in a set is called the cardinality of the set. The current cardinality of the effective set refers to the cardinality of the effective set obtained in this loop, that is, the number of elements in the effective set obtained in this loop.

[0069] As Figure 8 shown, the specific steps of a camera movement determination method based on image key points further include: Step 1.3, determine the current optimal effective set according to the cardinality of the effective set. The specific method is:

[0070] Judge whether the effective set obtained in this loop is the optimal current effective set according to the cardinality of the effective set. If so, update the current optimal effective set. If not, keep the current optimal effective set. The number of elements in a set is called the cardinality of the set. The cardinality of the effective set refers to the number of elements in the effective set.

[0071] The criterion for judging whether the effective set obtained in this loop is the optimal effective set is: the cardinality of the effective set obtained in this loop is greater than the cardinality of the effective sets obtained in each loop operation in history. If the cardinality of the effective set obtained in this loop is greater than the cardinality of the effective sets obtained in each loop operation in history, it means that the effective set obtained in this loop is the set that is effective for the most key-point pairs currently, so it is the current optimal effective set.

[0072] Step S340, determine the optimal single mapping transformation matrix according to the current optimal effective set.

[0073] As Figure 8 shown, the specific steps of a camera movement determination method based on image key points further include: Step 1.4, determine the optimal single mapping transformation matrix

[0074] In some embodiments, for the specific process of determining the optimal single mapping transformation matrix according to the current optimal effective set in step S340, refer to the following exemplary description.

[0075] The following combines Figure 5 The flowchart of an embodiment for determining the optimal single mapping transformation matrix in the method of the present invention shown in the figure further illustrates the specific process of determining the optimal single mapping transformation matrix in step S340, including: step S510 to step S530.

[0076] Step S510, determine whether the number of cycles of the current optimal valid set is less than the set number of cycles, or whether the current error probability of the sample set is greater than the set error probability. The current error probability of the sample set is the ratio of the number of key point pairs in the other key point pairs with a back-projection error rate greater than or equal to the set error rate to the total number of key point pairs in the sample set.

[0077] Step S520, if it is determined that the number of cycles of the current optimal valid set is less than the set number of cycles, or the current error probability of the sample set is greater than the set error probability, then continue to cycle to determine the current optimal valid set.

[0078] Step S530, if it is determined that the number of cycles of the current optimal valid set is greater than or equal to the set number of cycles, and / or the current error probability of the sample set is less than or equal to the set error probability, then determine the current optimal valid set as the optimal single mapping transformation matrix.

[0079] As Figure 8 shown, the specific steps of a camera movement determination method based on image key points further include:

[0080] In step 1.4, determine the optimal single mapping transformation matrix The specific method is:

[0081] Judge whether the loop condition is satisfied: the number of loops is less than 2000 or the current error probability p > 0.1. If satisfied, loop and execute steps 1.1 to 1.3. If not satisfied, take the single mapping transformation matrix corresponding to the current optimal valid set as the optimal single mapping transformation matrix

[0082]

[0083] Among them, p represents the current error probability, and the calculation method is:

[0084]

[0085] In the formula, N is the number of key points in the sample set that satisfy the condition of the back-projection error rate E k ≥10, and L is the total number of key points in the sample set.

[0086] At step S140, based on the original coordinates of the corner points and the center point in the second picture, and the optimal single mapping transformation matrix, determine the new coordinates of the corner points and the center point in the second picture.

[0087] In some embodiments, in step S140, determining the new coordinates of the corner points and the center point in the second picture based on the original coordinates of the corner points and the center point in the second picture, and the optimal single mapping transformation matrix includes: after arranging the original coordinates of a set number of corner points and the center point in the second picture into a matrix with a set number of rows and columns, multiplying it by the optimal single mapping transformation matrix to obtain the new coordinates of the set number of corner points and the center point in the second picture with the first picture as the reference system.

[0088] Similarly, obtain the new coordinates of the other corner points and the center point in the second picture with the first picture as the reference system. Among them, the other corner points are the remaining corner points in all the corner points of the second picture except the set number of corner points.

[0089] As Figure 8 shown, the specific steps of a camera movement determination method based on image key points further include:

[0090] Step 2, according to the original coordinates of 4 corner points and the center point in Picture B, and the optimal single mapping transformation matrix Calculate the new coordinates of 4 corner points and the center point in Picture B under the reference system of Picture A. The specific method is as follows:

[0091] The center point coordinates of Picture A (x cA , y cA ) = (w A / 2, h A / 2), where w A and h A are the width and height of Picture A respectively.

[0092] The original coordinates of 4 corner points and the center point in Picture B are: the upper left corner point (x 11 , y 11 ) = (0, 0), the lower left corner point (x 21 , y 21 ) = (0, h B ), the upper right corner point (x 12 , y 12 ) = (w B , 0), the lower right corner point (x 22 , y 22 ) = (w B , h B ), the center point (x cB , y cB ) = (w B / 2, h B / 2).

[0093] Step 2.1. The corner points refer to the four vertices of a rectangle. The center point refers to the intersection of the two diagonals of the rectangle. After arranging the original coordinates of the 4 corner points and the center point in picture B into a matrix of 3 rows and 1 column, multiply them by the optimal single mapping transformation matrix to obtain the new coordinates of the 4 corner points and the center point in picture B in the reference system of picture A:

[0094] Upper left corner point (x' 11 , y' 11 ), lower left corner point (x' 21 , y' 21 ), upper right corner point (x' 12 , y' 12 ), lower right corner point (x' 22 , y' 22 ), center point (x' cB , y' cB ). Now, take the calculation of the new coordinates (x' 11 , y' 11 ) of the upper left corner point as an example to illustrate the specific calculation method:

[0095]

[0096]

[0097] Among them, v0 represents the relative value of the abscissa of the upper left corner point in picture B in the reference system of picture A, v1 represents the relative value of the ordinate of the upper left corner point in picture B in the reference system of picture A, and v2 represents the coordinate reference value. The calculation methods of v0, v1, and v2 are as follows:

[0098]

[0099] Here, in order to multiply with the optimal single mapping transformation matrix , arrange the original coordinates of the 4 corner points and the center point in picture B into a matrix of 3 rows and 1 column, and for the convenience of calculation, normalize the element in the 3rd row and 1st column: make the value of the element in the 3rd row and 1st column equal to 1.

[0100] It can be seen that

[0101] Similarly, the new coordinates of the other corner points and the center point in picture B in the reference system of picture A can be calculated: lower left corner point (x' 21 , y' 21 ), upper right corner point (x' 12 , y' 12 ), lower right corner point (x' 22 , y' 22), center point (x' cB , y' cB ).

[0102] At step S150, based on the new coordinates of the corner points and the center point in the second picture, determine the new width and new height of the second picture.

[0103] In some embodiments, in step S150, determining the new width and new height of the second picture according to the new coordinates of the corner points and the center point in the second picture includes: calculating the new width and new height of the second picture according to the new coordinates of the other corner points and the center point in the second picture when the first picture is used as a reference system.

[0104] As Figure 8 shown, the specific steps of a method for determining camera movement based on image key points further include:

[0105] Step 3: Calculate the new width w' B and new height h' B of picture B according to the new coordinates of the other corner points and the center point in picture B under the reference system of picture A:

[0106]

[0107]

[0108] As Figure 8 shown, the specific steps of a method for determining camera movement based on image key points further include:

[0109] Step 4: In an actual application scenario, there are several situations where the camera moves, such as: translation in the up, down, left, and right directions, translation in the front and back directions, and / or rotation. Therefore, according to the new coordinates of the 4 corner points and the center point in picture B under the reference system of picture A and the new width and height of picture B, respectively determine whether the camera has moved for these situations. For specific reference, see the following exemplary description.

[0110] At step S160, determine whether the camera has moved based on at least one of the new coordinates of the corner points and the center point in the second picture, and the new width and new height of the second picture, so as to determine whether the position of the camera has changed.

[0111] Considering that, to accurately detect whether the monitored object in the picture has moved, it is first necessary to detect whether the position of the camera has moved, which is one of the technical shortcomings in the field of intelligent security in related solutions. Therefore, based on the key point detection and key point matching technology, the solution of the present invention proposes a method for determining the movement of a camera based on image key points, which accurately calculates the translation distance or rotation angle of the camera through two photos taken by the camera at different times, so as to accurately determine whether the camera has moved, effectively avoiding a large number of false alarm messages caused by the movement of the camera.

[0112] In some embodiments, in step S160, determining whether the camera has moved according to the new coordinates of the corner points and the center point in the second picture, and at least one of the new width and new height of the second picture includes any of the following determination cases:

[0113] The first determination case: determining the translation distance of the camera according to the new coordinates of the corner points and the center point in the second picture: if the translation distance is 0, it is determined that the camera has not undergone translation in the up, down, left, or right direction. If the translation distance is not 0, it is determined that the camera has undergone translation in the up or down or left or right direction, and the distance of the translation of the camera is the translation distance.

[0114] As Figure 8 shown, the specific steps of a method for determining the movement of a camera based on image key points further include:

[0115] Step 4.1, Figure 9 is a schematic diagram of the translation of the camera in the up, down, left, and right directions in an embodiment of a method for determining the movement of a camera based on image key points of the present invention. Determine whether the camera has undergone translation in the up, down, left, and right directions. If the camera has undergone translation in the up, down, left, and right directions, the pictures taken are as Figure 9 shown.

[0116] Calculate the translation distance d:

[0117]

[0118] If the translation distance d≠0, the camera has undergone translation in the up or down or left or right direction. If the translation distance d = 0, the camera has not undergone translation in the up, down, left, and right directions, and the value of the translation distance d is the distance of the translation.

[0119] The second determination scenario: Based on the new width and new height of the second picture, determine the ratio of the product of the new width and new height of the second picture to the product of the original width and original height of the first picture, denoted as the width-to-height ratio. If the width-to-height ratio is equal to 1, it is determined that the camera has not undergone translational movement in the front-back direction. If the width-to-height ratio is greater than 1, it is determined that the camera has moved forward. If the width-to-height ratio is less than 1, it is determined that the camera has moved backward.

[0120] As Figure 8 shown, the specific steps of a method for determining camera movement based on image key points further include:

[0121] Step 4.2, Figure 10 This is a schematic diagram of the translational movement of the camera in the front-back direction in an embodiment of the method for determining camera movement based on image key points of the present invention. Determine whether the camera has undergone translational movement in the front-back direction. If there is translational movement in the front-back direction, the size of the captured picture will change, and the captured picture is as Figure 10 shown.

[0122] Therefore, it is judged through the following formula, and the width-to-height ratio r is calculated:

[0123]

[0124] If the width-to-height ratio r = 1, the camera has not undergone translational movement in the front-back direction. If the width-to-height ratio r > 1, the camera moves forward. If the width-to-height ratio r < 1, the camera moves backward.

[0125] The third determination scenario: Based on the new coordinates of the corner points and the center point in the second picture, determine the rotation angle of the camera. If the rotation angle is 0, it is determined that the camera has not rotated. If the rotation angle of the camera is not 0, it is determined that the camera has rotated, and the rotation angle of the camera is the rotation angle.

[0126] As Figure 8 shown, the specific steps of a method for determining camera movement based on image key points further include:

[0127] Step 4.3, Figure 11 This is a schematic diagram of the rotation of the camera in an embodiment of the method for determining camera movement based on image key points of the present invention. Determine whether the camera has rotated. If it has rotated, the captured picture is as Figure 11 shown.

[0128] Calculate the rotation angle θ:

[0129]

[0130] If the rotation angle θ = 0, the camera does not rotate. If the rotation angle θ ≠ 0, the camera rotates, and the value of the rotation angle θ is the specific rotation angle.

[0131] A method for determining the movement of a camera based on image key points provided by the solution of the present invention can accurately determine whether the camera moves by comparing two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement, and the calculation results are accurate and reliable. At the same time, the present invention effectively solves the technical problem of false alarms in the areas monitored by the camera (such as the intelligent security field) due to the inability to determine whether the camera moves, avoiding the waste of a large amount of time and energy by monitoring management personnel when receiving false alarm information, saving time and effort.

[0132] Adopting the technical solution of this embodiment, it can accurately determine whether the camera moves by comparing two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement. Thus, by judging whether the camera itself moves, it is possible to reduce or even avoid false alarms in the area monitored by the camera due to the movement of the camera itself.

[0133] According to an embodiment of the present invention, there is also provided a camera position determination device corresponding to the camera position determination method. Refer to Figure 6 the structural schematic diagram of an embodiment of the device of the present invention shown. The camera position determination device may include: an acquisition unit 102, a detection unit 104, and a determination unit 106.

[0134] Among them, the acquisition unit 102 is configured to acquire two pictures taken by the camera for the same shooting object at different times, denoted as the first picture and the second picture. The specific functions and processing of the acquisition unit 102 are referred to step S110.

[0135] The detection unit 104 is configured to detect key points in the first picture and the second picture, and perform matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture. The specific functions and processing of the detection unit 104 are referred to step S120.

[0136] In some embodiments, the detection unit 104 detects key points in the first picture and the second picture, and performs matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture, including:

[0137] The detection unit 104 is specifically further configured to detect the key points of the first picture by using the key point detection technology, denoted as the first key point set; and detect the key points of the second picture, denoted as the second key point set. For the specific functions and processing of the detection unit 104, refer to step S210.

[0138] The detection unit 104 is specifically further configured to sample the key point matching technology to match the similar key points among the key points in the first key point set and the second key point set, and obtain l pairs of key points, where l is a positive integer. The l pairs of key points form a sample set. For the specific functions and processing of the detection unit 104, refer to step S220.

[0139] Considering that in the actual application scenario, the camera may perform translational or rotational movements in any direction. Figure 7 This is a schematic diagram of pictures A and B taken by a camera in an embodiment of a camera movement determination device based on image key points according to the present invention. As Figure 8 shown, the camera takes the first picture (such as picture A) at the first moment (such as time t0), and takes the second picture (such as picture B) after a set time period (such as time Δt). The width of picture A is w A and the height is h A , the original width of picture B is w B and the original height is h B . Among them, the size markings of picture A and picture B are unified.

[0140] In the solution of the present invention, the key points in pictures A and B are detected and the similar key points are matched. Specifically, the key point detection and key point matching technology can be used to detect the key points in pictures A and B and match the similar key points. Among them, the key point detection and key point matching technology can adopt the key point detection and key point matching technology described in the documents David G. Lowe. Object Recognition from Local Scale-Invariant Features. 1999 and David G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. 2004.

[0141] In the solution of the present invention, it is assumed that after detecting the key points in pictures A and B, there are m key points Ri in picture A i , i = 1, 2,..., m, and there are n key points Sj in picture B j, j = 1, 2, …, n, where both m and n are positive integers. After performing similar key-point matching on the key points in Picture A and Picture B, a total of l groups of key-point pairs are matched between Picture A and Picture B These matched key-point pairs form a sample set. The pixel coordinates of the key points in Picture A and Picture B are respectively

[0142] Among them, are the pixel coordinates of the key points in Picture A, are the pixel coordinates of the key points in Picture B. 1 ≤ i k ≤ m, 1 ≤ j k ≤ n, k = 1, 2, …, l, where l is the number of groups of key points matched between Picture A and Picture B, and l is a positive integer.

[0143] Based on the data information obtained by detecting the key points in Picture A and Picture B and performing similar key-point matching, such as the pixel coordinates of the key points in Picture A and Picture B being respectively The solution of the present invention provides a camera movement determination device based on image key points.

[0144] The determination unit 106 is configured to determine the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture with the first picture as the reference system based on the sample set. For the specific functions and processing of this determination unit 106, refer to step S130.

[0145] In some embodiments, the determination unit 106 determines the optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture with the first picture as the reference system based on the sample set, including:[[]]

[0146] The determination unit 106 is specifically further configured to select a set of key-point pairs that have not been selected from the sample set and calculate the single mapping transformation matrix. For the specific functions and processing of this determination unit 106, refer to step S310 as well.

[0147] Figure 8 is a schematic flowchart of a camera movement determination device based on image key points of the present invention. As Figure 8 shown, the specific steps of a camera movement determination device based on image key points include:[[]]

[0148] Step 1: With Picture A as the reference system, calculate the optimal single mapping transformation matrix for mapping the key points in Picture B to Picture A. The specific calculation means is:[[]]

[0149] Step 1.1: Calculate the single mapping transformation matrix H 3×3 :

[0150]

[0151] For the convenience of processing and calculation, the single mapping transformation matrix H 3×3 and the element h 3×3 in it are normalized as follows: Let h 3×3 = 1.

[0152] The single mapping transformation matrix H 3×3 contains eight unknown elements. Therefore, from the l groups of key point pairs matched between picture A and picture B in the composed sample set, 4 groups of key point pairs that have not been selected are randomly selected. Suppose these 4 groups of key point pairs are The pixel coordinates of the key points in picture A and picture B are respectively where k = k1, k2, k3, k4, which represents four random numbers among k = 1, 2,..., l, and l is the number of groups of key points matched between picture A and picture B.

[0153] For each group of key point pairs, the following system of equations can be established:

[0154]

[0155] where k takes k1, k2, k3, k4 respectively. A total of eight equations can be obtained from the 4 groups of key point pairs. By combining the eight equations, the eight unknown elements of the single mapping transformation matrix H 3×3 can be obtained.

[0156] The determining unit 106 is specifically further configured to determine the effective set in the sample set according to the single mapping transformation matrix. For the specific functions and processing of the determining unit 106, refer to step S320.

[0157] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include: Step 1.2, determining the effective set according to the single mapping transformation matrix.

[0158] In some embodiments, the determining unit 106 determines the effective set in the sample set according to the single mapping transformation matrix, including:

[0159] The determining unit 106 is specifically further configured to calculate the back-projection error rate of each key point pair for other key point pairs in the sample set one by one according to the single mapping transformation matrix. The other key point pairs are the remaining key point pairs that have not been selected in the sample set. For the specific functions and processing of the determining unit 106, refer to step S410.

[0160] The determination unit 106 is further specifically configured to determine that the single mapping transformation matrix is valid for a key point pair if the back-projection error rate of any key point pair in the other key point pairs is less than a set error rate, and mark the key point pair as a valid key point pair. For the specific functions and processes of the determination unit 106, refer to step S420.

[0161] The determination unit 106 is further specifically configured to determine the set composed of the set of set key point pairs and all valid key point pairs in the other key point pairs as a valid set. For the specific functions and processes of the determination unit 106, refer to step S430.

[0162] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0163] In step 1.2, the specific device for determining a valid set according to the single mapping transformation matrix is as follows:

[0164] Judge whether the single mapping transformation matrix H 3×3 calculated in step 1.1 is valid for other key point pairs in the sample set. Here, the other key points refer to the key point pairs in the sample set that are not selected in step 1.1 for calculating the single mapping transformation matrix H 3×3 of.

[0165] The single mapping transformation matrix H 3×3 is calculated from 4 groups of key point pairs randomly selected from the sample set. Therefore, the single mapping transformation matrix H 3×3 is necessarily valid for these 4 groups of key points. Whether the single mapping transformation matrix H 3×3 is valid for other key points in the sample set needs to be further verified. The specific verification device is as follows:

[0166] Substitute the single mapping transformation matrix H 3×3 into the following formula to calculate and judge the back-projection error rate E of each key point pair in the sample set one by one k :

[0167]

[0168] where E k represents the back-projection error rate, k = 1, 2,..., l, and l is the number of groups of key points that match between picture A and picture B.

[0169] If the back-projection error rate E of a certain key point pair k <10, it is considered that the single mapping transformation matrix is valid for this key point pair. After calculating and judging each key point pair in the sample set one by one, count the number of key point pairs in the sample set that meet the back-projection error rate E kAll key point pairs of <10 conditions, and the set composed of these key point pairs is the valid set, and each key point pair is an element of this valid set.

[0170] The determining unit 106 is further specifically configured to determine the current cardinality of the valid set. If the current cardinality of the valid set is greater than the historical cardinality, it is determined that the valid set is the current optimal valid set. Wherein, the current cardinality of the valid set is the number of elements in the valid set determined in this loop. For the specific functions and processing of this determining unit 106, please also refer to step S330.

[0171] Such as Figure 8 As shown, the specific steps of a camera movement determination device based on image key points further include: Step 1.3, determining the current optimal valid set according to the cardinality of the valid set. The specific device is:

[0172] Judge whether the valid set obtained in this loop is the optimal current valid set according to the cardinality of the valid set. If so, update the current optimal valid set. If not, keep the current optimal valid set.

[0173] The criterion for judging whether the valid set obtained in this loop is the optimal valid set is: the cardinality of the valid set obtained in this loop is greater than the cardinality of the valid sets obtained in each loop operation in the history. If the cardinality of the valid set obtained in this loop is greater than the cardinality of the valid sets obtained in each loop operation in the history, it means that the valid set obtained in this loop is the set that is valid for the current most key point pairs, so it is the current optimal valid set.

[0174] The determining unit 106 is further specifically configured to determine the optimal single mapping transformation matrix according to the current optimal valid set. For the specific functions and processing of this determining unit 106, please also refer to step S340.

[0175] Such as Figure 8 As shown, the specific steps of a camera movement determination device based on image key points further include: Step 1.4, determining the optimal single mapping transformation matrix

[0176] In some embodiments, the determining unit 106 determines the optimal single mapping transformation matrix according to the current optimal valid set, including:

[0177] The determining unit 106 is further specifically configured to determine whether the number of cycles of the current optimal valid set is less than a set number, or whether the current error probability of the sample set is greater than a set error probability. The current error probability of the sample set is the ratio of the number of key point pairs in the other key point pairs whose back-projection error rate is greater than or equal to the set error rate to the total number of key point pairs in the sample set. For the specific functions and processing of this determining unit 106, refer to step S510.

[0178] The determining unit 106 is further specifically configured to, if it is determined that the number of cycles of the current optimal valid set is less than the set number, or the current error probability of the sample set is greater than the set error probability, continue to cycle to determine the current optimal valid set. For the specific functions and processing of this determining unit 106, refer to step S520.

[0179] The determining unit 106 is further specifically configured to, if it is determined that the number of cycles of the current optimal valid set is greater than or equal to the set number, and / or the current error probability of the sample set is less than or equal to the set error probability, determine the current optimal valid set as the optimal single mapping transformation matrix. For the specific functions and processing of this determining unit 106, refer to step S530.

[0180] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0181] In step 1.4, determine the optimal single mapping transformation matrix The specific device is:

[0182] Judge whether the loop condition is satisfied: the number of cycles is less than 2000 or the current error probability p > 0.1. If it is satisfied, loop and execute steps 1.1 to 1.3. If it is not satisfied, take the single mapping transformation matrix corresponding to the current optimal valid set as the optimal single mapping transformation matrix

[0183]

[0184] Among them, p represents the current error probability, and the calculation device is:

[0185]

[0186] In the formula, N is the number of key points in the sample set that satisfy the condition that the back-projection error rate E k ≥10, and L is the total number of key points in the sample set.

[0187] The determining unit 106 is further configured to determine new coordinates of the corner points and the center point in the second picture according to the original coordinates of the corner points and the center point in the second picture and the optimal single mapping transformation matrix. For the specific functions and processes of this determining unit 106, refer to step S140 for details.

[0188] In some embodiments, the determining unit 106 determines new coordinates of the corner points and the center point in the second picture according to the original coordinates of the corner points and the center point in the second picture and the optimal single mapping transformation matrix, including:

[0189] The determining unit 106 is specifically further configured to arrange the original coordinates of a set number of corner points and the center point in the second picture into a matrix with a set number of rows and a set number of columns, and then multiply it by the optimal single mapping transformation matrix to obtain the new coordinates of the set number of corner points and the center point in the second picture in the reference system of the first picture.

[0190] Similarly, the determining unit 106 is specifically further configured to obtain the new coordinates of the other corner points and the center point in the second picture in the reference system of the first picture. Herein, the other corner points are the remaining corner points among all the corner points in the second picture except the set number of corner points.

[0191] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0192] Step 2: Calculate the new coordinates of the 4 corner points and the center point in picture B in the reference system of picture A according to the original coordinates of the 4 corner points and the center point in picture B and the optimal single mapping transformation matrix. The specific device is as follows: The center point coordinates (x

[0193] , y cA , y cA ) of picture A = (w A / 2, h A / 2), where w A , h A are the width and height of picture A respectively.

[0194] The original coordinates of the 4 corner points and the center point in picture B are: the upper left corner point (x 11 , y 11 ) = (0, 0), the lower left corner point (x 21 , y 21 ) = (0, h B ), the upper right corner point (x 12 , y 12 ) = (w B , 0), the lower right corner point (x 22 , y22 ) = (w B , h B ), the center point (x cB , y cB ) = (w B / 2, h B / 2).

[0195] Step 2.1: After arranging the original coordinates of the 4 corner points and the center point in Picture B into a matrix of 3 rows and 1 column, multiply them by the optimal single mapping transformation matrix to obtain the new coordinates of the 4 corner points and the center point in Picture B in the reference system of Picture A:

[0196] The upper left corner point (x' 11 , y' 11 ), the lower left corner point (x' 21 , y' 21 ), the upper right corner point (x' 12 , y' 12 ), the lower right corner point (x' 22 , y' 22 ), the center point (x' cB , y' cB ). Now, taking the calculation of the new coordinates (x' 11 , y' 11 ) of the upper left corner point as an example, the specific calculation device is as follows:

[0197]

[0198]

[0199] Among them, v0 represents the relative value of the abscissa of the upper left corner point in Picture B in the reference system of Picture A, v1 represents the relative value of the ordinate of the upper left corner point in Picture B in the reference system of Picture A, and v2 represents the coordinate reference value. The calculation devices for v0, v1, and v2 are as follows:

[0200]

[0201] Here, in order to multiply with the optimal single mapping transformation matrix , the original coordinates of the 4 corner points and the center point in Picture B are arranged into a matrix of 3 rows and 1 column, and for the convenience of calculation, the element in the 3rd row and 1st column is normalized: make the value of the element in the 3rd row and 1st column equal to 1.

[0202] It can be seen that

[0203] Similarly, the new coordinates of the other corner points and the center point in Picture B in the reference system of Picture A can be calculated: the lower left corner point (x' 21 , y' 21 ), the upper right corner point (x'12 , y' 12 ), the lower right corner point (x' 22 , y' 22 ), the center point (x' cB , y' cB ).

[0204] The determining unit 106 is further configured to determine the new width and new height of the second picture according to the new coordinates of the corner points and the center point in the second picture. For the specific functions and processes of the determining unit 106, refer to step S150.

[0205] In some embodiments, the determining unit 106 determines the new width and new height of the second picture according to the new coordinates of the corner points and the center point in the second picture, including: the determining unit 106 is specifically further configured to calculate the new width and new height of the second picture according to the new coordinates of the other corner points and the center point in the second picture when the first picture is used as the reference system.

[0206] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0207] Step 3: Calculate the new width w' B and new height h' B of picture B according to the new coordinates of the other corner points and the center point in picture B under the reference system of picture A:

[0208]

[0209]

[0210] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0211] Step 4: In an actual application scenario, there are several situations where the camera moves, for example: translation in the up, down, left, and right directions, translation in the front and back directions, and / or rotation. Therefore, according to the new coordinates of the 4 corner points and the center point in picture B under the reference system of picture A and the new width and height of picture B, it is determined whether the camera moves for each of these situations respectively. For specific examples, refer to the following.

[0212] The determining unit 106 is further configured to determine whether the camera moves according to at least one of the new coordinates of the corner points and the center point in the second picture, and the new width and new height of the second picture, so as to determine whether the position of the camera changes. For the specific functions and processes of the determining unit 106, refer to step S160.

[0213] Considering that in order to accurately detect whether the monitored object in the picture has changed, it is first necessary to detect whether the position of the camera has moved, which is one of the technical shortcomings in the field of intelligent security in related solutions. Therefore, based on the key point detection and key point matching technologies, the solution of the present invention proposes a camera movement determination device based on image key points, which accurately calculates the translation distance or rotation angle of the camera through two photos taken by the camera at different times, so as to accurately determine whether the camera has moved, effectively avoiding a large number of false alarm messages caused by the movement of the camera.

[0214] In some embodiments, the determining unit 106 determines whether the camera has moved according to at least one of the new coordinates of the corner points and the center point in the second picture, and the new width and new height of the second picture, including any of the following determining situations:

[0215] The first determining situation: The determining unit 106 is specifically further configured to determine the translation distance of the camera according to the new coordinates of the corner points and the center point in the second picture: If the translation distance is 0, it is determined that the camera has not undergone translation in the up, down, left, or right directions. If the translation distance is not 0, it is determined that the camera has undergone translation in the up or down or left or right direction, and the distance of the camera's translation is the translation distance.

[0216] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0217] Step 4.1, Figure 9 This is a schematic diagram of the translation of a camera in the up, down, left, or right direction in an embodiment of a camera movement determination device based on image key points of the present invention. Determine whether the camera has undergone translation in the up, down, left, or right directions. If the camera has undergone translation in the up, down, left, or right directions, the captured picture is as Figure 9 shown.

[0218] Calculate the translation distance d:

[0219]

[0220] If the translation distance d≠0, the camera has undergone translation in the up or down or left or right direction. If the translation distance d = 0, the camera has not undergone translation in the up, down, left, or right directions, and the value of the translation distance d is the distance of the translation.

[0221] The second determination scenario: The determination unit 106 is further specifically configured to determine, according to the new width and new height of the second picture, the ratio of the product of the new width and new height of the second picture to the product of the original width and original height of the first picture, denoted as the width-to-height ratio. If the width-to-height ratio is equal to 1, it is determined that the camera has not undergone translational movement in the front-back direction. If the width-to-height ratio is greater than 1, it is determined that the camera has moved forward. If the width-to-height ratio is less than 1, it is determined that the camera has moved backward.

[0222] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0223] Step 4.2, Figure 10 is a schematic diagram of the translational movement in the front-back direction of a camera in an embodiment of a camera movement determination device based on image key points of the present invention. To determine whether the camera has undergone translational movement in the front-back direction, if there is translational movement in the front-back direction, the size of the captured picture will change, and the captured picture is as Figure 10 shown.

[0224] Therefore, it is judged through the following formula by calculating the width-to-height ratio r:

[0225]

[0226] If the width-to-height ratio r = 1, the camera has not undergone translational movement in the front-back direction. If the width-to-height ratio r > 1, the camera moves forward. If the width-to-height ratio r < 1, the camera moves backward.

[0227] The third determination scenario: The determination unit 106 is further specifically configured to determine the rotation angle of the camera according to the new coordinates of the corner points and the center point in the second picture. If the rotation angle is 0, it is determined that the camera has not rotated. If the rotation angle of the camera is not 0, it is determined that the camera has rotated, and the rotation angle of the camera is the rotation angle.

[0228] As Figure 8 shown, the specific steps of a camera movement determination device based on image key points further include:

[0229] Step 4.3, Figure 11 is a schematic diagram of the rotation of a camera in an embodiment of a camera movement determination device based on image key points of the present invention. To determine whether the camera has rotated, if it has rotated, the captured picture is as Figure 11 shown.

[0230] Calculate the rotation angle θ:

[0231]

[0232] If the rotation angle θ = 0, the camera does not rotate. If the rotation angle θ ≠ 0, the camera rotates, and the value of the rotation angle θ is the specific rotation angle.

[0233] A camera movement determination device based on image key points provided by the solution of the present invention can accurately determine whether the camera moves by using two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement, and the calculation results are accurate and reliable. At the same time, the present invention effectively solves the technical problem of false alarms in the areas monitored by the camera (such as the intelligent security field) due to the inability to determine whether the camera moves, and avoids the waste of a large amount of time and energy by the monitoring management personnel in dealing with false alarm information when receiving false alarm information, saving time and effort.

[0234] Since the processing and functions implemented by the device in this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, for the details not described in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and no further details will be given here.

[0235] By adopting the technical solution of the present invention, it can accurately determine whether the camera moves by using two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement, effectively avoiding a large number of false alarm information caused by the movement of the camera.

[0236] According to an embodiment of the present invention, a security system corresponding to the camera position determination device is also provided. The security system may include: the above-mentioned camera position determination device.

[0237] Since the processing and functions implemented by the security system in this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing device, for the details not described in the description of this embodiment, reference can be made to the relevant descriptions in the foregoing embodiments, and no further details will be given here.

[0238] By adopting the technical solution of the present invention, it can accurately determine whether the camera moves by using two photos taken by the camera at different times, and can accurately calculate the translation distance or rotation angle in the case of movement, effectively solving the technical problem of false alarms in the areas monitored by the camera (such as the intelligent security field) due to the inability to determine whether the camera moves.

[0239] According to an embodiment of the present invention, a storage medium corresponding to the camera position determination method is also provided. The storage medium includes a stored program, wherein when the program runs, it controls the device where the storage medium is located to execute the above-mentioned camera position determination method.

[0240] Since the processing and functions implemented by the storage medium of this embodiment are basically corresponding to the embodiments, principles and examples of the foregoing method, for the details not described in the description of this embodiment, reference may be made to the relevant descriptions in the foregoing embodiments and will not be elaborated herein.

[0241] By adopting the technical solution of the present invention, it is possible to accurately determine whether the camera has moved by using two photos taken by the camera at different times, and to accurately calculate the translation distance or rotation angle in the case of movement, avoiding the waste of a large amount of time and energy by monitoring management personnel when receiving false alarm information, saving time and effort.

[0242] In summary, it is easy for those skilled in the art to understand that, on the premise of no conflict, the above advantageous ways can be freely combined and superimposed.

[0243] The above are only the embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for determining the position of a camera, characterized in that, Including: Obtain two pictures captured by the camera for the same shooting object at different times, denoted as the first picture and the second picture; Detect key points in the first picture and the second picture, and perform matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture; Based on the sample set, with the first picture as the reference system, determine the optimal single mapping transformation matrix for the key points in the second picture to map to the first picture; According to the original coordinates of the corner points and the center point in the second picture, and the optimal single mapping transformation matrix, determine the new coordinates of the corner points and the center point in the second picture; According to the new coordinates of the corner points and the center point in the second picture, determine the new width and new height of the second picture; According to the new coordinates of the corner points and the center point in the second picture, and at least one of the new width and new height of the second picture, determine whether the camera has moved to determine whether the position of the camera has changed; Among them, according to the new coordinates of the corner points and the center point in the second picture, and at least one of the new width and new height of the second picture, determine whether the camera has moved to determine whether the position of the camera has changed, including: According to the new coordinates of the corner points and the center point in the second picture, determine the translation distance of the camera: if the translation distance is 0, it is determined that the camera has not undergone translation in the up, down, left, or right direction; if the translation distance is not 0, it is determined that the camera has undergone translation in the up or down or left or right direction; According to the new width and new height of the second picture, determine the ratio of the product of the new width and new height of the second picture to the product of the original width and original height of the first picture, denoted as the width-height ratio: if the width-height ratio is equal to 1, it is determined that the camera has not undergone translation in the front-back direction; if the width-height ratio is greater than 1, it is determined that the camera has moved forward; if the width-height ratio is less than 1, it is determined that the camera has moved backward; According to the new coordinates of the corner points and the center point in the second picture, determine the rotation angle of the camera: if the rotation angle is 0, it is determined that the camera has not rotated; if the rotation angle is not 0, it is determined that the camera has rotated, and the rotation angle of the camera is the rotation angle; 2. The method for determining the position of the camera according to claim 1, wherein Detect key points in the first picture and the second picture, and perform matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture, including: Adopt key point detection technology to detect the key points of the first picture, denoted as the first key point group; and detect the key points of the second picture, denoted as the second key point group; Sample key point matching technology to match the similar key points among the key points in the first key point group and the second key point group to obtain l pairs of key points, where l is a positive integer; the l pairs of key points form a sample set; 3. The method for determining the position of the camera according to claim 1 or 2, characterized in that Based on the sample set, with the first picture as the reference system, determine the optimal single mapping transformation matrix for the key points in the second picture to map to the first picture, including: Select unselected key point pairs of a set from the sample set and calculate a single mapping transformation matrix; Determine an effective set in the sample set according to the single mapping transformation matrix; Determine the current cardinality of the effective set. If the current cardinality of the effective set is greater than the historical cardinality, determine the effective set as the current optimal effective set; wherein, the current cardinality of the effective set is the number of elements in the effective set determined in this loop; Determine the optimal single mapping transformation matrix according to the current optimal effective set.

4. The method for determining the position of the camera according to claim 3, wherein, Wherein, Determining the effective set in the sample set according to the single mapping transformation matrix includes: According to the single mapping transformation matrix, calculate the back-projection error rate of each key point pair for each of the other key point pairs in the sample set one by one; the other key point pairs are the remaining unselected key point pairs in the sample set; If the back-projection error rate of any key point pair in the other key point pairs is less than the set error rate, determine that the single mapping transformation matrix is effective for this key point pair, and record this key point pair as an effective key point pair; Determine the set composed of the set of key point pairs of the set and all effective key point pairs in the other key point pairs as the effective set; And / or, Determining the optimal single mapping transformation matrix according to the current optimal effective set includes: Determine whether the number of loops of the current optimal effective set is less than the set number of loops, or whether the current error probability of the sample set is greater than the set error probability; the current error probability of the sample set is the ratio of the number of key point pairs with back-projection error rates greater than or equal to the set error rate in the other key point pairs to the total number of key point pairs in the sample set; If it is determined that the number of loops of the current optimal effective set is less than the set number of loops, or the current error probability of the sample set is greater than the set error probability, continue to loop to determine the current optimal effective set; If it is determined that the number of loops of the current optimal effective set is greater than or equal to the set number of loops, and / or the current error probability of the sample set is less than or equal to the set error probability, determine the current optimal effective set as the optimal single mapping transformation matrix.

5. The camera position determination method according to claim 3, characterized in that Determine the new coordinates of the corner points and the center point in the second picture according to the original coordinates of the corner points and the center point in the second picture and the optimal single mapping transformation matrix, including: After arranging the original coordinates of the set number of corner points and the center point in the second picture into a matrix of set rows and set columns, multiply it by the optimal single mapping transformation matrix to obtain the new coordinates of the set number of corner points and the center point in the second picture when the first picture is used as the reference system; Similarly, obtain the new coordinates of the other corner points and the center point in the second picture when the first picture is used as the reference system; wherein, the other corner points are the remaining corner points in all the corner points in the second picture except the set number of corner points.

6. The method for determining the position of the camera according to claim 5, wherein, Determine the new width and new height of the second picture according to the new coordinates of the corner points and the center point in the second picture, including: Calculate the new width and new height of the second picture based on the new coordinates of other corner points and the center point in the second picture when the first picture is used as the reference system.

7. A camera position determination device, characterized in that, Including: An acquisition unit configured to acquire two pictures taken by the camera at different times for the same shooting object, denoted as the first picture and the second picture; A detection unit configured to detect key points in the first picture and the second picture, and perform matching of similar key points to obtain a sample set composed of pairs of similar key points in the first picture and the second picture; A determination unit configured to determine, based on the sample set and with the first picture as the reference system, an optimal single mapping transformation matrix for mapping the key points in the second picture to the first picture; The determination unit is further configured to determine the new coordinates of the corner points and the center point in the second picture according to the original coordinates of the corner points and the center point in the second picture and the optimal single mapping transformation matrix; The determination unit is further configured to determine the new width and new height of the second picture according to the new coordinates of the corner points and the center point in the second picture; The determination unit is further configured to determine whether the camera has moved and thus whether the position of the camera has changed according to at least one of the new coordinates of the corner points and the center point in the second picture and the new width and new height of the second picture; The determination unit is further configured to: Determine the translation distance of the camera according to the new coordinates of the corner points and the center point in the second picture: if the translation distance is 0, it is determined that the camera has not undergone translation in the up, down, left, or right direction; if the translation distance is not 0, it is determined that the camera has undergone translation in the up or down or left or right direction; Determine the ratio of the product of the new width and new height of the second picture to the product of the original width and original height of the first picture, denoted as the width-height ratio, according to the new width and new height of the second picture: if the width-height ratio is equal to 1, it is determined that the camera has not undergone translation in the front-back direction; if the width-height ratio is greater than 1, it is determined that the camera has moved forward; If the width-height ratio is less than 1, it is determined that the camera has moved backward; Determine the rotation angle of the camera according to the new coordinates of the corner points and the center point in the second picture: if the rotation angle is 0, it is determined that the camera has not rotated; If the rotation angle is not 0, it is determined that the camera has rotated, and the rotation angle of the camera is the rotation angle.

8. A security system, characterized in that, Including: The camera position determination device according to claim 7.

9. A storage medium, characterized in that, The storage medium includes a stored program, wherein when the program runs, it controls the device where the storage medium is located to execute the camera position determination method according to any one of claims 1 to 6.

Images