Image view angle transformation type detection device and method

Abstract

The embodiment of the invention provides an image view angle transformation type detection device and method, and the device comprises a first extraction unit which extracts a first number of sampling points from a first image; the first determining unit is used for determining the proportion of the positions of the first number of sampling points in the second image relative to the sampling points of which the positions in the first image have predetermined changes; the second extraction unit is used for extracting a first feature point from the first image and extracting a second feature point from the second image when the proportion is greater than or equal to a first threshold value; and the second determination unit is used for determining an image view angle transformation type according to the first feature point, the second feature point and a second number of reference points in the first image.

Description

Technical Field

[0001] The present application relates to the field of image processing. Background Art

[0002] With the improvement of computer performance and the popularization of camera equipment, the application of target tracking is becoming more and more extensive. Among them, in different images captured by the same camera in the same scene, the view of the camera may be different. In order to analyze the correlation of the regions of interest in different images, it is necessary to propose a method to detect how the view of the camera corresponding to different images changes.

[0003] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because they are described in the background technology section of this document. Summary of the invention

[0004] In response to at least one of the above problems, an embodiment of the present application provides an image perspective change type detection device and method.

[0005] According to a first aspect of an embodiment of the present application, a device for detecting an image perspective change type is provided, the device comprising:

[0006] A first extraction unit, which extracts a first number of sampling points from the first image;

[0007] a first determining unit, which determines a proportion of the sampling points of which positions of the first number of sampling points in the second image undergo a predetermined change relative to their positions in the first image;

[0008] a second extraction unit, which extracts first feature points from the first image and second feature points from the second image when the ratio is greater than or equal to a first threshold;

[0009] A second determining unit determines an image perspective transformation type according to the first feature point, the second feature point, and a second number of reference points in the first image.

[0010] According to a second aspect of an embodiment of the present application, a method for detecting an image perspective change type is provided, the method comprising:

[0011] extracting a first number of sampling points from the first image;

[0012] determining a proportion of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image;

[0013] When the ratio is greater than a first threshold, extracting first feature points from the first image and extracting second feature points from the second image;

[0014] An image perspective transformation type is determined according to the first feature point, the second feature point, and a second number of reference points in the first image.

[0015] One of the beneficial effects of the embodiments of the present application is that a two-step image perspective transformation type detection process is used to first extract global information, i.e., sampling points, from the image, and then extract local information, i.e., feature points, from the image. The image perspective transformation type is determined based on the global information and the local information. Thus, the image perspective transformation type can be automatically determined and the accuracy of the image perspective transformation type determination can be improved.

[0016] With reference to the following description and accompanying drawings, the specific embodiments of the present application are disclosed in detail, indicating the way in which the principles of the present application can be adopted. It should be understood that the embodiments of the present application are not limited in scope. Within the scope of the spirit and clauses of the appended claims, the embodiments of the present application include many changes, modifications and equivalents.

[0017] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0018] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps or components. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 is a schematic diagram of an image perspective change type detection device according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of sampling points and optical flow matching points in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a first determination unit in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a second determination unit in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0025] Figure 6 is a schematic block diagram of the system structure of the electronic device according to an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a method for detecting image perspective change type according to an embodiment of the present application;

[0027] Figure 8 is a schematic diagram of a method for detecting image perspective change type according to an embodiment of the present application;

[0028] Figures 9 to 11 It is a schematic diagram of the polygon in the second image of an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0030] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0031] With reference to the accompanying drawings, the above and other features of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents falling within the scope of the attached claims.

[0032] Embodiments of the first aspect

[0033] An embodiment of the present application provides a device for detecting image perspective change type. Figure 1is a schematic diagram of an image perspective change type detection device according to an embodiment of the present application. Figure 1 As shown, the image perspective change type detection device 100 includes:

[0034] A first extraction unit 101, which extracts a first number of sampling points from a first image;

[0035] A first determining unit 102, which determines a proportion of the sampling points of which positions of the first number of sampling points in the second image undergo a predetermined change relative to their positions in the first image;

[0036] A second extraction unit 103, which extracts first feature points from the first image and second feature points from the second image when the ratio is greater than or equal to a first threshold;

[0037] The second determining unit 104 determines an image perspective transformation type according to the first feature point, the second feature point, and a second number of reference points in the first image.

[0038] In some embodiments, the first image and the second image are two frames of images in a video stream acquired by the same camera, which can be obtained from a camera shooting in an outdoor or indoor area, for example, obtained by shooting with a set surveillance camera. After acquiring a frame of a real-time surveillance video, the image is used as the first image. After a predetermined time, another frame of image is acquired from the surveillance video, and the other frame of image is used as the second image. The predetermined time can be one day, one hour, 12 hours, or one week, etc., and the embodiments of the present application are not limited thereto. The first image and the second image can be two-dimensional color images (RGB) or two-dimensional grayscale images, and the embodiments of the present application are not limited thereto. The first image and the second image have the same size or are processed to be the same.

[0039] In the embodiment of the present application, a two-step image perspective change type detection process is used, firstly extracting global information from the image, determining whether the image perspective has changed based on the global information, and then extracting local information from the image when it is determined that a change has occurred, and determining the image perspective change type based on the local information. Each unit and the specific processing process are described below.

[0040] In some embodiments, the first extraction unit 101 extracts a first number of sampling points from the first image; the first number of sampling points are evenly distributed on the first image, and can reflect the global information of the first image. For example, the first extraction unit 101 can evenly extract the first number of sampling points at the same interval in the length direction and the width direction of the first image. Figure 2 is a schematic diagram of the first number of sampling points extracted in the embodiment of the present application, such as Figure 2As shown, the distances between the sampling points are the same. However, the present application is not limited to this, and the first number of sampling points can be extracted at a first interval in the length direction and at a second interval in the width direction, which will not be exemplified one by one here.

[0041] In some embodiments, the first determining unit 102 determines a proportion of the sampling points of the first number of sampling points (global sampling points) whose positions in the second image undergo a predetermined change relative to their positions in the first image; Figure 3 Schematic diagram of the first determination unit in the embodiment of the present application. Figure 3 As shown, the first determining unit 102 includes:

[0042] A first calculation module 301, which calculates the optical flow of the first number of sampling points in the second image;

[0043] The first determination module 302 determines whether the positions of the first number of sampling points in the second image relative to the positions in the first image undergo the predetermined change according to the optical flows corresponding to the first number of sampling points, and counts the proportion of the sampling points that undergo the predetermined change.

[0044] In some embodiments, since the camera perspective may change, the positions of the first number of sampling points in the second image may change. For example, the first calculation module 301 may calculate the optical flow of the first number of sampling points in the second image, and calculate the optical flow matching points of the first number of sampling points (global sampling points) in the second image. For the method of calculating the optical flow, reference may be made to the relevant technology, which will not be described in detail here. Figure 2 The optical flow matching points and the optical flow (line segment) schematic diagram of the second intra-image optical flow method are shown.

[0045] In some embodiments, the first determination module 302 determines whether the positions of the first number of sampling points in the second image relative to the positions in the first image undergo the predetermined change according to the optical flows corresponding to the first number of sampling points, and counts the proportion of the sampling points that undergo the predetermined change. Among them, it can be determined that the optical flow matching points in the second image are determined according to the optical flow calculated by the first calculation module 301, that is, each sampling point has a corresponding optical flow matching point in the second image, and then determine whether the positions of the optical flow matching points in the second image have undergone a predetermined change relative to the first number of sampling points in the first image, or determine whether the positions of the optical flow matching points in the second image are roughly consistent with the positions of the first number of sampling points in the first image. For example, the pixel coordinate position of the optical flow matching point in the second image can be determined, and the pixel coordinate position of the first number of sampling points in the first image can be determined, and the pixel coordinate position difference between the two can be calculated one by one. If the difference is equal to 0, it means that the positions of the corresponding sampling points are exactly the same and have not changed. If the difference is not 0, it means that the positions of the corresponding sampling points have changed. However, the embodiments of the present application are not limited to this. Taking the error into consideration, if the absolute value of the pixel coordinate position difference is greater than a preset value, it indicates that the position of the corresponding sampling point has undergone a predetermined change. If it is less than or equal to the preset value, it indicates that the position of the corresponding sampling point has not undergone a predetermined change. The preset value can be determined as needed and will not be given one by one here.

[0046] In some embodiments, the first determination module 302 counts the proportion of sampling points that undergo the predetermined change, that is, among the first number of sampling points, if the number of sampling points that undergo the predetermined change is a third number according to the aforementioned method, then the proportion of sampling points that undergo the predetermined change is the third number / the first number.

[0047] In some embodiments, the ratio is compared with a first threshold value, and whether the image viewing angle has changed is determined based on the comparison result. The device also includes:

[0048] The third determination unit 105 (optional) determines that the image perspective has not changed when the ratio is less than the first threshold and less than the second threshold, and determines that the image perspective change type is that the image perspective has not changed but the layout of objects in the image has changed when the ratio is less than the first threshold but greater than or equal to the second threshold.

[0049] In some embodiments, when the ratio is less than the first threshold, it indicates that the number of sampling points where the predetermined change occurs is relatively small, that is, the predetermined change is not caused by the camera (image) perspective change. Therefore, when the ratio is less than the first threshold, it is determined that the image perspective has not changed. Further, when it is determined that the perspective has not changed, it is further possible to determine whether the predetermined change is caused by the object layout change in combination with the second threshold, and the second threshold is less than the first threshold. For example, when the ratio is less than the second threshold, the predetermined change may be caused by the object entering or moving out of the image, and the image perspective has not changed. When the ratio is less than the first threshold but greater than or equal to the second threshold, the predetermined change is relatively increased, but it is still not caused by the perspective conversion, but caused by the change of the object layout in the image, and the image perspective change type is determined to be the image perspective has not changed but the object layout in the image has changed. For example, the first threshold is set to 90%, and the second threshold is set to 50%. When the statistical ratio is less than 90% but greater than 50%, it indicates that the image perspective change type is the image perspective has not changed but the object layout in the image has changed. For example, the change of the object layout in the image indicates the change caused by the decoration of the scene in the image or the movement of furniture due to holidays. When the statistical proportion is less than 50%, it indicates that the image perspective transformation type is that the image perspective is not transformed. The threshold is determined according to needs, and the embodiment of the present application is not limited thereto.

[0050] In some embodiments, when the statistical ratio is greater than or equal to the first threshold, it means that the number of sampling points that have undergone predetermined changes is large, and the change is caused by the camera (image) perspective transformation. The transformation type includes zooming in, zooming out, rotating, and resetting. The following describes how to make the judgment.

[0051] In some embodiments, when the ratio is greater than or equal to the first threshold, the second extraction unit 103 extracts the first feature point from the first image and the second feature point from the second image; for example, the second extraction unit uses a predetermined feature operator to extract the first feature point and the second feature point, and the predetermined feature operator may be a SIFT operator, etc., and the embodiments of the present application are not limited thereto. For example, a feature extraction neural network may also be used to extract the first feature point and the second feature point, which is not exemplified here. The first feature point and the second feature point may reflect information such as texture, color, and edge of the image, and compared to the aforementioned sampling points, the first feature point and the second feature point may be regarded as local features of the image.

[0052] In some embodiments, the second determining unit 10 determines whether the image perspective transformation type is enlargement, reduction, rotation, or reset according to the first feature point, the second feature point, and a second number of reference points in the first image. Figure 4 is a schematic diagram of the second determination unit in an embodiment of the present application. Figure 4 As shown, the second determining unit includes:

[0053] A matching module 401, which determines a matching feature point pair from the first feature point and the second feature point;

[0054] A second calculation module 402, which determines a homography matrix according to the matched feature point pairs;

[0055] A generating module 403, which generates a polygon in the second image according to the homography matrix and the reference point;

[0056] The second determining module 404 determines an image perspective transformation type according to the polygon and the second image.

[0057] In some embodiments, the matched feature points represent corresponding feature points in different images. For example, the matching module 401 searches for a matching point consistent with the first feature point among the second feature points in the second image. If found, the first feature point has a matching second feature point in the second image, and the two can be called a matched feature point pair. The matching method can be based on the FLANN feature matching algorithm, or other feature matching algorithms can be used, such as k-nearest neighbor matching, etc. The embodiments of the present application are not limited to this.

[0058] In some embodiments, the second calculation module 402 determines a homography matrix (perspective transformation matrix) based on the matched feature point pairs. The homography matrix describes the transformation relationship between two images taken at different viewing angles for the same object. The homography matrix H is a 3×3 matrix with 8 degrees of freedom. Let the first feature point in the matched feature point pair be p1, and the coordinates of the second feature point in the matched feature point pair be p2, then p2=Hp1. Since the coordinate values ​​of p1 and p2 are known, when there are multiple matched feature point pairs (for example, at least four pairs), the homography matrix can be obtained. The specific calculation method can refer to the prior art and will not be repeated here.

[0059] In some embodiments, the generation module generates a polygon in the second image according to the homography matrix and the reference point. The reference point may be at least three points in the first image. The at least three reference points may form a plane. The generation module is equivalent to projecting the plane formed by the at least three reference points to a new visual plane (polygon) in the second image according to the homography matrix. The generation module multiplies the coordinates of the reference point in the first image by the homography matrix to obtain the coordinates of the reference point in the plane of the second image (the coordinates of the projection point). The projection point of the reference point may be located in the second image or outside the second image. The outline of the closed area formed by connecting the projection points is used as the polygon.

[0060] For example, the polygon may be generated using the cv2.perspectiveTransform() function in OpenCV. For details about the function, please refer to the relevant technology and will not be described in detail here.

[0061] In some embodiments, the reference point can be a point at any three or more positions, but for ease of calculation, the reference point includes at least three (the second number is greater than or equal to 3) vertices in the first image. For example, the four vertices (the second number) of the upper left corner, upper right corner, lower left corner, and lower right corner in the first image can be used as reference points. Assuming the width and height of the image are w and h respectively, the coordinates of the four reference points can be expressed as (0,0), (0,w-1), (w-1,h-1), (w-1,0). The coordinates of the four reference points are perspective transformed, that is, multiplied by the homography matrix, to obtain the coordinates of the positions of the projection points of the four reference points in the second image. The outline of the closed area formed by connecting the projection points of the four reference points is used as the polygon, which can reflect the projection range of the first image in the second image. Figures 9 to 11 A schematic diagram of the polygon generated in the second image is shown.

[0062] In some embodiments, the second determination module 404 determines whether the polygon is located in the second image, and when it is determined that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is less than or equal to a third threshold, determines that the image perspective transformation type is perspective reset (e.g., Fig.11 ). If the polygon is located in the second image but the area of ​​the polygon is very small, the second image is obtained by taking the camera after resetting the viewing angle, and the first image is obtained before resetting the viewing angle. Due to the resetting viewing angle, the first image and the second image have little correlation, or the first image and the second image are respectively taken at viewing angles with a large difference.

[0063] In some embodiments, the second determination module 404 determines whether the polygon is located in the second image, and when it is determined that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is greater than a third threshold, determines that the image perspective transformation type is zooming out (e.g., Fig.10 As shown). If the polygon is located in the second image, but the area of ​​the polygon is larger, the camera obtains the second image after reducing the viewing angle, and the first image is obtained before reducing the viewing angle. Due to the reduced viewing angle, the information in the first image can be completely projected in the second image (the polygon is located in the second image), but the projected area needs to reflect all the contents of the first image, so the ratio of the projected area to the area of ​​the second image needs to be greater than the third threshold.

[0064] For example, the third threshold value can be set to a smaller value, such as 10%, etc., and the embodiments of the present application are not limited thereto. That is, if the polygon is completely located in the second image, but the ratio of the area of ​​the polygon to the area of ​​the second image is less than or equal to 10%, then the image perspective transformation type is determined to be perspective reset, and if the ratio of the area of ​​the polygon to the area of ​​the second image is greater than 10%, then the image perspective transformation type is determined to be reduction.

[0065] In some embodiments, the second determination module 404 determines whether the polygon is located in the second image. When the polygon is not located in the second image, but the second image is located in the polygon, the second determination module 404 determines that the image perspective transformation type is magnification. If the second image is located in the polygon, the camera is the second image captured after the perspective is magnified, and the first image is captured before the perspective is magnified. Due to the magnified perspective, the information in the first image cannot be completely projected in the second image (the second image is located in the polygon), and the second image reflects part of the content of the first image, which is equivalent to the magnified perspective.

[0066] In some embodiments, the second determination module 404 determines whether the polygon is located in the second image. When the polygon is not located in the second image and the second image is not located in the polygon, the second determination module 404 determines whether the second image and the polygon have an overlapping area. When there is an overlapping area, the image perspective transformation type is determined to be rotation (e.g., Fig. 9 As shown), when there is no overlapping area, the image perspective transformation type is determined to be perspective reset. Wherein, if the second image is not located in the polygon and the polygon is not located in the second image, the perspective transformation type will not be zooming in or out. If the second image and the polygon have overlapping areas, it means that the first image and the second image have a certain correlation. The camera is the second image taken after the perspective is rotated, and the first image is taken before the perspective is rotated. Due to the rotation of the perspective, part of the information in the first image can be projected in the second image, but the other part cannot be projected in the second image. Therefore, the second image and the polygon have overlapping areas, but cannot contain each other. If the second image and the polygon have no overlapping areas, it means that the correlation is small, or the first image and the second image are taken at different perspectives. The camera is the second image taken after resetting the perspective, and the first image is taken before resetting the perspective. Due to the reset perspective, the correlation between the first image and the second image is small, and the information in the first image may not be projected in the second image at all. Therefore, the second image and the polygon have no overlapping areas.

[0067] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The image perspective change type detection device 100 may also include other components or modules, and the specific contents of these components or modules may refer to the relevant technology.

[0068] To keep it simple, Figure 1 The connection relationship or signal direction between various components or modules is only exemplified, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors and memories; the embodiments of the present application are not limited to this.

[0069] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0070] It can be seen from the above embodiments that a two-step image perspective change type detection process is used to first extract global information, i.e., sampling points, from the image, and then extract local information, i.e., feature points, from the image. The image perspective change type is determined based on the global information and the local information. In this way, the determination of the image perspective change type can be automated and the accuracy of the determination of the image perspective change type can be improved.

[0071] Embodiments of the second aspect

[0072] The embodiment of the present application also provides an electronic device, Figure 5 A schematic diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes an image perspective change type detection device 501. The structure and function of the image perspective change type detection device 501 are the same as those of the image perspective change type detection device 100 in the embodiment of the first aspect. For specific contents, please refer to the description in the embodiment of the first aspect, and will not be repeated here.

[0073] Figure 6 1 is a schematic block diagram of the system structure of the electronic device according to the embodiment of the present application. Figure 6 As shown, the electronic device 600 may include a processor 601 and a memory 602; the memory 602 is coupled to the processor 601. This figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0074] In some embodiments, processor 601 includes at least one of a central processing unit (CPU) and a graphics processing unit (GPU).

[0075] like Figure 6 As shown, the electronic device 600 may further include: an input device 603 , a display 604 , and a power supply 605 .

[0076] In one embodiment, the function of the image perspective change type detection device can be integrated into the processor 601. The processor 601 can be configured to: extract a first number of sampling points from the first image; determine the proportion of sampling points whose positions in the second image of the first number of sampling points undergo a predetermined change relative to their positions in the first image; when the proportion is greater than or equal to a first threshold, extract a first feature point from the first image and a second feature point from the second image; determine the image perspective change type based on the first feature point, the second feature point and the second number of reference points in the first image. For other implementations, please refer to the embodiment of the first aspect, which will not be described in detail here.

[0077] In another embodiment, the image perspective change type detection device can be configured separately from the processor 601. For example, the image perspective change type detection device can be configured as a chip connected to the processor 601, and the function of the image perspective change type detection device is realized through the control of the processor 601.

[0078] In this embodiment, the electronic device 600 does not necessarily include Figure 6 All parts shown in .

[0079] like Figure 6 As shown, the processor 601 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The processor 601 receives inputs and controls the operations of various components of the electronic device 600 .

[0080] The memory 602 may be, for example, a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The processor 601 may execute the program stored in the memory 602 to implement information storage or processing. The functions of other components are similar to those of the prior art and are not described in detail herein. The components of the electronic device 600 may be implemented by dedicated hardware, firmware, software or a combination thereof without departing from the scope of the present application.

[0081] It can be seen from the above embodiments that a two-step image perspective change type detection process is used to first extract global information, i.e., sampling points, from the image, and then extract local information, i.e., feature points, from the image. The image perspective change type is determined based on the global information and the local information. In this way, the determination of the image perspective change type can be automated and the accuracy of the determination of the image perspective change type can be improved.

[0082] Embodiments of the third aspect

[0083] An embodiment of the present application also provides a method for detecting an image perspective change type, which corresponds to the image perspective change type detection device of the first aspect embodiment. Figure 7 FIG. 1 is a schematic diagram of a method for detecting image perspective change type according to an embodiment of the present application. Figure 7 As shown, the method includes:

[0084] 701, extracting a first number of sampling points from a first image;

[0085] 702, determining a proportion of the sampling points of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image;

[0086] 703, when the ratio is greater than or equal to a first threshold, extracting first feature points from the first image and extracting second feature points from the second image;

[0087] 704 : Determine an image perspective transformation type according to the first feature point, the second feature point, and a second number of reference points in the first image.

[0088] In this embodiment, the specific implementation method of each of the above steps is the same as that described in the embodiment of the first aspect and will not be repeated here.

[0089] Figure 8 FIG. 1 is a schematic diagram of a method for detecting image perspective change type according to an embodiment of the present application. Figure 8 As shown, the method includes:

[0090] 801, extracting a first number of sampling points from a first image;

[0091] 802, determining a proportion of the sampling points of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image;

[0092] 803, determine whether the ratio is greater than or equal to a first threshold; if the determination result is no, execute 804 to 806, otherwise execute 807 to 811;

[0093] 804, determining whether the ratio is greater than or equal to a second threshold, if the determination result is yes, executing 805, otherwise executing 806;

[0094] 805, determining that the image perspective transformation type is that the image perspective is not transformed but the layout of objects in the image is changed;

[0095] 806, determining that the image viewing angle has not been changed;

[0096] 807, extracting first feature points from the first image, and extracting second feature points from the second image;

[0097] 808, determining a matching feature point pair from the first feature point and the second feature point;

[0098] 809, determining a homography matrix according to the matched feature point pairs;

[0099] 810, generating a polygon in the second image according to the homography matrix and the reference point;

[0100] 811, determine whether the polygon is located in the second image, if the determination result is yes, execute 812 to 814, otherwise execute 815;

[0101] 812, determine whether the ratio of the area of ​​the polygon to the area of ​​the second image is less than or equal to a third threshold; if the determination result is yes, execute 813; otherwise, execute 814;

[0102] 813, determining that the image perspective transformation type is perspective reset;

[0103] 814, determining that the image perspective transformation type is zooming out;

[0104] 815, determining whether the second image is located within the polygon, if the determination result is yes, executing 816, otherwise executing 817 to 819;

[0105] 816, determining that the image perspective transformation type is magnification;

[0106] 817, determining whether the second image and the polygon have an overlapping area, if the determination result is yes, executing 818, otherwise executing 819;

[0107] 818, determining that the image perspective transformation type is rotation;

[0108] 819, determine that the image perspective transformation type is perspective reset.

[0109] The implementation methods of 801 to 819 can refer to the embodiment of the first aspect and will not be described in detail here.

[0110] The above only describes the steps or processes related to the present application, but the present application is not limited to this. The image perspective change type detection method may also include other steps or processes. For the specific content of these steps or processes, reference may be made to the prior art. In addition, the above only uses some structures of the model used in the image perspective change type detection method as an example to illustrate the embodiments of the present application, but the present application is not limited to these structures, and these structures may be appropriately modified, and the implementation methods of these modifications should all be included in the scope of the embodiments of the present application.

[0111] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0112] It can be seen from the above embodiments that a two-step image perspective change type detection process is used to first extract global information, i.e., sampling points, from the image, and then extract local information, i.e., feature points, from the image. The image perspective change type is determined based on the global information and the local information. In this way, the determination of the image perspective change type can be automated and the accuracy of the determination of the image perspective change type can be improved.

[0113] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in an image perspective change type detection device or an electronic device, the program enables a computer to execute the image perspective change type detection method described in the third aspect embodiment in the image perspective change type detection device or electronic device.

[0114] An embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the image perspective change type detection method described in the third aspect embodiment in an image perspective change type detection device or electronic device.

[0115] The image perspective change type detection device or electronic device described in the embodiments of the present application may be directly implemented as hardware, a software module executed by a processor, or a combination of the two. Figure 1 One or more of the functional block diagrams shown in and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to Figures 7 and 8 These hardware modules can be implemented by solidifying these software modules using, for example, a field programmable gate array (FPGA).

[0116] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the electronic device uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0117] against Figure 1 One or more of the functional block diagrams and / or one or more combinations of the functional block diagrams described herein may be implemented as a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in the present application. Figure 1 One or more of the described functional block diagrams and / or one or more combinations of the functional block diagrams may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0118] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the spirit and principles of the present application, and these modifications and variations are also within the scope of the present application.

[0119] This application also provides the following notes:

[0120] 1. A method for detecting image perspective change type, the method comprising:

[0121] extracting a first number of sampling points from the first image;

[0122] determining a proportion of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image;

[0123] When the ratio is greater than or equal to a first threshold, extracting first feature points from the first image and extracting second feature points from the second image;

[0124] An image perspective transformation type is determined according to the first feature point, the second feature point, and a second number of reference points in the first image.

[0125] 2. The method according to Supplement 1, wherein determining the proportion of the sampling points whose positions in the second image of the first number of sampling points undergo a predetermined change relative to their positions in the first image comprises:

[0126] Calculating the optical flow of the first number of sampling points in the second image;

[0127] Determine whether positions of the first number of sampling points in the second image relative to positions in the first image undergo the predetermined change according to the optical flows corresponding to the first number of sampling points, and count the proportion of the sampling points that undergo the predetermined change.

[0128] 3. The method according to Note 1, wherein the method further comprises:

[0129] When the ratio is less than the first threshold and less than the second threshold, it is determined that the image perspective has not changed; when the ratio is less than the first threshold but greater than or equal to the second threshold, it is determined that the image perspective change type is that the image perspective has not changed but the layout of objects in the image has changed.

[0130] 4. The method according to Note 1, wherein the first feature point and the second feature point are extracted using a predetermined feature operator.

[0131] 5. The method according to Note 1, wherein determining the image perspective transformation type comprises:

[0132] Determine a matching feature point pair from the first feature point and the second feature point;

[0133] Determine a homography matrix according to the matched feature point pairs;

[0134] generating a polygon in the second image based on the homography matrix and the reference points;

[0135] An image perspective transformation type is determined based on the polygon and the second image.

[0136] 6. The method according to Note 5, wherein the reference points include at least three vertices in the first image.

[0137] 7. The method according to Note 5, wherein determining the image perspective transformation type based on the polygon and the second image includes: judging whether the polygon is located in the second image, and when judging that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is less than or equal to a third threshold, determining that the image perspective transformation type is perspective reset; and when judging that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is greater than a third threshold, determining that the image perspective transformation type is reduction.

[0138] 8. The method according to Note 7, wherein, when the polygon is not located in the second image but the second image is located in the polygon, the image perspective transformation type is determined to be magnification; when the polygon is not located in the second image and the second image is not located in the polygon, it is determined whether the second image and the polygon have an overlapping area; when there is an overlapping area, the image perspective transformation type is determined to be rotation; when there is no overlapping area, the image perspective transformation type is determined to be perspective reset.

[0139] 9. The method according to Note 1, wherein the first image and the second image are two frames of images in a video stream acquired by the same camera.

[0140] 10. A storage medium storing a computer-readable program, wherein the computer-readable program causes a processor coupled to the storage medium to execute the following method:

[0141] extracting a first number of sampling points from the first image;

[0142] determining a proportion of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image;

[0143] When the ratio is greater than or equal to a first threshold, extracting first feature points from the first image and extracting second feature points from the second image;

[0144] An image perspective transformation type is determined according to the first feature point, the second feature point, and a second number of reference points in the first image.

Claims

1. A device for detecting image perspective change type, characterized in that: The device comprises: A first extraction unit, which extracts a first number of sampling points from the first image; a first determining unit, which determines a proportion of the sampling points of which positions of the first number of sampling points in the second image undergo a predetermined change relative to their positions in the first image; a second extraction unit, which extracts first feature points from the first image and second feature points from the second image when the ratio is greater than or equal to a first threshold; A second determining unit determines an image perspective transformation type according to the first feature point, the second feature point, and a second number of reference points in the first image.

2. The device according to claim 1, wherein: The first determining unit includes: A first calculation module, which calculates the optical flow of the first number of sampling points in the second image; A first determination module determines whether positions of the first number of sampling points in the second image relative to their positions in the first image undergo the predetermined change according to the optical flows corresponding to the first number of sampling points, and counts the proportion of the sampling points that undergo the predetermined change.

3. The device according to claim 1, wherein: The device also includes: A third determination unit determines that the image perspective has not changed when the ratio is less than the first threshold and less than the second threshold, and determines that the image perspective change type is that the image perspective has not changed but the layout of objects in the image has changed when the ratio is less than the first threshold but greater than or equal to the second threshold.

4. The device according to claim 1, wherein: The second extraction unit extracts the first feature points and the second feature points using a predetermined feature operator.

5. The device according to claim 1, wherein: The second determining unit includes: A matching module, which determines a matching feature point pair from the first feature point and the second feature point; A second calculation module, which determines a homography matrix according to the matched feature point pairs; a generating module, which generates a polygon in the second image according to the homography matrix and the reference point; A second determination module determines an image perspective transformation type according to the polygon and the second image.

6. The device according to claim 5, wherein: The reference points include at least three vertices in the first image.

7. The device according to claim 5, wherein: The second determination module determines whether the polygon is located in the second image. When it is determined that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is less than or equal to a third threshold, the image perspective transformation type is determined to be perspective reset. When it is determined that the polygon is located in the second image and the ratio of the area of ​​the polygon to the area of ​​the second image is greater than the third threshold, the image perspective transformation type is determined to be reduction.

8. The device according to claim 7, wherein: The second determination module determines that the image perspective transformation type is magnification when the polygon is not located in the second image but the second image is located in the polygon; determines whether the second image and the polygon have an overlapping area when the polygon is not located in the second image and the second image is not located in the polygon; when there is an overlapping area, determines that the image perspective transformation type is rotation; when there is no overlapping area, determines that the image perspective transformation type is perspective reset.

9. The device according to claim 1, wherein: The first image and the second image are two frames of images in a video stream acquired by the same camera.

10. A method for detecting image perspective change type, characterized in that: The method comprises: extracting a first number of sampling points from the first image; determining a proportion of the first number of sampling points whose positions in the second image undergo a predetermined change relative to their positions in the first image; When the ratio is greater than a first threshold, extracting first feature points from the first image and extracting second feature points from the second image; An image perspective transformation type is determined according to the first feature point, the second feature point, and a second number of reference points in the first image.

Images