A method for identifying a target area, a monitoring method and a monitoring system

Abstract

The present invention discloses a method for identifying a target area, a monitoring method and a monitoring system. Among them, the method for identifying a target area includes the steps of: obtaining a current image at a current shooting angle; identifying a target area in the current image by performing feature matching on the previous frame image and the current image, the previous frame image being an image collected at a previous shooting angle and including the target area; repeating the steps of obtaining the current image and identifying the target area to identify the target areas of images at different shooting angles.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method for identifying a target area, a monitoring method, and a monitoring system. Background Art

[0002] In recent years, the advantages of digital and networked video surveillance systems over traditional closed-circuit television (CCTV) surveillance systems have become increasingly obvious. Their high standardization, openness, integration, and flexibility have provided a broader development space for the development of the entire security industry. Among them, intelligent video surveillance technology uses the powerful data processing capabilities of a computer to perform high-speed analysis on the massive data in video images, filtering out information that users are not interested in and only providing useful key information to the monitor.

[0003] In view of this, intelligent video surveillance has become one of the forefront application development directions in the field of networked video surveillance. Currently, the application fields of intelligent video surveillance systems are very extensive, and they are not only used for security protection in industries such as finance, cultural relics, military, shopping malls, and schools, but also used for safety production and on-site management in industries such as public security, transportation, medical care, airport stations and ports, factories, and substations.

[0004] In existing video surveillance solutions, a video analysis module can perform real-time analysis on the continuous video image content captured by a camera to determine the target area in the image (the target area can be, for example, a key monitoring area). Once an abnormality occurs within the target area, it is determined that an abnormal situation has occurred, and an alarm message is sent to the system via a network. However, in existing video surveillance solutions, the camera can only correctly identify the target area in a fixed state. Once the camera starts to rotate, the previously marked target area cannot be dynamically adjusted to obtain the correct position, resulting in the video analysis module being unable to timely and accurately identify the target area and make an abnormality determination.

[0005] In view of this, a solution that can continuously identify the target area in the monitored area is needed. Summary of the Invention

[0006] To this end, the present invention provides a method for identifying a target area, a monitoring method, and a monitoring system, in an attempt to solve or at least alleviate at least one of the above problems.

[0007] According to one aspect of the present invention, a method for identifying a target area is provided, including the steps of: obtaining a current image at a current shooting angle; identifying a target area in the current image by performing feature matching on the previous frame image and the current image, where the previous frame image is an image collected at a previous shooting angle and the previous frame image contains the target area; repeating the steps of obtaining the current image and identifying the target area to identify the target areas of images at different shooting angles.

[0008] Optionally, the method according to the present invention further includes the step of: displaying the identified target area at a corresponding position of the current image.

[0009] Optionally, the method according to the present invention further includes the steps of: determining, from the current image, pixel points that match the pixel points in the previous frame image as matching point pairs; calculating a homography matrix between the previous frame image and the current image according to the matching point pairs; and determining the position of the target area in the current image based on the homography matrix.

[0010] Optionally, before the step of obtaining the current image at the current shooting angle, the method according to the present invention further includes the steps of: obtaining a previous frame image at a previous shooting angle and obtaining the coordinates of at least one pixel representing the target area from the previous frame image.

[0011] Optionally, the method according to the present invention further includes the steps of: transforming the coordinates of at least one pixel of the target area in the previous frame image based on the homography matrix to obtain the coordinates of the transformed pixel; and determining the position of the target area in the current image according to the coordinates of the transformed pixel.

[0012] Optionally, in the method according to the present invention, at least one pixel is at the edge of the corresponding target area, and / or at least one pixel is a vertex on the edge of the corresponding target area.

[0013] Optionally, the method according to the present invention further includes the step of: presetting a plurality of different shooting angles.

[0014] Optionally, in the method according to the present invention, the time interval between two adjacent shooting angles is fixed, or the difference between two adjacent shooting angles is fixed.

[0015] Optionally, the method according to the present invention further includes the step of: performing feature matching on the current image and the previous frame image to obtain a plurality of matching point pairs.

[0016] According to another aspect of the present invention, a monitoring method is provided, including the steps of: collecting images during the rotation of an image acquisition device; for each frame of the collected images, identifying the target area of each image by executing the method described above, where the target area points to the monitoring area; and displaying the identified target area at the corresponding position of the corresponding image.

[0017] Optionally, in the method according to the present invention, the step of collecting images during the rotation of the image acquisition device includes: collecting one frame of image whenever the image acquisition device rotates to a predetermined angle, or collecting one frame of image at every predetermined time interval during the rotation of the image acquisition device.

[0018] Optionally, the method according to the present invention further includes the step of: displaying monitoring information on the displayed target area.

[0019] According to another aspect of the present invention, a monitoring method is provided, including the steps of: receiving images collected during the rotation of at least one image acquisition device; for the images collected by each image acquisition device, identifying the target area of each image by executing the method of identifying the target area described above; and when the target area of the identified image acquisition device is inconsistent with the monitoring area corresponding to the image acquisition device, controlling other image acquisition devices to replace the image acquisition device to perform image acquisition.

[0020] According to another aspect of the present invention, a monitoring system is further provided, including: an image acquisition device, adapted to collect different images at different shooting angles; a support device, adapted to fix the image acquisition device and rotate the image acquisition device to different shooting angles; a target area recognition device, adapted to recognize the target area in the images at each shooting angle; and a display device, adapted to display the collected images and display the identified target area at the corresponding position of the corresponding image.

[0021] Optionally, in the monitoring system according to the present invention, the target area recognition device includes: a marking unit, adapted to obtain the target area when the image acquisition device collects an initial image at an initial shooting angle; and a feature matching unit, adapted to recognize the target area in the current image by performing feature matching between the current image and the previous frame of image.

[0022] Optionally, in the monitoring system according to the present invention, the image acquisition device is adapted to collect one frame of image when rotating to a predetermined angle; and the image acquisition device is further adapted to collect one frame of image at every predetermined time interval during the rotation.

[0023] According to another aspect of the present invention, there is provided a monitoring system, comprising: at least one image acquisition device adapted to acquire different images at different shooting angles; at least one support device coupled to the at least one image acquisition device, adapted to fix the at least one image acquisition device and rotate the image acquisition device to different shooting angles; a target area recognition device adapted to recognize the target area in the images at each shooting angle; a control device adapted to associate each image acquisition device with its corresponding monitoring area; and a display device adapted to display the acquired images and display the recognized target area at the corresponding position of the corresponding image.

[0024] Optionally, in the monitoring system according to the present invention, the control device is further adapted to control other image acquisition devices to replace the image acquisition device for image acquisition when the target area of the recognized image acquisition device is inconsistent with the monitoring area associated with the image acquisition device.

[0025] According to another aspect of the present invention, there is provided a computing device, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing any of the methods described above.

[0026] According to another aspect of the present invention, there is provided a readable storage medium storing program instructions, which, when read and executed by a computing device, cause the computing device to execute any of the methods described above.

[0027] According to the solution of the present invention, the current image at the current shooting angle is acquired in real time, and then, based on the position of the target area in the previous adjacent frame image, the position of the target area in the current image is recognized in real time. Further, this solution uses the feature matching method to determine the homography matrix of two adjacent frames of images, and then calculates the coordinate positions of the pixels representing the target area in the next frame of image. The calculation is simple and efficient, facilitating real-time dynamic tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To achieve the above and related purposes, certain illustrative aspects are described herein in connection with the following description and drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalent aspects are intended to fall within the scope of the claimed subject matter. The above and other objects, features, and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the drawings. Throughout the present disclosure, the same reference numerals generally refer to the same components or elements.

[0029] Figure 1 FIG. 1 shows a schematic diagram of a monitoring system 100 according to an embodiment of the present invention;

[0030] Figure 2 FIG. 1 shows a schematic diagram of a computing device 200 according to an embodiment of the present invention;

[0031] Figure 3 FIG. 2 shows a schematic flowchart of a method 300 for identifying a target area according to an embodiment of the present invention;

[0032] Figure 4 FIG. 3 shows a schematic diagram of a target area according to an embodiment of the present invention;

[0033] Figure 5 FIG. 4 shows a schematic flowchart of a monitoring method 500 according to another embodiment of the present invention;

[0034] Figure 6 FIG. 5 shows a schematic flowchart of a monitoring method 600 according to still another embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0036] Figure 1 FIG. 6 shows a schematic diagram of a monitoring system 100 according to an embodiment of the present invention. As Figure 1 shown, the monitoring system 100 includes at least one image acquisition device 110, at least one support device 120, a target area identification device 130, and a display device 140. It should be noted that Figure 1 merely for illustration, the embodiments of the present invention do not limit the number of each part included in the system 100.

[0037] In the implementation scenario according to the present invention, the image acquisition devices 110 can be distributed in each area to be monitored to collect the corresponding images from the current perspective. The support device 120 is used to fix the image acquisition device 110 and rotate the image acquisition device 110 to different shooting angles for scanning and monitoring. Usually, the support device 120 and the image acquisition device 110 appear in pairs (as Figure 1 shown). The support device 120 can be, for example, a pan-tilt head, and the pan-tilt head is controlled by a control device to rotate at a constant speed or an accelerated speed to drive the image acquisition device 110 to rotate. In this way, when the image acquisition device 110 rotates to different angles, it can correspondingly collect the images at the shooting angles.

[0038] In one embodiment, a plurality of different shooting angles (for example, 15 degrees, 25 degrees, etc.) are preset as predetermined angles. When the image acquisition device 110 rotates to a predetermined angle, a frame of image is acquired. In another embodiment, the image acquisition device 110 may also acquire a frame of image at every predetermined time interval (such as every 5 seconds) during the rotation process. Alternatively, the image acquisition device 110 may continuously acquire images during the rotation process to obtain a continuous video, which is regarded as a plurality of image frames for processing. The embodiments of the present invention do not impose excessive restrictions on this.

[0039] Whenever a frame of image is acquired, the image acquisition device 110 will send the image to the target area recognition device 130, and the target area recognition device 130 will recognize the target areas in the images at each shooting angle. According to the embodiments of the present invention, the target area refers to the area that needs to be monitored key points. For example, in an urban road monitoring system, the target area may be the illegal parking area beside the road; in another example, in a computer room monitoring system, the target area may be the area where each device is located, and so on. And the number of target areas is not limited to one, and multiple target areas can be set according to the monitoring needs.

[0040] The target area recognition device 130 may be a computing device, including a personal computer configured with a desktop computer and a notebook computer, or a server, etc. According to the embodiments of the present invention, the target area recognition device 130 further includes a marking unit 132 and a feature matching unit 134.

[0041] Among them, the marking unit 132 acquires the target area when the image acquisition device 110 acquires the initial image at the initial shooting angle. Optionally, when the image acquisition device 110 starts shooting and acquires the first frame of image, that is, the initial image, at this time, the marking unit 132 may mark the target area in the initial image through a marking tool. In one embodiment, the target area is marked with the coordinates of at least one pixel. These at least one pixel are usually the pixels on the edge of the target area. In particular, to simplify the calculation, when the target area is a regular quadrilateral, these at least one pixel may be the vertices of the target area edge.

[0042] The feature matching unit 134 identifies the target area in the current image by performing feature matching on the current image and the previous frame image. According to an embodiment of the present invention, the feature matching unit 134 may adopt feature matching algorithms such as SIFT, SURF, ORB, etc. to perform feature matching on two adjacent frame images, and obtain some matching point pairs in the two frame images. It should be noted that the embodiments of the present invention do not impose excessive restrictions on the specific feature matching algorithm adopted. Any known or future-known feature matching algorithm can be combined with the embodiments of the present invention to implement the monitoring solution of the present invention, and all are within the protection scope of the present invention. After that, the feature matching unit 134 calculates the homography matrix between the previous frame image and the current image based on these matching point pairs, and determines the position of the target area in the current image based on the homography matrix. Specifically, the feature matching unit 134 transforms the coordinates of at least one pixel representing the target area in the previous frame image based on the homography matrix, and the coordinates of the transformed pixel represent the position of the target area in the current image.

[0043] After that, the target area recognition device 130 sends the image and the position information of the target area therein to the display device 140. The display device 140 displays the acquired image and displays the recognized target area at the corresponding position of the corresponding image.

[0044] According to other embodiments, in addition to the at least one image acquisition device 110, the at least one support device 120, the target area recognition device 130, and the display device 140 described above, the monitoring system 100 further includes a control device 150( Figure 1 (not shown). The control device 150 is coupled to the at least one image acquisition device 110 to control the image acquisition device 110 to perform image acquisition; at the same time, the control device 150 is also coupled to the target area recognition device 130 to obtain the recognized target area.

[0045] In some implementation scenarios, it is necessary to arrange multiple image acquisition devices 110 to monitor multiple areas in a scene. At this time, at least one image acquisition device 110 can be correspondingly arranged for each area to be monitored. The control device 150 can associate each image acquisition device 110 with its corresponding monitored area. For example, Monitoring Area No. 1 corresponds to Image Acquisition Device A1 and Image Acquisition Device A2, and the examples are not limited thereto. According to one embodiment, when a monitored area corresponds to more than one image acquisition device 110, the control device 150 can also set a main image acquisition device and a slave image acquisition device. Under normal conditions, the main image acquisition device is used to acquire images. When the main image acquisition device fails to acquire images normally, the control device 150 calls the slave image acquisition device to acquire images. According to another embodiment, assuming that there are 5 monitored areas in the system 100 in total, 5 image acquisition devices 110 can be set to correspond to these 5 monitored areas respectively, and several other image acquisition devices 110 are reserved for backup. When these 5 image acquisition devices fail to acquire images normally, the control device 150 then calls an image acquisition device 110 from several other image acquisition devices 110 for replacement and performs the image acquisition operation.

[0046] According to an embodiment of the present invention, the situations where the image acquisition device 110 fails to acquire images normally include, but are not limited to, the image acquisition device 110 deviating from the monitored area, or being scheduled to monitor other areas, or powering off. In short, the control device 150 can obtain the target areas of the identified image acquisition devices from the target area identification device 130. When the target area of the identified image acquisition device is inconsistent with the monitored area associated with the image acquisition device, the control device 150 controls other image acquisition devices 110 to replace the image acquisition device 110 to perform image acquisition.

[0047] Taking the monitoring system 100 being applied to the urban traffic management scenario as an example, the solution for identifying the target area according to the embodiment of the present invention is generally described below.

[0048] In the urban traffic management scenario, the camera needs to identify the illegal area (i.e., the target area), and the illegal area is, for example, the sidewalk. The target area can be circular or polygonal, can be composed of straight lines, or can include curves. The embodiments of the present invention do not limit this. Once it is detected that a vehicle enters the target area, it can be determined that the vehicle has entered the illegal area, and then an alarm message can be sent to the control center and the information of the vehicle can be recorded.

[0049] According to an embodiment of the present invention, the image acquisition device 110 (such as a camera) and the support device 120 (such as a pan-tilt head) are turned on, and the first frame of image is acquired at the current shooting angle. Meanwhile, the position of the target area is marked in the first frame of image. In the embodiment of the present invention, the position of the target area can be marked by a marking tool, or the position of the target area can be identified by an image processing algorithm. The embodiments of the present invention do not limit this.

[0050] After that, during the rotation of the image acquisition device 110, a frame of image at the corresponding shooting angle is acquired according to a predetermined rule and sent to the target area recognition device 130. The predetermined rule is, for example, to acquire a frame of image every few seconds, or to acquire a frame of image every time it rotates a certain angle, or to acquire a frame of image when it rotates to a fixed angle, and is not limited thereto.

[0051] The target area recognition device 130 calculates the homography matrix between two adjacent frames of images by performing feature matching on the two adjacent frames of images. Then, according to the homography matrix and the position of the target area in the previous frame of image, the position of the target area in the subsequent frame of image is calculated. For example, when the second frame of image is acquired, the homography matrix of the first frame of image and the second frame of image is calculated, and the position of the target area in the second frame of image is determined by using the homography matrix and the position of the target area in the first frame of image; similarly, when the third frame of image is acquired, the homography matrix of the second frame of image and the third frame of image is calculated, and the position of the target area in the third frame of image is determined by using the homography matrix and the position of the target area in the second frame of image; and so on, until the target areas in the images acquired at each shooting angle are determined.

[0052] Meanwhile, the target area recognition device 130 can also be coupled to the background display device 140. When the target area of each frame of image is determined, the target area recognition device 130 sends the image and the position information of the target area in the image to the display device 140, which displays the frame of image and displays the recognized target area at the corresponding position of the frame of image for the background monitoring personnel to view in real time.

[0053] In addition, the target area recognition device 130 can also be coupled to the backend processing device. When the target area of each frame of image is determined, the image and the position information of the target area in the image are sent to the backend processing device so that the backend processing device can detect in real time whether a vehicle enters the target area.

[0054] According to the monitoring system 100 of the present invention, during the rotation of the image acquisition device 110, the target area in the acquired image can be accurately determined through feature matching, realizing dynamic following of the target area.

[0055] It should be noted that in some other embodiments according to the present invention, when the image acquisition device 110 (such as a camera) has sufficient storage space and computing power, the target area recognition device 130 can also be implemented as the image acquisition device 110 itself.

[0056] According to an embodiment of the present invention, the monitoring system 100 and each component thereof can be implemented by the computing device 200 as described below. Figure 2 FIG. shows a schematic diagram of a computing device 200 according to an embodiment of the present invention.

[0057] As Figure 2 shown, in the basic configuration 202, the computing device 200 typically includes a system memory 206 and one or more processors 204. A memory bus 208 can be used for communication between the processor 204 and the system memory 206.

[0058] Depending on the desired configuration, the processor 204 can be any type of processing, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. The processor 204 can include one or more levels of cache such as a level 1 cache 210 and a level 2 cache 212, a processor core 214, and registers 216. An example processor core 214 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 218 can be used with the processor 204, or in some implementations, the memory controller 218 can be an internal part of the processor 204.

[0059] Depending on the desired configuration, the system memory 206 can be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. The system memory 206 can include an operating system 220, one or more applications 222, and program data 224. In some embodiments, the applications 222 can be arranged to execute instructions on the operating system by one or more processors 204 using the program data 224.

[0060] The computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output device 242, peripheral interface 244, and communication device 246) to the basic configuration 202 via the bus / interface controller 230. Example output devices 242 include a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 252. Example peripheral interfaces 244 may include a serial interface controller 254 and a parallel interface controller 256, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I / O ports 258. Example communication device 246 may include a network controller 260, which may be arranged to facilitate communication with one or more other computing devices 262 via one or more communication ports 264 through a network communication link.

[0061] The network communication link may be an example of a communication medium. A communication medium generally may embody computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery medium. A "modulated data signal" may be a signal that one or more of its data sets or its changes can encode information in the way of the signal. As a non-limiting example, the communication medium may include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR), or other wireless media. The term computer-readable medium as used herein may include both storage media and communication media.

[0062] The computing device 200 may be implemented as a server, such as a file server, a database server, an application server, and a WEB server, etc., or may be implemented as a personal computer including a desktop computer and a laptop computer configuration. Of course, the computing device 200 may also be implemented as part of a small-sized portable (or mobile) electronic device. In an embodiment according to the present invention, the computing device 200 is configured to execute the method for identifying a target area and / or the monitoring method according to the present invention. Multiple program instructions for executing these methods are included in the application 222 of the computing device 200.

[0063] Figure 3 A flowchart of a method 300 for identifying a target area according to an embodiment of the present invention is shown. The method 300 is executed in the above system 100, especially in the target area identification device 130, as Figure 3 described, the method 300 begins with step S310.

[0064] In step S310, a current image is acquired at the current shooting angle.

[0065] According to an embodiment of the present invention, during the rotation of the image acquisition device 110, every time a shooting angle is reached, the image acquired at the current shooting angle is sent to the target area recognition device 130. The target area recognition device 130 is used to recognize the target area in the image.

[0066] In addition, according to an embodiment of the present invention, before step S310, method 300 further includes the steps of: acquiring a previous frame of image at the previous shooting angle, and acquiring the coordinates of at least one pixel used to characterize the target area from the previous frame of image.

[0067] As described in system 100, when starting image acquisition, the image acquisition device 110 first acquires an image at the initial shooting angle as the first frame of image, and marks the target area in the image. For the images acquired during the subsequent image acquisition process, the position of the target area in the current image is determined based on the position of the target area in the previous frame of image.

[0068] In an embodiment of the present invention, the target area refers to a key area to be monitored, such as a jewelry counter in a shopping mall, an illegal parking area on an urban road, an area where equipment is placed in a substation, an entrance and exit area of a station or port, etc., which is not limited thereto. The target area can be marked in the first frame of image according to the actual application scenario.

[0069] In addition, regarding the shooting angle, in an embodiment of the present invention, there can be various setting methods.

[0070] In one embodiment, method 300 further includes the steps of: presetting a plurality of different shooting angles. For example, setting 5 degrees, 10 degrees, 15 degrees, etc. as the preset angles, and the shooting angle corresponding to when the image acquisition device 110 starts to rotate is 0 degrees. Then, every time the image acquisition device 110 rotates to a preset angle, a frame of image is acquired.

[0071] In another embodiment, the time interval between two adjacent shooting angles is fixed. For example, during the rotation of the image acquisition device 110, a frame of image is acquired every certain time. This time interval can be set according to the image acquisition requirements and the rotation angle of the pan-tilt head. If a large number of images need to be acquired, the time interval is reduced, which is not limited thereto.

[0072] In yet another embodiment, the difference between two adjacent shooting angles is fixed. For example, during the rotation of the image acquisition device 110, an image is acquired every 10 degrees of rotation, that is, the difference between two adjacent shooting angles is 10 degrees. Similarly, the difference between two adjacent shooting angles can be set according to the image acquisition requirements and the rotation angle of the pan-tilt head. Assuming that the rotation angle of the pan-tilt head is from 0 degrees to 300 degrees and 20 images are required during one rotation acquisition process, then it can be set to acquire an image every 15 degrees of rotation.

[0073] In still some other embodiments, it can also be set that during the rotation of the image acquisition device 110, images are continuously acquired to obtain a video, which contains multiple frames of images, and these images can be processed.

[0074] Subsequently, in step S320, by performing feature matching on the previous frame image and the current image, a target area is identified in the current image. As described above, the previous frame image is the image acquired at the previous shooting angle, and the previous frame image contains the target area.

[0075] According to the embodiment of the present invention, step S320 can be executed in the following three steps.

[0076] First step, determine the pixel points in the current image that match the pixel points in the previous frame image as matching point pairs. Specifically, perform feature matching on the current image and the previous frame image to obtain multiple matching point pairs. In accordance with the embodiments of the present invention, feature matching algorithms such as SIFT, SURF, ORB, etc. can be used to perform feature matching on the current image and the previous frame image, so as to determine some matching pixel points in the current image and the previous frame image to form matching point pairs. Generally, the matching points extracted from the image are pixel points where the image gray value changes drastically, or points with a large curvature on the image edge (i.e., the intersection of two edges).

[0077] Second step, calculate the homography matrix between the previous frame image and the current image according to the matching point pairs.

[0078] Briefly speaking, in computer vision, the homography of a plane is defined as the projection mapping from one plane to another plane. Therefore, the mapping of a point on a two-dimensional plane to the camera imager is an example of plane homography. If the homogeneous coordinates are used to map a point P on the calibration board to the point m on the imager, this mapping can be represented by the homography matrix. Generally, when there are 4 pairs of matching point pairs between two frames of images, the homography matrix between the two frames of images can be calculated. The calculation of the homography matrix is known in the art, so it will not be elaborated here.

[0079] Third step, based on the homography matrix, determine the position of the target area in the current image.

[0080] In one embodiment, based on the homography matrix, the coordinates of at least one pixel in the target region of the previous frame image are transformed to obtain the coordinates of the transformed pixel; then, according to the coordinates of the transformed pixel, the position of the target region in the current image is determined.

[0081] Briefly, assume that in the previous frame image, the coordinate of the i-th pixel in the target region is X i , and the homography matrix is represented as H, then the coordinate Y i of the pixel corresponding to the i-th pixel in the current image can be expressed as:

[0082] Y i = HX i .

[0083] After performing the above transformation on all the pixels representing the target region in the previous frame image, the coordinates of all the pixels representing the target region in the current image are obtained. According to the coordinates of these pixels, the position of the target region can be determined.

[0084] Generally, at least one pixel representing the target region is at the edge of the corresponding target region. Figure 4 FIG. shows a schematic diagram of a target region according to an embodiment of the present invention. As Figure 4 shown, in an urban road, roadside parking spaces are provided on both sides of the main road 410 for vehicles to park. Trees are planted on both sides. Among them, the road (i.e., the sidewalk) where trees are planted at the back is the target region 420 and parking is prohibited. The pixels on the edges AB, BD, DC, and CA of the target region 420 are used to represent the target region 420.

[0085] In still other embodiments, to simplify the calculation, the vertices on the edge of the corresponding target region are taken as the pixels representing the target region to reduce the amount of calculation. Continuing as Figure 4 , the pixels A, B, C, and D can also be taken to represent the target region 420.

[0086] Subsequently, in step S330, the steps of repeatedly obtaining the current image (i.e., step S310) and identifying the target region (i.e., step S320) are performed to identify the target regions of images at different shooting angles.

[0087] According to one implementation manner, during the execution of step S330, when the target region of each frame of image is identified, the identified target region can also be displayed at the corresponding position of the current image for the background monitoring personnel to view in real time.

[0088] According to method 300 of the present invention, the current image at the current shooting angle is acquired in real time. Then, based on the position of the target area in the previous adjacent frame image, the position of the target area in the current image is identified in real time. In addition, this solution uses feature matching to determine the homography matrix between two adjacent frames of images, and then calculates the coordinate positions of the pixels representing the target area in the next frame of image. The calculation is simple and efficient, facilitating real-time dynamic tracking.

[0089] Figure 5 Fig. shows a schematic flowchart of a monitoring method 500 according to another embodiment of the present invention. Figure 5 The shown method 500 is suitable for execution in the monitoring system 100 and is a further illustration of Figure 3 the shown method 300.

[0090] Method 500 starts with step S510, and images are acquired during the rotation of the image acquisition device 110.

[0091] For example, whenever the image acquisition device 110 rotates to a predetermined angle, a frame of image is acquired. Another example is that during the rotation of the image acquisition device 110, a frame of image is acquired every predetermined time. It is not limited to this.

[0092] Subsequently, in step S520, for each frame of the acquired images, by executing the above-mentioned method 300, the target areas of each image are identified. As described above, the target area refers to the monitoring area. The embodiments of the present invention do not limit the shape, quantity, etc. of the target area.

[0093] Subsequently, in step S530, the identified target areas are displayed at the corresponding positions of the corresponding images.

[0094] In addition, corresponding monitoring information can also be displayed on the displayed target areas. The monitoring information can be whether there is an abnormality in the target area, the environmental information in the target area, etc. For example, in the urban traffic management scenario, the displayed monitoring information is whether a vehicle has entered this area. Another example is that in the school cafeteria management scenario, the displayed monitoring information is information such as temperature, humidity, and sanitation grade in this area. Providing these monitoring information in real time on the target area facilitates the background monitoring personnel to quickly grasp the situation in the monitoring area.

[0095] It should be understood that for the specific description of method 500, reference can be made to the relevant descriptions of system 100 and method 300 above. Due to space limitations, it will not be elaborated here.

[0096] Based on method 500, according to some other embodiments of the present invention, another monitoring method 600 is also provided. Figure 6 Fig. shows a schematic flowchart of a monitoring method 600 according to another embodiment of the present invention.Figure 6 The illustrated method 600 is suitable for execution in the monitoring system 100 .

[0097] The method 600 begins at step S610 , where an image captured by at least one image capturing device 110 during rotation is received.

[0098] Then in step S620, the target area of each image is identified by executing the method 300 as described above for the images captured by each image capture device.

[0099] Then in step S630, when the target area of the identified image acquisition device 110 is inconsistent with the monitoring area corresponding to the image acquisition device, other image acquisition devices 110 are controlled to replace the image acquisition device to perform image acquisition.

[0100] It should be understood that method 600 and method 500 complement each other. For a detailed description of method 600, reference may be made to the foregoing description of system 100, method 300, and method 500. Due to space limitations, they will not be elaborated herein.

[0101] According to the monitoring scheme of the present invention, during the rotation of the image acquisition device, the target area in the acquired image can be accurately determined through feature matching, thereby achieving dynamic tracking of the target area. In addition, multiple image acquisition devices in the scene can work in conjunction. When one image acquisition device cannot acquire images normally, other image acquisition devices can be controlled to fill in the gaps in real time to ensure the accuracy of monitoring.

[0102] The following takes the equipment monitoring scenario of a power station as an example to illustrate a solution for identifying a target area according to an embodiment of the present invention.

[0103] In a computer room where multiple devices are arranged, the surveillance camera needs to identify each device (i.e., the target area). At this time, the shape of the target area can be consistent with the shape of the device, such as square, circle, etc., and the embodiment of the present invention does not limit this. Based on this solution, in the process of scanning and collecting images by the camera, each device in the collected image can be identified in real time, and the area where each device is located is displayed to the control center in the background for real-time viewing by the staff.

[0104] According to an embodiment of the present invention, the image acquisition device 110 (such as a camera) and the support device 120 (such as a pan / tilt) are turned on, and the first frame of image is acquired at the current shooting angle. At the same time, the position of the target area is marked in the first frame of image (optionally, assuming that there are 5 devices to be monitored, 5 target areas are marked). In an embodiment of the present invention, the position of the target area can be marked by a marking tool, or the position of the target area can be identified by an image processing algorithm, and the embodiment of the present invention does not limit this.

[0105] After that, during the rotation of the image acquisition device 110, a frame of image at a corresponding shooting angle is acquired according to a predetermined rule and sent to the target area recognition device 130. The predetermined rule can be, for example, acquiring one frame of image every few seconds, or acquiring one frame of image every certain angle of rotation, or acquiring one frame of image when rotating to a fixed angle, and is not limited thereto.

[0106] The target area recognition device 130 calculates the homography matrix between two adjacent frames of images by performing feature matching on the two adjacent frames of images. Then, according to the homography matrix and the positions of the target areas in the previous frame of image, the positions of the target areas in the subsequent frame of image are calculated respectively. For example, when the second frame of image is acquired, the homography matrix between the first frame of image and the second frame of image is calculated, and the positions of the target areas in the second frame of image are respectively determined by using the homography matrix and the positions of the target areas in the first frame of image; similarly, when the third frame of image is acquired, the homography matrix between the second frame of image and the third frame of image is calculated, and the positions of the target areas in the third frame of image are respectively determined by using the homography matrix and the positions of the target areas in the second frame of image; and so on, until the target areas in the images acquired at each shooting angle are determined.

[0107] As described above, the target area recognition device 130 can also be coupled to the background display device 140. When the target area of each frame of image is determined, the target area recognition device 130 sends the image and the position information of the target area in the image to the display device 140 together. The display device 140 displays the frame of image and displays the recognized target area at the corresponding position of the frame of image for the background monitoring personnel to view in real time.

[0108] In addition, the target area recognition device 130 can also be coupled to the back-end processing system. The back-end processing system correspondingly obtains monitoring data such as the status information and sensor information of each device at each shooting angle, and when the target area of each frame of image is determined, the monitoring data of the device is synchronously displayed at the position of the target area corresponding to the device.

[0109] The various technologies described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the method and device of the present invention, or certain aspects or parts of the method and device of the present invention, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium. When the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present invention.

[0110] When the program code is executed on a programmable computer, a computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memories and / or storage elements), at least one input device, and at least one output device. Among them, the memory is configured to store the program code; the processor is configured to execute the method of the present invention according to the instructions in the program code stored in the memory.

[0111] By way of example, and not limitation, a readable medium includes a readable storage medium and a communication medium. The readable storage medium stores information such as computer-readable instructions, data structures, program modules, or other data. The communication medium generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and includes any information delivery medium. A combination of any of the above is also included within the scope of the readable medium.

[0112] In the specification provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other device. A variety of general-purpose systems may also be used with the examples of the present invention. Based on the above description, the structure required to construct such a system is obvious. In addition, the present invention is not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using a variety of programming languages, and the description of a particular language above is for the purpose of disclosing the best mode of the present invention.

[0113] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0114] Similarly, it should be understood that, in order to streamline the present disclosure and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, where each claim stands on its own as a separate embodiment of the present invention.

[0115] Those skilled in the art should understand that the modules or units or components of the devices in the examples disclosed herein can be arranged in the devices as described in the embodiments, or alternatively can be located in one or more devices different from the devices in the examples. The modules in the foregoing examples can be combined into one module or further can be divided into multiple sub-modules.

[0116] Those skilled in the art can understand that the modules in the devices of the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be adopted to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or device thus disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0117] In addition, those skilled in the art can understand that although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0118] In addition, some of the embodiments herein are described as combinations of methods or method elements that can be implemented by a processor of a computer system or by other devices performing the functions. Therefore, a processor having the necessary instructions for implementing the method or method element forms a device for implementing the method or method element. In addition, the elements described herein in the device embodiments are examples of the following devices: the device is used to implement the functions performed by the elements for the purpose of implementing the invention.

[0119] As used herein, unless otherwise specified, the use of ordinal numbers "first", "second", "third", etc. to describe ordinary objects only indicates different instances of similar objects, and does not intend to imply that the objects so described must have a given order in terms of time, space, sorting or in any other way.

[0120] Although the present invention has been described in terms of a limited number of embodiments, those skilled in the art will appreciate, in light of the above description, that other embodiments can be contemplated within the scope of the invention as thus described. Additionally, it should be noted that the language used in this specification has been principally selected for readability and instructional purposes, rather than for the purpose of interpreting or limiting the subject matter of the invention. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. For the scope of the present invention, the disclosure made herein is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for identifying a target area, comprising the steps of: Obtaining a current image at a current shooting angle; Identifying a target area in the current image by performing feature matching on the previous frame image and the current image, where the previous frame image is an image collected at a previous shooting angle and the previous frame image contains the target area; Repeating the steps of obtaining the current image and identifying the target area to identify the target areas of images at different shooting angles; Displaying the identified target area at a corresponding position in the current image and displaying corresponding monitoring information on the displayed target area; Wherein, The step of identifying a target area in the current image by performing feature matching on the previous frame image and the current image includes: Determining, from the current image, pixel points that match the pixel points in the previous frame image as matching point pairs; calculating a homography matrix of the previous frame image and the current image based on the matching point pairs; Based on the homography matrix, determining the position of the target area in the current image, including: transforming the coordinates of at least one pixel of the target area in the previous frame image based on the homography matrix to obtain the coordinates of the transformed pixel; determining the position of the target area in the current image based on the coordinates of the transformed pixel.

2. The method according to claim 1, wherein Before the step of obtaining a current image at a current shooting angle, the method further includes the steps of: Obtaining a previous frame image at a previous shooting angle; Obtaining the coordinates of at least one pixel representing the target area from the previous frame image.

3. The method according to claim 1, wherein The at least one pixel is at the edge of the corresponding target area.

4. The method according to claim 3, wherein The at least one pixel is a vertex on the edge of the corresponding target area.

5. The method according to claim 1, further comprising the step of: Pre-setting a plurality of different shooting angles.

6. The method according to claim 1, wherein, The time interval between two adjacent shooting angles is fixed.

7. The method according to claim 1, wherein The difference between two adjacent shooting angles is fixed.

8. The method according to claim 1, wherein, The step of determining, from the current image, pixel points that match the pixel points in the previous frame image as matching point pairs includes: Performing feature matching on the current image and the previous frame image to obtain a plurality of matching point pairs.

9. A monitoring method, comprising the steps of: Collecting images during the rotation of an image acquisition device; For each frame of the collected images, identifying the target areas of each image by executing the method according to any one of claims 1-8, where the target area points to a monitoring area; Displaying the identified target area at a corresponding position in the corresponding image and displaying corresponding monitoring information on the displayed target area.

10. The method according to claim 9, wherein, The step of collecting images during the rotation of the image acquisition device includes: Collecting one frame of image whenever the image acquisition device rotates to a predetermined angle.

11. The method according to claim 9, wherein, The step of collecting images during the rotation of the image acquisition device includes: Collecting one frame of image at every predetermined time during the rotation of the image acquisition device.

12. The method according to claim 9, wherein, The step of displaying the identified target area at a corresponding position in the corresponding image further includes: Displaying monitoring information on the displayed target area.

13. A monitoring method, comprising the steps of: Receive images collected by at least one image acquisition device during rotation; For the images collected by each image acquisition device, identify the target area of each image by performing the method described in any one of claims 1-8; When the target area of the identified image acquisition device is inconsistent with the monitoring area corresponding to the image acquisition device, control other image acquisition devices to replace the image acquisition device to perform image acquisition.

14. A monitoring system, comprising: An image acquisition device, adapted to collect different images at different shooting angles; A support device, adapted to fix the image acquisition device and rotate the image acquisition device to different shooting angles; A target area recognition device, adapted to identify the target area in the images at each shooting angle; A display device, adapted to display the collected images, display the identified target area at the corresponding position of the corresponding image, and display corresponding monitoring information on the displayed target area; Wherein, the being adapted to identify the target area in the images at each shooting angle includes: identifying the target area in the current image by performing feature matching between the previous frame of image and the current image, the previous frame of image being the image collected at the previous shooting angle and including the target area; Wherein, the step of identifying the target area in the current image by performing feature matching between the previous frame of image and the current image includes: Determine the pixel points in the current image that match the pixel points in the previous frame of image as matching point pairs; calculate the homography matrix between the previous frame of image and the current image according to the matching point pairs; Based on the homography matrix, determine the position of the target area in the current image, including: based on the homography matrix, transform the coordinates of at least one pixel of the target area in the previous frame of image to obtain the coordinates of the transformed pixel; determine the position of the target area in the current image according to the coordinates of the transformed pixel.

15. The monitoring system according to claim 14, wherein, The target area recognition device includes: A marking unit, adapted to obtain the target area when the image acquisition device collects the initial image at the initial shooting angle; A feature matching unit, adapted to identify the target area in the current image by performing feature matching between the current image and the previous frame of image.

16. The monitoring system according to claim 14, wherein, The image acquisition device is adapted to collect a frame of image when rotated to a predetermined angle.

17. The monitoring system according to claim 14, wherein, The image acquisition device is adapted to collect a frame of image every predetermined time during rotation.

18. A monitoring system, comprising: At least one image acquisition device, adapted to collect different images at different shooting angles; At least one support device coupled to the at least one image acquisition device, adapted to fix the at least one image acquisition device and rotate the image acquisition device to different shooting angles; A target area recognition device, adapted to identify the target area in the images at each shooting angle; A control device, adapted to associate each image acquisition device with its corresponding monitoring area; A display device, adapted to display the acquired images, and display the identified target area at the corresponding position of the corresponding image, and display the corresponding monitoring information on the displayed target area; Wherein, the adapted to identify the target area in the images at each shooting angle includes: identifying the target area in the current image by performing feature matching on the previous frame image and the current image, the previous frame image being the image acquired at the previous shooting angle and including the target area; Wherein, the step of identifying the target area in the current image by performing feature matching on the previous frame image and the current image includes: Determining the pixel points in the current image that match the pixel points in the previous frame image as matching point pairs; calculating the homography matrix of the previous frame image and the current image according to the matching point pairs; Based on the homography matrix, determining the position of the target area in the current image includes: based on the homography matrix, transforming the coordinates of at least one pixel of the target area in the previous frame image to obtain the coordinates of the transformed pixel; determining the position of the target area in the current image according to the coordinates of the transformed pixel.

19. The monitoring system according to claim 18, wherein, The control device is further adapted to control other image acquisition devices to replace the image acquisition device to perform image acquisition when the target area of the identified image acquisition device is inconsistent with the monitoring area associated with the image acquisition device.

20. A computing device, comprising: At least one processor; And A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1-13.

21. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, causing the computing device to execute the method according to any one of claims 1-13.

Images