Method and system for testing video analytics of video surveillance system

Abstract

A simulated object for display on a video stream is generated by receiving a first user input identifying one or more user identification areas with respect to an image, tagging each of the one or more user identification areas with a corresponding area tag, a second user input identifying one or more simulation parameters is received, and the one or more simulation parameters are identified based at least in part on the second user input. The method includes determining a plurality of characteristics of the simulated objects based on the simulation parameters, the user identification regions, and the region tags. The simulation objects are superimposed on the video stream to create an enhanced video stream, and the enhanced video stream is processed using one or more video analysis algorithms to test the validity of each of the one or more video analysis algorithms.

Description

Technical Field

[0001] The present disclosure generally relates to video surveillance systems. More specifically, the present disclosure relates to testing video analytics for video surveillance systems. Background Art

[0002] Many video surveillance systems employ cameras mounted or otherwise arranged around a surveillance area, such as a city, a portion of a city, a facility, or a building. Video surveillance systems may also include mobile cameras, such as drones carrying cameras. Using video analytics, video surveillance systems may employ a variety of different detection algorithms. What is desired is an improved method for testing video analytics for video surveillance systems. Summary of the invention

[0003] The present disclosure relates generally to video surveillance systems, and more specifically to testing video analytics for video surveillance systems. Examples may be found in a computer-implemented method for testing one or more video analytics algorithms for a video surveillance system. The video surveillance system includes a camera having a field of view (FOV), the field of view covering a portion of a facility (or other venue). An exemplary method includes generating a simulated object to be superimposed on a video stream captured by the camera, wherein generating the simulated object includes displaying an image captured by the camera on a display, receiving a first user input identifying one or more user-identified regions relative to the image, marking each of the one or more user-identified regions with a corresponding region label, receiving a second user input for identifying one or more simulation parameters, and identifying the one or more simulation parameters based at least in part on the second user input. Based at least in part on the one or more simulation parameters and based on one or more of the user-identified regions and the corresponding region labels, the method includes determining a plurality of characteristics of the simulated object, including one or more of the following: a number of simulated objects in the FOV, a distribution of simulated objects in the FOV, a starting point for each of the simulated objects in the FOV, and a movement of each of the simulated objects in the FOV. The method includes superimposing a simulated object on the FOV of a video stream captured by a camera to create an enhanced video stream, and processing the enhanced video stream using one or more video analytics algorithms to test the effectiveness of each of the one or more video analytics algorithms.

[0004] Another example may be found in a computer-implemented method for testing one or more video analytics algorithms of a video surveillance system, wherein the video surveillance system includes a camera having a field of view (FOV), the field of view covering a portion of a facility (or other area). The exemplary method includes generating a simulated object to be superimposed on a video stream captured by the camera, wherein generating the simulated object includes receiving user input identifying a particular video analytics algorithm from a plurality of video analytics algorithms, and determining a plurality of characteristics of the simulated object based at least in part on the particular video analytics algorithm identified by the user input, including one or more of the following: the number of simulated objects in the FOV, the distribution of simulated objects in the FOV, the origin of each of the simulated objects in the FOV, and the movement of each of the simulated objects in the FOV. The method includes superimposing the simulated object on the FOV of a video stream captured by the camera to create an enhanced video stream, and processing the enhanced video stream using the particular video analytics algorithm identified by the user input to test the effectiveness of the particular video analytics algorithm.

[0005] Another example may be found in a computer-implemented method for testing a video surveillance system that includes a camera having a field of view (FOV) that encompasses an area of ​​a facility (or other area). An illustrative method includes generating a simulated object to be superimposed on a video stream captured by the camera, wherein generating the simulated object includes receiving a speech- or text-based description of one or more scenes to be tested from a user. Based at least in part on the speech- or text-based description of the one or more scenes to be tested, the method includes determining a plurality of characteristics of the simulated object, including one or more of: the number of simulated objects in the FOV, the distribution of the simulated objects in the FOV, the origin of each of the simulated objects in the FOV, and the movement of each of the simulated objects in the FOV. The method includes superimposing the simulated object on the FOV of the video stream captured by the camera to create an augmented video stream, and processing the augmented video stream to test one or more scenes described in the speech- or text-based description.

[0006] The above summary is provided to facilitate understanding of some innovative features unique to the present disclosure, but is not intended to be a complete description. A comprehensive understanding of the present disclosure can be obtained by viewing the entire specification, claims, drawings and abstract as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present disclosure may be more fully understood by considering the following description of various examples in conjunction with the accompanying drawings, in which:

[0008] Figure 1 is a schematic block diagram of an exemplary video surveillance system;

[0009] Figure 2is a schematic block diagram of an illustrative method for installing and configuring a video surveillance system in the field;

[0010] Figure 3 is a schematic block diagram of an illustrative method of configuring a camera of a video surveillance system;

[0011] Figure 4 is a schematic block diagram of an illustrative method of configuring a video surveillance system, the method comprising testing one or more video analysis algorithms of the video surveillance system;

[0012] Figure 5 is a schematic block diagram of an illustrative method for overlaying a simulated object onto a video stream to create an enhanced video stream;

[0013] Fig. 6A and Figure 6B are flowcharts that collectively illustrate an illustrative computer-implemented method;

[0014] Fig. 7A and Figure 7B are flowcharts that collectively illustrate an illustrative computer-implemented method;

[0015] Fig. 8A and Figure 8B are flowcharts that collectively illustrate an illustrative computer-implemented method;

[0016] Fig. 9 is a schematic block diagram of an exemplary object simulation module; and

[0017] Fig.10 is a schematic block diagram of an illustrative method for generating object movement of a video stream to create an enhanced video stream.

[0018] Although the present disclosure is subject to various modifications and alternative forms, the details thereof have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the present disclosure to the specific examples described. On the contrary, it is intended to cover all modifications, equivalents and alternatives that fall within the essence and scope of the present disclosure. DETAILED DESCRIPTION

[0019] The following description should be read with reference to the accompanying drawings, in which similar elements in different drawings are numbered in the same manner. The accompanying drawings are not necessarily drawn to scale, and they depict examples that are not intended to limit the scope of the present disclosure. Although examples of various elements are shown, those skilled in the art will recognize that many of the examples provided have suitable alternatives that can be utilized.

[0020] It is assumed herein that all numbers are modified by the term "about" unless the content clearly dictates otherwise. The recitation of numerical ranges by endpoints includes all numbers subsumed within that range (eg, 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).

[0021] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally employed in its sense including "and / or" unless the content clearly dictates otherwise.

[0022] It should be noted that references to "one embodiment", "some embodiments", "other embodiments", etc. in the specification indicate that the described embodiments may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is contemplated that the feature, structure, or characteristic may be applied to other embodiments, whether or not explicitly described, unless otherwise explicitly stated to the contrary.

[0023] Figure 1 1 is a schematic block diagram of an exemplary video surveillance system 10. The exemplary video surveillance system 10 may include a camera 12. Although a single camera 12 is shown, it should be understood that the video surveillance system 10 may include a large number of cameras 12. The camera 12 has a field of view (FOV) 14 therein. For example, the FOV 14 encompasses what the camera 12 can see, or a physical area or field that can be seen within a video stream generated by the camera 12. The video surveillance system 10 includes a controller 16 operably coupled to the camera 12 such that the video stream generated by the camera 12 can be sent to the controller 16.

[0024] The controller 16 is operably coupled to the memory 18. For example, the memory 18 may be used to store recorded video streams and video clips. The memory 18 may be used to store video analysis algorithms 20. For example, the video analysis algorithms 20 may be used by the controller 16 when analyzing video streams or recorded video clips. The video analysis algorithms 20 may include any of a variety of different video analysis algorithms 20, such as, but not limited to, one or more of the following: crowd detection algorithms (e.g., crowd presence or formation), crowd analysis algorithms (e.g., crowd size, rate of crowd formation or dispersion, density of crowd, movement within crowd, movement of crowd, etc.), people counting algorithms (e.g., number of people in a defined target area of ​​FOV), behavior detection algorithms (e.g., running, walking, loitering, yelling, shoplifting, protesting, vandalizing property, tailing through a secure entry point, wearing a mask, social distance, etc.), intrusion detection algorithms (e.g., entering a secure area), perimeter protection algorithms (e.g., crossing a secure perimeter), and tailing detection algorithms (e.g., tailing through a secure entry point). These are just examples.

[0025] The controller 16 is operably coupled to a user interface 22. For example, the user interface 22 includes a display 24 that can be used to display a video stream. The user interface 22 may also include data input capabilities, such as a keyboard and mouse, that a user can use when entering information into the controller 16. In some cases, the user interface 22 may include a voice and / or text receiving device for receiving natural language input (sometimes from a remote location). In some cases, at least some of the controller 16, memory 18, and user interface 22 may appear in a computer (such as a PC or laptop). In some cases, the controller 16 and / or memory 18 may appear in a computer server (such as a cloud-based server). The controller 16 and / or memory 18 may represent a portion of a distributed computer system, with a portion of the distributed computer being set locally and a portion of the distributed computer being set remotely.

[0026] Figure 22 is a schematic block diagram of an exemplary method 26 for installing and configuring a video surveillance system 10 in the field. As shown in box 28, the video surveillance system 10 can be installed. This can include installing a VMS (video management system). This can include installing an NVR (network video recorder), which is a dedicated computer configured to record security video surveillance in a digital format to a hard drive or other large storage device. As shown in box 30, the cameras within the video surveillance system 10 can be installed and configured. In some cases, configuring the camera can include receiving input from a simulation configuration module 32. As will be discussed, in some cases, it may be beneficial to include simulated objects in the video stream for testing specific video analysis algorithms, especially when the video stream is directed to a largely unoccupied portion of the facility and is therefore not expected to include actual objects that can be used to test the desired video analysis algorithm. For example, the camera can be installed in a large warehouse that is mostly unoccupied for a long period of time. In some cases, adding simulated objects to the video stream can provide something for a specific video analysis algorithm to look for. Once the simulated objects are added, the video analysis can be run, as shown in box 34. If the simulated object is detected, an alarm can be generated, as shown in box 36.

[0027] Figure 3 Yes Figure 2 30 of the video surveillance system 10. In some cases, the illustrated method may be implemented by the controller 16 of the video surveillance system 10. A camera may be selected for configuration, as indicated by block 40. A variety of different video analysis algorithms (each using various specified parameter values) may be used to analyze the video stream from the selected camera. In some cases, these parameters may be assigned and defined, as indicated by block 42. Simulated objects may then be added to the video stream of the selected camera, and a visual check may be performed to see if the video analysis algorithm is working correctly, as indicated by block 44. Once the installer is confident that the video analysis algorithm is working correctly for the selected camera, simulated objects may no longer be added to the video stream of the selected camera, and the video analysis algorithm may be used to analyze the video stream from the camera during monitoring, as indicated by block 46.

[0028] Figure 4 is a schematic block diagram of an exemplary method 48 for configuring a video surveillance system, the method including testing one or more video analysis algorithms of the video surveillance system. In some cases, the method 48 may be implemented by the controller 16 of the video surveillance system 10. The configuration page may be opened and displayed on Figure 122, as shown in block 50. As shown in block 52, the desired parameters may be configured using the user interface 22. The configured parameters may be transmitted to the object simulation module 54 and block 56, where the simulated object is superimposed on a preview showing the object superimposed on a video stream from a video camera of the video surveillance system. The preview may be thought of as displaying an enhanced video stream that enhances the video stream from the camera with the superimposed simulated object.

[0029] The object simulation module 54 generates simulated objects and the movement of the simulated objects. The augmented video stream including the simulated objects moving in the field of view is visualized, as shown in block 58. One or more detection algorithms are run on the augmented video stream, as shown in block 60. A visual check is run to see if the superimposed simulated objects are detected as expected by the one or more video analysis algorithms being tested, as shown in block 62. This may include a user selecting a detection algorithm via a visual check. Figure 1 The user interface 22 views the augmented video stream to see whether the superimposed simulated objects are appropriately generated for the one or more video analytics algorithms being tested, and if so, whether the alarms and / or other outputs from the one or more video analytics algorithms are appropriate.

[0030] Figure 5 6 is a schematic block diagram of an exemplary method 64 for superimposing simulated objects onto a video stream to create an enhanced video stream. In some cases, the method 64 may be implemented by the controller 16 of the video surveillance system 10. Images of various objects (including people and / or vehicles) are collected, as shown in block 66. Paths showing the movement of the various objects are output, as shown in block 68. In some cases, the movement is performed by the object simulation module 54 ( Figure 4) is provided. The movement of the object within the FOV may be based on one or more simulation parameters. The simulation parameters may include one or more of the following: the object type of one or more simulation objects in the simulation object, the metric related to the number of simulation objects, the distribution of two or more simulation objects in the simulation object, the starting point of one or more simulation objects in the simulation object, the moving speed of one or more simulation objects in the simulation object, the change of the moving speed of one or more simulation objects in the simulation object over time, the change of the moving speed between two or more simulation objects, and the movement type of one or more simulation objects in the simulation object. In some cases, the distribution of two or more simulation objects in the simulation object may include one or more of Weibull distribution, Gaussian distribution, binomial distribution, Poisson distribution, uniform distribution, random distribution and geometric distribution. In some cases, the distribution of two or more simulation objects in the simulation object may include a mixed distribution of two or more different distributions. In some cases, the type of movement may include one or more of the following: random directional movement, random directional movement constrained by the movement of other simulation objects and / or by the user identification area of ​​FOV, directional movement along a predefined channel, variable speed movement, and group movement associated with two or more simulation objects in the simulation object.

[0031] In some cases, one or more of the simulation parameters may be scaled by a scaling factor that depends on the position of the simulated object in the FOV of the camera. For example, for simulated objects in the upper part of the FOV, indicating that these objects are farther away from the camera, the simulated objects may be smaller in size (fewer pixels) relative to similar objects in the lower part of the FOV. In another example, for simulated objects in the upper part of the FOV, the speed of the simulated object (across pixels) may be smaller relative to similar objects in the lower part of the FOV. In addition, in some cases, the scaling factor may depend on the placement of the camera and the size of the site being monitored. For example, for a camera placed in a large warehouse with a FOV covering a long distance, the scaling factor may be greater than a camera placed in a smaller enclosed room. When determining the scaling factor, the downward angle of the camera may also be considered. In some cases, the scaling factor is based on one or more inputs from a user.

[0032] The simulated object and its motion are superimposed on each frame of the video stream, as shown in box 72. A GIF layer can be added to the original RGB (red, green, blue) image and previewed, as seen in box 74. The RGB image with the superimposed object is temporarily saved and reloaded into the preview, as shown in box 76.

[0033] Fig. 6A and Figure 6Bis a flow chart that collectively illustrates an exemplary computer-implemented method 78 for testing one or more video analysis algorithms (such as video analysis algorithm 20) of a video surveillance system (such as video surveillance system 10). The video surveillance system includes a camera (such as camera 12) having a field of view (FOV) (such as FOV 14), and the field of view (FOV) covers a portion of a facility. The exemplary method 78 includes generating a simulated object to be superimposed on a video stream captured by the camera, as shown in box 80. In some cases, the simulated object may include a person, an animal, or a vehicle. The simulated object may include inanimate objects such as bags, laptops, chairs, and other furniture. In some cases, generating the simulated object includes displaying an image captured by the camera on a display, as shown in box 82. In some cases, generating the simulated object includes receiving a first user input identifying one or more user-identified areas relative to the image, as shown in box 84.

[0034] In some cases, receiving the first user input may include drawing an annotation on the image when the image is displayed on the display screen to identify one or more user-identified areas. The annotation may indicate a user-identified area that includes possible obstacles that may interfere with the place where the object simulator module allows the simulated object to be placed or moved. In some cases, generating the simulated object includes automatically marking each of the one or more user-identified areas with a corresponding area label, as shown in box 86. In some cases, the area label may define a corresponding area type. Examples of areas and / or area types include, but are not limited to, one or more of the following: a wall visible in the FOV, a fence visible in the FOV, an obstacle visible in the FOV, a safe area visible in the FOV, a window visible in the FOV, a door visible in the FOV, a sidewalk visible in the FOV, and a roadway visible in the FOV.

[0035] In some cases, a user may draw an annotation indicating a target area or venue. As an example, a user may draw a box or other shape around a specific target venue in the FOV where one or more video analysis algorithms will be run. In some cases, each target venue may be marked to determine which of multiple video analysis algorithms are run in the associated target venue. For example, a user may draw a shape covering the venue in front of a cash machine, and the shape may be marked with a loitering video analysis algorithm that detects whether / when people are loitering in the target area. In another example, a user may draw a shape covering an area of ​​a train station, and the shape may be marked with a crowd detection video analysis algorithm that detects whether / when people gather in the target area. These are just examples. In another example, a user may draw a box or other shape around a specific venue, indicating that the venue is not a target venue, and no video analysis algorithm needs to be run on the specific venue. A bathroom or a locker room may represent a non-target venue. This is just an example.

[0036] In some cases, generating the simulation object includes receiving a second user input identifying one or more simulation parameters, as shown in block 88. In some cases, one or more of the simulation parameters are based at least in part on the second user input, as shown in block 90. ​​In some cases, and as shown in Figure 6B Continuing from above, generating the simulated object includes determining a plurality of characteristics of the simulated object based at least in part on the one or more simulation parameters and based on one or more of the user-identified regions and corresponding region labels, as shown in block 92. The plurality of characteristics of the simulated object include, for example, determining a position in the FOV where each of the one or more simulated objects is allowed to move and / or a position where each of the one or more simulated objects is not allowed to move based on the one or more user-identified regions and corresponding region labels.

[0037] In some cases, the characteristics of the simulated objects include the number of simulated objects in the FOV, as shown in box 94. In some cases, the characteristics of the simulated objects include the distribution and / or distribution type of the simulated objects in the FOV, as shown in box 96. In some cases, the characteristics of the simulated objects include the origin (e.g., time and / or position) of each simulated object in the FOV, as shown in box 98. In some cases, the characteristics of the simulated objects include the movement (e.g., speed, movement type) of each simulated object in the FOV, as shown in box 100. In some cases, the characteristics of the simulated objects may be affected by the target area indicated by the user. The simulated objects can be guided into or through the target area, particularly for testing specific video analysis algorithms. The simulated objects can be guided away from non-target sites, because running the video analysis algorithm on these sites can be considered a waste of computing resources.

[0038] In some cases, the method 78 includes superimposing a simulated object on the FOV of the video stream captured by the camera to create an enhanced video stream, as shown in block 102. The method 78 may include processing the enhanced video stream using one or more video analysis algorithms to test the effectiveness of each of the one or more video analysis algorithms, as shown in block 104. The one or more video analysis algorithms may include, but are not limited to, one or more of the following: crowd detection algorithms (e.g., crowd presence or formation), crowd analysis algorithms (e.g., crowd size, rate of crowd formation or dispersion, density of crowd, movement within crowd, movement of crowd, etc.), people counting algorithms (e.g., number of people in a defined target area of ​​the FOV), behavior detection algorithms (e.g., running, walking, loitering, yelling, shoplifting, protesting, vandalizing property, tailing through a secure entry point, wearing a mask, social distancing, etc.), intrusion detection algorithms (e.g., entering a secure area), perimeter protection algorithms (e.g., crossing a secure perimeter), and tailing detection algorithms (e.g., tailing through a secure entry point). These are just examples. In some cases, the method 78 may include receiving a third user input that marks at least one of the one or more user identification areas with a corresponding area tag that includes an area tag type, as shown in block 106. The method 78 may include using video analysis to automatically identify at least one of the one or more user identification areas and mark it with a corresponding area tag, as shown in block 108. That is, the video analysis may be used to identify walls, doors, windows, fences, parking spaces, and other areas in the FOV, and automatically mark each of the identified areas with an appropriate area tag. The method 78 may include using video analysis to automatically identify and mark at least some of the venues (including the one or more user identification areas) as non-target venues. For example, a potential target person is unlikely to appear through a solid wall.

[0039] In some cases, the one or more simulation parameters may include one or more of the following: object type of one or more simulation objects in the simulation object, a metric related to the number of simulation objects, the distribution of two or more simulation objects in the simulation object, the starting point of one or more simulation objects in the simulation object, the moving speed of one or more simulation objects in the simulation object, the change of the moving speed of one or more simulation objects in the simulation object over time, the change of the moving speed between two or more simulation objects in the simulation object, and the type of movement of one or more simulation objects in the simulation object. These are just examples. In some cases, the distribution of two or more simulation objects in the simulation object may include one or more of Weibull distribution, Gaussian distribution, binomial distribution, Poisson distribution, uniform distribution, random distribution, and geometric distribution. In some cases, the distribution of two or more simulation objects in the simulation object may include a mixed distribution of two or more different distributions. In some cases, the type of movement may include one or more of the following: random directional movement, random directional movement constrained by the movement of other simulation objects and / or by the user identification area of ​​FOV, directional movement along a predefined channel, variable speed movement, and group movement associated with two or more simulation objects in the simulation object.

[0040] Fig. 7A and Figure 7B 1 are flow charts collectively illustrating an exemplary computer-implemented method 110 for testing one or more video analytics algorithms (such as video analytics algorithm 20) of a video surveillance system (such as video surveillance system 10). The video surveillance system includes a camera (such as camera 12) having a field of view (FOV) (such as FOV 14), the field of view (FOV) covering a portion of a facility. The method 110 includes generating a simulated object to be superimposed on a video stream captured by the camera, as shown in block 112. Generating the simulated object may include receiving a user input identifying a particular video analytics algorithm from a plurality of video analytics algorithms, as shown in block 114. Generating the simulated object may include determining a plurality of characteristics of the simulated object desired for testing the particular video analytics algorithm based at least in part on the particular video analytics algorithm identified by the user input, as shown in block 116. The plurality of characteristics of the simulated object may include a number of simulated objects in the FOV, as shown in block 118. The plurality of characteristics of the simulated object may include a distribution of the simulated objects in the FOV, as shown in block 120. The plurality of characteristics of the simulated object may include a starting point for each of the simulated objects in the FOV, as shown in block 122. The plurality of characteristics of the simulated objects may include the movement of each of the simulated objects in the FOV, as shown in block 124. The plurality of characteristics of the simulated objects may include the position or area in the FOV of one or more of the simulated objects in the FOV. These are merely examples.

[0041] Method 110 Figure 7B Continuing above, the simulated object is superimposed on the FOV of the video stream captured by the camera to create an enhanced video stream, as shown in block 126. The enhanced video stream is processed using a specific video analytics algorithm identified by the user input to test the effectiveness of the specific video analytics algorithm, as shown in block 128. In some cases, the specific video analytics algorithm may include one of the following: a crowd detection algorithm (e.g., crowd presence or formation), a crowd analysis algorithm (e.g., crowd size, rate of crowd formation or dispersion, density of the crowd, movement within the crowd, movement of the crowd, etc.), a people counting algorithm (e.g., number of people in a defined target area of ​​the FOV), a behavior detection algorithm (e.g., running, walking, loitering, yelling, shoplifting, protesting, vandalizing property, tailing through a secure entry point, wearing a mask, social distancing, etc.), an intrusion detection algorithm (e.g., entering a secure area), a perimeter protection algorithm (e.g., crossing a secure perimeter), and a tailing detection algorithm (e.g., tailing through a secure entry point). These are just examples. In some cases, the user input may include user input to select one or more video analytics algorithms from a plurality of video analytics algorithms to use in conjunction with the video stream of the camera.

[0042] Fig. 8A and Figure 8B 1 is a flowchart that collectively illustrates a computer-implemented method 130 for testing a video surveillance system, such as video surveillance system 10, that includes a camera, such as camera 12, having a field of view (FOV), such as FOV 14, that encompasses an area of ​​a facility. The exemplary method 130 includes generating a simulated object to be superimposed on a video stream captured by the camera, as shown in block 132. Generating the simulated object may include receiving a speech-based or text-based description of one or more scenes to be tested from a user, as shown in block 134. Generating the simulated object may include determining a plurality of characteristics of the simulated object based at least in part on the speech-based or text-based description of the one or more scenes to be tested, as shown in block 136. The characteristics of the simulated object may include the number of simulated objects in the FOV, as shown in block 138. The characteristics of the simulated object may include the distribution of the simulated objects in the FOV, as shown in block 140. The characteristics of the simulated object may include a starting point for each of the simulated objects in the FOV, as shown in block 142. The characteristics of the simulated object may include movement for each of the simulated objects in the FOV, as shown in block 144. These are merely example characteristics of the simulated object. The exemplary method 130 also includes overlaying the simulated object on the FOV of the video stream captured by the camera to create an augmented video stream, as shown in block 146. The augmented video stream is processed to test one or more scenes described in the speech or text-based description, as shown in block 148.

[0043] In some cases, and Figure 8B Continuing from above, the method 130 may include extracting keywords from the speech or text-based description of the one or more scenes to be tested using natural language processing, as shown in block 150. The keywords may be mapped to one or more of the plurality of characteristics of the simulated object, as shown in block 152. One or more of the plurality of characteristics of the simulated object may be determined based at least in part on the mapping, as shown in block 154. In some cases, one or more video analytics algorithms may be identified based at least in part on the description of the one or more scenes, as shown in block 156. The enhanced video stream may be processed to test the effectiveness of each of the one or more identified video analytics algorithms, as shown in block 158. In some cases, the method 130 may also include determining one or more of the plurality of characteristics of the simulated object based at least in part on the identified one or more video analytics algorithms, as shown in block 160.

[0044] In one example, the voice-based or text-based description may be "Testing for unauthorized entry through door A of area 1". When the description is a voice-based description, the description may be converted to a text-based description by a voice-to-text translator. The system may parse the voice-based or text-based description and identify the phrases "door A" and "area 1". Based on these phrases, the system may identify which camera(s) of the video surveillance system 10 has a FOV that includes door A of area 1. The system may also parse the voice-based or text-based description and identify the phrase "unauthorized entry". The phrase may be mapped to one of a plurality of video analytics algorithms, such as an intrusion detection video analytics algorithm (or unauthorized entry detection video analytics algorithm). The system may then be configured to automatically identify characteristics of a simulated object that are suitable for testing the identified video analytics algorithm, i.e., an intrusion detection video analytics algorithm, for intrusion of door A of area 1. The system may then overlay a simulated object having the identified characteristics on the FOV of a video stream captured by the identified camera to create an enhanced video stream. The system may then process the enhanced video stream using a particular video analytics algorithm (e.g., an intrusion detection video analytics algorithm) to test the effectiveness of the particular video analytics algorithm (e.g., an intrusion detection video analytics algorithm) in identifying an intrusion at Door A of Area 1.

[0045] In another example, the speech or text-based description may be "crowd detection in area 4". The system may parse the speech or text-based description and identify the phrase "area 4". Based on the phrase, the system may identify which camera(s) of the video surveillance system 10 has a FOV that includes area 4. The system may also parse the speech or text-based description and identify the phrase "crowd detection". The phrase may be mapped to one of a plurality of video analytics algorithms, such as a crowd detection video analytics algorithm. The system may then be configured to automatically identify characteristics of a simulated object that are suitable for testing the video analytics algorithm identified in area 4, i.e., the crowd detection video analytics algorithm. The system may then overlay a simulated object having the identified characteristics on the FOV of a video stream captured by the identified camera to create an enhanced video stream. The system may then process the enhanced video stream using a specific video analytics algorithm (e.g., a crowd detection video analytics algorithm) to test the effectiveness of the specific video analytics algorithm (e.g., the crowd detection video analytics algorithm) in identifying a crowd in area 4. This method may facilitate testing the video surveillance system 10 from a remote device and / or remote location.

[0046] The mapping between various phrases of the text-based description and the location and / or video analysis algorithm can be stored in a mapping table. In some cases, the mapping can be learned by the artificial intelligence model over time, and once trained, the artificial intelligence model can automatically generate appropriate tests for the video surveillance system 10 based on the voice or text-based description.

[0047] Fig. 9 1 is a schematic block diagram of an exemplary object simulation module 162. The exemplary object simulation module 162 includes receiving a text description of how the simulated object should move, as shown in box 164. Natural language processing can be used to analyze the text description, as shown in box 166. As a result, keywords are extracted, as shown in box 168. The keywords are mapped to the simulation, as shown in box 170. The mapping is one of the inputs to the modeling box 172. Additional inputs may include one or more of the following: target area 174, number of objects 176, distribution type (e.g., Weibull distribution, Gaussian distribution, binomial distribution, Poisson distribution, uniform distribution, random distribution, geometric distribution, mixed distribution) 178, annotations 180 identifying walls, doors, windows, fences, parking spaces and other areas in the FOV, and in some cases, the type of object movement (e.g., walking, running, wandering), which are all provided to the distribution box 182. Based on all of these inputs, the output from the distribution box 182 is the simulated object movement, as shown in box 184. The simulated object movement is then superimposed on the video stream from the camera to test the effectiveness of one or more video analytics algorithms in the field.

[0048] Fig.10is a schematic block diagram of an exemplary method 186 for generating object movement in a video stream to create an enhanced video stream. The exemplary method 186 begins with Fig. 9 , including a target area 174, a number of objects 176, and an annotation 180. Method 186 also includes an object movement type 188. Annotations 180 and / or target areas 174 are converted into X, Y areas on the FOV of the video stream, as shown in boxes 190 and 192, respectively. The number of objects 176 and the object movement type 188 are provided to box 194, where the requested object distribution, object movement, and object shape are generated. For different object types, the object shape can be different. For example, an adult may have a larger size than a child. In another example, a vehicle may have a larger size and a different shape from an adult. In addition, the object distribution, object movement, and object shape can be scaled according to the position of the object in the field of view. For example, an object located in the top area of ​​the FOV may be farther away from the camera than an object located in the bottom area of ​​the FOV, and therefore may have a tighter distribution (on a pixel-by-pixel basis), a slower movement (on a pixel-by-pixel basis), and a smaller size (on a pixel-by-pixel basis).

[0049] Blocks 190, 192, and 194 each provide information to decision block 196, where it is determined whether the object simulator is attempting to move a polygonal shape (object) into one of the regions received from block 190 or block 192. If so, control passes to block 198, and the simulator does not move the object to position X1, Y1. If not, control passes to block 200, and the simulator moves the object to position X1, Y1.

[0050] In this embodiment, both boxes 190 and 192 define areas where the simulator is not allowed to move objects. In other embodiments, one of these areas (such as the area defined by box 192) may represent an area where the simulator is encouraged to move objects. For example, if you attempt to test a crowd detection algorithm in area A of the FOV, the area defined by box 192 may correspond to area A, and the simulator may be encouraged or even required to move one or more objects into the area defined by box 192. In this embodiment, decision box 196 may determine whether the object simulator attempts to move a polygonal shape (object) into the area defined by box 192. If so, the simulator is encouraged to move the object to position X1, Y1. If not, the simulator is not encouraged, but is still allowed to move the object to position X1, Y1. In this embodiment, control does not go to box 198 or 200.

[0051] Although several exemplary embodiments of the present disclosure have been described in this manner, it will be readily understood by those skilled in the art that other embodiments may be made and used within the scope of the claims appended hereto. However, it should be understood that the present disclosure is in many respects merely illustrative. Details (especially details related to shape, size, arrangement of parts, and exclusion and order of steps) may be changed without exceeding the scope of the present disclosure. Of course, the scope of the present disclosure is defined in the language of the appended claims.

Claims

1. A computer-implemented method for testing one or more video analysis algorithms of a video surveillance system, the video surveillance system comprising a camera having a field of view (FOV), the field of view covering a portion of a facility, the method comprising: Generating a simulated object to be superimposed on a video stream captured by the camera, wherein generating the simulated object comprises: displaying images captured by the camera on a display; receiving a first user input identifying one or more user-identified regions relative to the image; marking each of the one or more user identification regions with a corresponding region label; receiving a second user input identifying one or more simulation parameters; identifying one or more simulation parameters based at least in part on the second user input; Determining a plurality of characteristics of the simulated object based at least in part on the one or more simulation parameters and on one or more of the user-identified regions and the corresponding region labels, the plurality of characteristics comprising one or more of: the number of the simulated objects in the FOV; distribution of the simulated objects in the FOV; a starting point for each of the simulated objects in the FOV; movement of each of the simulated objects in the FOV; superimposing the simulated object over the FOV of the video stream captured by the camera to create an augmented video stream; and The enhanced video stream is processed using one or more video analytics algorithms to test the effectiveness of each of the one or more video analytics algorithms. The method of claim 1 , wherein the region label defines a region type.

3. The method of claim 2, wherein the area type comprises one or more of a wall, a fence, a barrier, a secure area, a window, a door, a sidewalk, and a roadway.

4. The method according to claim 3, wherein determining the multiple characteristics of the simulated object includes determining a position where each of the one or more simulated objects is allowed to move in the FOV and / or a position where each of the one or more simulated objects is not allowed to move based on the one or more user identification areas and the corresponding area labels. The method of claim 1 , wherein the simulated object comprises one or more of a person, an animal, and a vehicle.

6. The method of claim 1, wherein the one or more video analysis algorithms include one or more of a crowd detection algorithm, a crowd analysis algorithm, a people counting algorithm, a behavior detection algorithm, an intrusion detection algorithm, a perimeter protection algorithm, and a tailgating detection algorithm. 7 . The method of claim 1 , wherein receiving a first user input comprises drawing an annotation on the image while the image is displayed on a display screen to identify the one or more user identification regions.

8. The method of claim 1, comprising receiving a third user input marking at least one of the one or more user identification regions with a corresponding region tag comprising a region tag type.

9. The method of claim 1, wherein the one or more simulation parameters include one or more of the following: an object type of one or more of the simulated objects; Metrics related to the number of simulated objects; a distribution of two or more of the simulated objects; a starting point of one or more of the simulated objects; a moving speed of one or more of the simulated objects; a change in the movement speed of one or more of the simulated objects over time; a change in movement speed between two or more of the simulated objects; and A movement type of one or more of the simulated objects.

10. A method according to claim 9, wherein the movement type includes one or more of the following: random directional movement, random directional movement constrained by the movement of other simulated objects and / or constrained by the user-identified area of ​​the FOV, directional movement along a predefined channel, variable speed movement, and group movement associated with two or more of the simulated objects.

Images