System and method for distributed video management in a traffic surveillance system - Patents.com

Abstract

The traffic monitoring system includes a remote monitoring system providing a video management platform through which a user reviews video data, and one or more traffic sensors communicatively coupled to the remote monitoring system via a wireless network, wherein at least one of the one or more traffic sensors includes a controller that generates low-bit video data and corresponding high-bit video data, and a transceiver that (a) initially transmits the low-bit video data to the remote monitoring system, excluding the high-bit video data, and (b) thereafter transmits the high-bit video data to the remote monitoring system in response to a request for the corresponding high-bit video data received from the remote monitoring system.

Description

[Technical field]

[0001] This application claims priority to U.S. Application No. 18 / 184,576, filed March 15, 2023, and Provisional Application No. 63 / 320,616, filed March 16, 2022, the entire contents of which are expressly incorporated herein by reference.

[0002] The present invention relates to traffic surveillance systems and methods, and more particularly to distributed video management for such systems and methods. [Background technology]

[0003] A traffic monitoring system generally includes traffic cameras that capture video clips of passing traffic, e.g., roadway traffic, for playback review by law enforcement or other interested users. Playback review of the captured video clips is generally performed via a video management system.

[0004] At the same time, the data intensity of video clips continues to increase, especially when additional back-end processing is desirable. For example, successful application of facial recognition to video clips requires higher data intensity. The current approach to meeting this need is to simply transmit all video clips at higher and higher data intensity.

[0005] However, historically, less than 0.5% of all collected video data has ever been reviewed. Thus, for example, when rich media is being generated – video data becomes increasingly data intensive, and so transmitting the data to system servers becomes increasingly expensive and inefficient.

[0006] It is therefore desirable to provide a traffic monitoring system that reduces the workload on the communications infrastructure while making this data-intensive video data available to a video management system as needed. Summary of the Invention

[0007] A system and method are disclosed for a traffic monitoring system that reduces the workload on the communications infrastructure while making high bit rate video data available to a video management system as needed.

[0008] In at least one embodiment, a traffic monitoring system includes a traffic sensor and a server system remote therefrom. The traffic sensor can have a controller that generates both low-bit video data and high-bit video data. The traffic sensor can also have a transceiver that initially transmits the low-bit video data to the remote server system for review by a user via the video management platform. The user can review the low-bit video data and request corresponding high-bit video data from the traffic sensor via the video management platform. The high-bit video data can be transmitted by the traffic sensor to the server system in response to the request. The user can then review the high-bit video data via the video management platform. Thus, the workload on the communications infrastructure can be reduced without limiting the availability of high-bit video data to the video management platform.

[0009] Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings. It is to be understood that the example or examples within this disclosure are non-limiting examples, and that the present invention is intended to cover modifications and equivalents of these examples.

[0010] The features, objects and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify elements throughout. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 illustrates an example system in accordance with at least one embodiment of the present invention. [Diagram 2] FIG. 1 illustrates an example method in accordance with at least one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The aforementioned drawings illustrate the present invention in at least one embodiment, which is further defined in detail in the following description. Those skilled in the art may make changes and modifications to what is described herein without departing from the spirit and scope thereof. While the present invention is susceptible of embodiment in many different forms, at least one preferred embodiment of the present invention is shown in the drawings and will be described in detail herein, with the understanding that the present disclosure is to be construed as an exemplification of the principles of the present invention and is not intended to limit the broad aspects of the present invention to any embodiment shown.

[0013] In accordance with the practices of those skilled in the art, the present invention is described below with reference to operations performed by a computer system or similar electronic system. Such operations are sometimes referred to as being computer-executed. It will be understood that the symbolically represented operations include the manipulation by a processor, e.g., a central processing unit, of electrical signals representing data bits and the maintenance of the data bits in memory locations, e.g., in a system memory, as well as other processing of the signals. The memory locations in which the data bits are maintained are physical locations having particular electrical, magnetic, optical, or organic properties corresponding to the data bits.

[0014] When implemented in software, the code segments perform certain tasks described herein. The code segments may be stored on a processor-readable medium. Examples of processor-readable media include electronic circuits, semiconductor memory devices, read-only memories (ROMs), flash memories or other non-volatile memories, floppy diskettes, CD-ROMs, optical disks, hard disks, and the like.

[0015] In the following detailed description and corresponding figures, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood that the present invention may be practiced without such specific details. Additionally, well-known methods, procedures, components, and circuits have not been described in detail.

[0016] The present invention relates generally to traffic monitoring systems and methods, and more particularly to such systems and methods for reducing the workload on the system's communications infrastructure while retaining the ability to provide high bit rate video data as needed for use by a video management platform.

[0017] FIG 1 is a schematic diagram of a traffic monitoring system 100 according to one or more aspects of the present invention. As shown in FIG 1, the traffic monitoring system 100 comprises one or more traffic sensors communicatively coupled to a system server 140 via a network 160. The system server may also be communicatively coupled to one or more user devices 180 via the network. The traffic monitoring system 100 generally enables collection of traffic-related data for transmission, via the network, to the system server 140. The traffic monitoring system 100 also generally enables user access to the traffic-related data stored in the system server 140 via the coupled user device 180.

[0018] The traffic sensor 120 may include an imaging device 122, a controller 124, a memory 126, and a transceiver 128, each communicatively coupled to a common data bus 130 that enables data communication between the respective components.

[0019] The imaging device 122 can capture images of traffic, particularly video images of the vehicles 10 that make up the traffic, from which video data is generated. The imaging device 122 can be any type of video camera that captures video images suitable for computerized image recognition of objects in the captured images. For example, the camera can use charge-coupled-device (CCD), complementary metal-oxide-semiconductor (CMOS), and / or other imaging technologies to capture standard, night vision, infrared, and / or other types of images having predetermined resolution, contrast, color depth, and / or other image characteristics. The video data can be time-stamped to indicate the date and time of recording. The video data can further include other identification information, including geolocation data and / or traffic sensor ID data.

[0020] The controller 124 may include image processing software for applying image processing to the video data captured by the imaging device 122 to generate processed video data. Some exemplary types of image processing that may be applied to the video data include image correction, encoding, compression, and recognition processing. The image processing may result in the generation of processed video data including, for example, low-bit video data and corresponding high-bit video data. The low-bit video data may have image resolution and / or other characteristics such that the low-bit video data is lower in bit size than the corresponding high-bit video data of the same scene. Additionally, each of the low-bit video data and the high-bit video data may be associated with the same identifier (e.g., timestamp, geolocation data, sensor ID data, etc.) such that an identifier of the low-bit video data may be used to identify the corresponding high-bit video data via a matching identifier.

[0021] In some embodiments, the image processing results in the generation of low-bit video data if the originally captured video data was high-bit video data. In further embodiments, the image processing results in the generation of high-bit video data if the originally captured video data was low-bit video data. In further embodiments, the image processing results in the generation of both high-bit video data and low-bit video data if neither was the originally captured video data.

[0022] During operation, low bit video data may be initially transmitted to system server 140 in place of high bit video data. The high bit video data may then be later transmitted to system server 140 in response to a request for high bit video data, which may be received by controller 124 from system server 140 in response to the transmission of the low bit video data. The controller 124 of the appropriate traffic sensor 120 may accordingly identify and retrieve the requested high bit video data from its memory 126.

[0023] The request may be a global request sent to each traffic sensor 120 or a targeted request sent to one or more specific traffic sensors 120. Thus, the request may include at least one identifier that identifies the requested high-bit video data sufficient for the appropriate traffic sensor 120 to retrieve the requested high-bit video data from its memory 126. The identifier may include, for example, one or more of the following: timestamp data, geolocation data, and a sensor ID. The timestamp data may include a timestamp range of the high-bit video data to define a temporal range of the requested high-bit video data. The geolocation and / or sensor ID data may be sufficient to identify the appropriate traffic sensor 120 from which to retrieve the high-bit video data.

[0024] The controller 124 may be embodied as one or more processors, collectively or individually, programmed to perform the functions of the unit in accordance with software stored in memory 126. Each processor may be a standard processor, e.g., a central processing unit (CPU), or a special purpose processor, e.g., an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or part thereof.

[0025] The memory 126 stores software and data that may be accessed by the processor and includes both temporary and persistent storage. The temporary storage is configured to temporarily store data to be processed or otherwise acted upon by other components and may include a data cache, RAM, or other temporary storage type. The persistent storage is configured to store the software and data until it is deleted. The memory 126 is suitably configured to store the data and information described herein. In particular, the memory 126 stores therein video data, including low-bit video data and its corresponding high-bit video data.

[0026] The transceiver communicatively couples the traffic sensor 120 to a network to enable data transmission thereto. The network may be any type of network, wired or wireless, configured to facilitate communication and transmission of data, instructions, etc., and may include a local area network (LAN) (e.g., Ethernet or other IEEE 802.03 LAN technology), Wi-Fi (e.g., IEEE 802.11 standard, wide area network (WAN), virtual private network (VPN), global area network (GAN)), cellular network, or any other type of network or combination thereof.

[0027] The system server 140 may include one or more server computers 142 connected to a network. Each server computer may include computer components, including one or more processors, memory, displays, and interfaces, and may also include software instructions and data for performing the server functions described herein. The server may also include one or more storage devices configured to store large amounts of data and / or information, and may further include one or more databases. For example, the storage device may be a collection of storage components, such as ROM, RAM, hard drives, solid state drives, removable drives, network storage, virtual memory, cache, registers, or a mixed collection of storage components, configured to allow the server computer 142 to access it. The storage components may also support one or more databases for storing data therein.

[0028] System server 140 is generally configured to provide centralized support for traffic sensors 120. System server 140 is configured to receive video data (and other data) from each traffic sensor 120 and to store the video data for access by users via user devices 180. Thus, system server 140 may include one or more databases configured to store the video data received from traffic sensors 120, including low-bit video data and high-bit video data.

[0029] The system server 140 may include one or more video management software applications stored in memory that, when executed by a processor, configure the server computer to provide and / or otherwise support the video management platform 144. The video management platform 144 may be an online platform (e.g., a website) or a local platform (e.g., a closed computer network).

[0030] Video management platform 144 may be generally configured to enable users to interact with video data stored by system server 140 via user devices 180. In particular, video management platform 144 may support a graphical user interface 182 that enables users to select and retrieve video data for video playback via user devices 180. In some embodiments, the video playback may amount to substantially real-time, or live stream, video playback.

[0031] The graphical user interface 182 may also enable one or more playback functions, including but not limited to allowing a user to pause, rewind, and fast forward video playback. The graphical user interface 182 may further enable other interactions, which may include, for example, object recognition (e.g., license plate recognition, vehicle recognition, etc.), object tagging, video frame annotation, data analysis, hit list comparison, and / or smart search capabilities.

[0032] In at least one embodiment, the video management platform 144 further enables a user to request high-bit video data from the traffic sensors 120 that corresponds to the identified low-bit video data. The system server 140 may be configured to send the request to the appropriate traffic sensor 120. In response, the traffic sensor 120 retrieves the requested high-bit video data from memory and sends it to the server system for storage and use by the video management platform 144.

[0033] During operation, a user may review low bit video playbacks that correspond to the low bit video data stored in the database. The low bit video playbacks may be sufficient for the user to identify one or more portions of the video playback if further review as a high bit video playback is desired. The user may request corresponding high bit video data from the appropriate traffic sensor 120, which when received by the video management system may be reviewed as a high bit video playback.

[0034] User device 180 is generally a computing device and may include mobile (e.g., laptop computer, tablet computer, smartphone, PDA, wearable, etc.) or stationary (e.g., desktop computer, etc.), general-purpose or special-purpose, devices configured to communicate data and information with system server 140. User device 180 may include components typically associated with such devices, such as one or more processors, physical memory, software instructions, data, displays, and interfaces. User device 180 may further include one or more software applications stored in memory that, when executed by the processor, configure user device 180 to function as described herein. In particular, user device 180 is configured to enable a user to interact with video management platform 144 as described herein.

[0035] FIG. 2 is a flow diagram illustrating an exemplary method 200 of operation of the traffic monitoring system 100 in accordance with one or more aspects of the present invention.

[0036] During operation, in step 202, the imaging devices 122 of each of the plurality of traffic sensors 120 capture images of vehicular traffic, i.e., video images of passing vehicles, and generate video data therefrom. The traffic sensors 120 are preferably located at various roadway locations where vehicular traffic is to be monitored. The imaging devices 122 are preferably positioned such that the captured images include the license plates of each of the passing vehicles, as well as other vehicle characteristics, such as vehicle type, class, make, model, color, year, drive type, license plate number, registration, trajectory, speed, location, etc., or any combination thereof.

[0037] At step 204, the controller 124 of each imaging device 122 processes the video data captured by that imaging device 122 to generate low-bit video data and high-bit video data. The controller 124 may use any image processing software suitable for this purpose. At least the high-bit video data is retrievably stored in memory until deleted.

[0038] At step 206, the traffic sensor 120 transmits the low-bit video data to the system server 140. The low-bit video data may have image resolution and / or other characteristics such that the low-bit video data has a lower bit size than corresponding high-bit video data of the same scene. Such transmission of the low-bit video data may include transmitting one or more identifiers (e.g., timestamp data, geolocation data, traffic sensor ID, etc.). In at least one embodiment, the low-bit video data is automatically transmitted, i.e., "pushed," to the system server 140.

[0039] At step 208, a user accesses the server system's video management platform 144 to review the low bit video playback of the low bit video data via a graphical user interface 182. Such access may be through one or more of the user devices 180 via a network connection. The graphical user interface 182 may enable playback functions, including but not limited to, allowing the user to pause, rewind, and fast forward the video playback to allow the user to identify one or more portions of the low bit video data for which further review of the captured image via high bit playback is desirable.

[0040] At step 210, a user requests high bit video data corresponding to a portion of the low bit video playback for which further high bit review is desired. The request may include an identifier of the corresponding low bit video data that also corresponds to an identifier of the corresponding high bit video data to be reviewed. The request is then sent at step 212 to the appropriate traffic sensor 120 from which the appropriate high bit video data is obtained.

[0041] At step 214, the traffic sensor 120 may retrieve the requested high bit video data from memory and transmit it to the server system for storage and use by the video management platform 144. In particular, the high bit video data may be accessed for high bit video playback via the video management platform 144.

[0042] In this manner, the workload on the communications infrastructure of the traffic monitoring system 100 is reduced while retaining the ability to provide high bit rate video data as needed for the video management platform 144 .

[0043] The embodiments detailed above are believed to be newer than the prior art and are believed to be essential to the operation of at least one aspect of the described system, method and / or apparatus and to the accomplishment of the aforementioned objectives. Words used herein to describe the instant embodiments should be understood not only in the spirit of their commonly defined meanings, but also to include by specific definitions herein: structures, materials or activities beyond the scope of their commonly defined meanings. Thus, if an element may be understood in the context of this specification as including multiple meanings, then its use should be understood as general to all possible meanings supported by this specification and by the word or words describing the element.

[0044] The definitions of words or drawing elements described herein are intended to include not only the literally recited combination of elements, but also all equivalent structures, materials, or acts for performing substantially the same function in substantially the same way to obtain substantially the same results. Thus, in this sense, it is contemplated that equivalent substitutions of multiple elements may be made for any one of the described elements and its various embodiments, or that a single element may be used in place of multiple elements.

[0045] Variations from the subject matter as would be seen by one of ordinary skill in the art, whether now known or later devised, are expressly contemplated as being equivalents and various embodiments thereof within the intended scope. Accordingly, obvious substitutions now or later known to those of ordinary skill in the art are defined to be within the scope of the defined elements. Thus, the present disclosure is intended to be understood to include what is specifically illustrated and described above, what is conceptually equivalent, what may obviously be substituted, and also what incorporates the essential ideas.

[0046] Additionally, the functionality described herein may be implemented via hardware, software, firmware, or any combination thereof, unless expressly indicated otherwise. If implemented in software, the functionality may be stored in memory as one or more instructions on a computer-readable medium, including any available medium accessible by a computer that can be used to store desired program code in the form of instructions, data structures, and the like. Accordingly, certain aspects may comprise a computer program product for performing operations presented herein, such computer program product comprising a computer-readable medium having instructions stored thereon, the instructions being executable by one or more processors to perform the operations described herein. It will be appreciated that software or instructions may also be transmitted over a transmission medium as is known in the art. Additionally, modules and / or other suitable means for performing the operations described herein may be used in implementing the functionality described herein.

[0047] The foregoing disclosure has been set forth merely to illustrate the present invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and scope of the present invention may occur to those skilled in the art, the present invention should be construed as including all within the scope of the described embodiments and their equivalents.

Claims

1. a remote surveillance system providing a video management platform through which a user reviews video data; one or more traffic sensors communicatively coupled to the remote monitoring system via a wireless network; a traffic monitoring system comprising: a controller for generating low bit video data and corresponding high bit video data; (a) initially transmitting the low-bit video data to the remote monitoring system, excluding the high-bit video data; and (b) thereafter, in response to a request for the corresponding high bit rate video data received from the remote monitoring system, transmitting the high bit rate video data to the remote monitoring system. Transmitter and receiver A traffic monitoring system comprising:

2. 2. The traffic monitoring system of claim 1, wherein the remote monitoring system is configured to enable the user to send the request to the traffic sensor via the video management platform in response to reviewing the low-bit video data.

3. The traffic monitoring system of claim 1 , wherein the request comprises a global request sent to each of the one or more traffic sensors.

4. The traffic monitoring system of claim 1 , wherein the requests include targeted requests sent only to the traffic sensors.

5. 2. The traffic monitoring system of claim 1, wherein the request identifies sufficient corresponding high bit video data for a controller to identify and retrieve the corresponding high bit video data from an on-sensor memory in which it is stored.

6. The traffic monitoring system of claim 1 , wherein both the low-bit video data and the high-bit video data are generated to include at least one of a timestamp, a geolocation, and a sensor identification.

7. 2. The traffic monitoring system of claim 1, wherein the low-bit video data is not suitable for automated image recognition and the corresponding high-bit video data is suitable for automated image recognition.

8. 2. The traffic monitoring system of claim 1, wherein the low-bit video data is generated through image processing of initial video data captured by the traffic sensor, and the corresponding high-bit video data is the initial video data.

9. 2. The traffic monitoring system of claim 1, wherein both the low-bit video data and the corresponding high-bit video data are generated via image processing of original video data captured by the traffic sensor.

10. 2. The traffic monitoring system of claim 1, wherein the traffic sensor is a video camera that captures video images of surrounding traffic.

11. providing, by a remote surveillance system, a video management platform through which a user reviews video data; communicatively coupling one or more traffic sensors to the remote monitoring system via a wireless network; generating low-bit video data and corresponding high-bit video data by a traffic sensor of the one or more traffic sensors; initially transmitting the low-bit video data from the traffic sensor to the remote monitoring system, excluding the high-bit video data; transmitting, by the user, a request for the corresponding high-bit video data via the video management platform; receiving the request at the traffic sensor; transmitting, in response to a request, the corresponding high bit video data from the traffic sensor to the remote monitoring system; A traffic monitoring method comprising:

12. The method of claim 11 , wherein the request is sent by the remote monitoring system to the traffic sensor in response to the user reviewing the low-bit video data via the video management platform.

13. The method of claim 11 , wherein the request comprises a global request that is sent to each of the one or more traffic sensors.

14. The method of claim 11 , wherein the request comprises a targeted request sent only to the traffic sensor.

15. 12. The method of claim 11, wherein the request identifies sufficient of the corresponding high bit video data for a controller to identify and retrieve the corresponding high bit video data from an on-sensor memory in which it is stored.

16. The method of claim 11 , wherein both the low-bit video data and the high-bit video data are generated to include at least one of a timestamp, a geolocation, and a sensor identification.

17. 12. The method of claim 11, wherein the low-bit video data is not suitable for automated image recognition and the corresponding high-bit video data is suitable for automated image recognition.

18. 12. The method of claim 11, wherein the low-bit video data is generated through image processing of initial video data captured by the traffic sensor, and the corresponding high-bit video data is the initial video data.

19. The method of claim 11 , wherein both the low-bit video data and the corresponding high-bit video data are generated via image processing of original video data captured by the traffic sensor.

20. The method of claim 11 , wherein the traffic sensor is a video camera that captures video images of surrounding traffic.

Images