Real-time deviation in video surveillance

Abstract

The present disclosure relates to real-time deviation in video surveillance. A method may include transmitting a video stream of a live scene over a network at a real-time transmission speed, and detecting an event associated with the transmitted video stream. The method may include transmitting the video stream over the network at a speed lower than the real-time transmission speed during the event. Transmitting the video stream at a speed lower than the real-time transmission speed may introduce a time stretch so that the video stream is played in slow motion. The method may include reducing the bit rate of the video stream after the event, and transmitting the video stream over the network at the reduced bit rate after the event to compensate for the time stretch.

Description

Technical Field

[0001] The present disclosure relates generally to video surveillance. Background Art

[0002] In a video management system, an operator (e.g., a security guard) can view multiple different video streams covering multiple different surveillance areas. In addition to monitoring the video streams, the operator may also be responsible for responding to alerts associated with events occurring in those surveillance areas. For example, the surveillance areas may have temperature sensors, motion detectors, arrival sensors, presence sensors, door open / close (e.g., contact) sensors, light sensors, etc. The operator may be alerted to motion and / or door open conditions in the surveillance areas. Summary of the invention

[0003] A method may include transmitting a video stream of a live scene over a network at a real-time transmission speed, and detecting an event associated with the transmitted video stream, wherein the event has a start time and an end time in the video stream. The method may also include transmitting the video stream over the network at a speed lower than the real-time transmission speed during the event, wherein transmitting the video stream at a speed lower than the real-time transmission speed introduces a time stretch so that the video stream is played in slow motion, and reducing the bit rate of the video stream after the event and transmitting the video stream over the network at the reduced bit rate after the event to compensate for the time stretch. This embodiment may enable encoding of additional information (e.g., motion information) rather than loss of that information (e.g., packet loss or encoding to another lower bit rate). This embodiment may enable an operator to view the video stream (including the additional information) in slow motion.

[0004] Transmitting the video stream at a reduced bit rate may include transmitting the video stream at a reduced bit rate at a higher than real-time transmission speed after the event to compensate for the time stretch. This embodiment may enable compensation for the time stretch and may enable an operator to view the video in fast motion. The method may include buffering the video stream during the event in a buffer, wherein the buffer is an application layer buffer. This embodiment may enable a decision to be made as to whether buffering is to be performed for a particular video stream rather than at a network level that may include multiple video streams or other information.

[0005] The method may include determining a channel capacity corresponding to a maximum transmission speed of a channel in a network for transmitting a video stream. Transmitting the video stream over the network at a real-time transmission speed may include transmitting the video stream over the network at the channel capacity. This embodiment may enable transmission of the video stream at or near the channel capacity while still allowing the video stream bit rate to exceed the channel capacity, for example, during an event.

[0006] The method may include receiving a video stream corresponding to a time during an event over a network at a lower than real-time transmission speed, and displaying the video corresponding to the time during the event on a display in slow motion corresponding to the time stretch. This embodiment may enable an operator to view additional information (e.g., motion) encoded in the video stream during the event.

[0007] The method may include: receiving a video stream corresponding to a time after an event over a network at a higher than real-time transmission speed, and displaying the video corresponding to the time after the event on a display in fast motion. This embodiment may enable an operator to "catch up" with real-time viewing in the video stream after an event.

[0008] The transmitted video stream has a first bit rate before the event and a second bit rate during the event. In one embodiment, the second bit rate is higher than the first bit rate. The transmitted video stream has a first bit rate before the event and a third bit rate after the event. The third bit rate corresponds to a reduced bit rate, and in one embodiment the third bit rate is lower than the first bit rate. Reducing the bit rate of the video stream may include reducing a quality factor of the video stream. Reducing the quality factor of the video stream may include reducing a frame rate in the video stream, reducing a quantization parameter, or reducing a resolution of an image.

[0009] Detecting an event of interest associated with a video stream may include detecting motion in video of the video stream, detecting an increase in the bit rate of the video stream, receiving an event message in a control system, or receiving an event message in a physical access control system. This embodiment may enable a monitoring system to capture additional information in a video stream (e.g., such as motion or increased quality) while still enabling transmission of that information when the bit rate of the video stream is greater than the channel capacity.

[0010] The device may include a transmitter and a processor. The transmitter may transmit a video stream of a live scene over a network at a real-time transmission speed; and the processor may detect an event associated with the transmitted video stream. The event has a start time and an end time in the video stream. The transmitter may transmit the video stream over the network at a speed lower than the real-time transmission speed during the event, wherein when the video stream is transmitted at a speed lower than the real-time transmission speed, a time stretch is introduced so that the video stream is played in slow motion. The processor may reduce the bit rate of the video stream after the event, and the transmitter may transmit the video stream over the network at a reduced bit rate after the event to compensate for the time stretch. This embodiment may enable encoding of additional information (e.g., motion information) instead of the information being lost (e.g., packet loss or encoding to another lower bit rate). This embodiment may enable an operator to watch the video stream (including the additional information) in slow motion.

[0011] In one embodiment, the transmitter may transmit the video stream at a reduced bit rate by transmitting the video stream at a speed higher than the real-time transmission speed after the event to compensate for the time stretch. This embodiment may enable compensation for the time stretch and may enable an operator to watch the video in fast motion. The device may include a buffer to buffer the video stream during the event in the buffer. In one embodiment, the buffer is an application layer buffer. This embodiment may enable a decision to be made about buffering to be performed for a specific video stream rather than for a network level that may include multiple video streams or other information.

[0012] The processor may be configured to determine a channel capacity corresponding to a maximum transmission speed of a channel in the network for transmitting the video stream. The transmitter may transmit the video stream through the network at a real-time transmission speed by transmitting the video stream through the network at the channel capacity. This embodiment may enable transmission of the video stream at or near the channel capacity while still allowing the video stream bit rate to exceed the channel capacity during an event, for example.

[0013] The device may be associated with a receiver to receive a video stream corresponding to the time during the event over a network at a transmission speed lower than real time. The device may be associated with a display to display the video corresponding to the time during the event on the display in slow motion corresponding to the time stretch. In one embodiment, the transmitter may transmit the video stream at a reduced bit rate by transmitting the video stream at a speed higher than the real time transmission speed after the event to compensate for the time stretch. This embodiment may enable an operator to view additional information (e.g., motion) encoded in the video stream during the event.

[0014] The receiver can receive a video stream corresponding to the time after the event through the network at a higher speed than the real-time transmission speed, and the display can display the video corresponding to the time after the event on the display in fast motion. This embodiment can enable the operator to "catch up" with real-time viewing in the video stream after the event.

[0015] The transmitted video stream has a first bit rate before the event and a second bit rate during the event. In one embodiment, the second bit rate is higher than the first bit rate. The transmitted video stream has a first bit rate before the event and a third bit rate after the event. In one embodiment, the third bit rate corresponds to a reduced bit rate and the third bit rate is lower than the first bit rate.

[0016] The processor may reduce the bit rate of the video stream by reducing the quality factor of the video stream. The processor may reduce the quality factor of the video stream by reducing the frame rate in the video stream, reducing the quantization parameter, or reducing the resolution. The processor may be configured to detect events of interest associated with the video stream by detecting motion in the video of the video stream, detecting an increase in the bit rate in the video stream, receiving an event message in a control system, or receiving an event message in a physical access control system. This embodiment may enable the monitoring system to capture additional information in the video stream (e.g., such as motion or improved quality) while still enabling the transmission of the information when the bit rate of the video stream is greater than the channel capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a block diagram illustrating an exemplary environment in one embodiment;

[0018] Figure 2 The diagram shows an embodiment of Figure 1 A block diagram of exemplary components of a camera;

[0019] Figure 3 is a block diagram illustrating exemplary components of a computing module in one embodiment;

[0020] Figure 4 In one embodiment Figure 1 A block diagram of exemplary functional components of a camera and / or video surveillance system;

[0021] Figure 5 In one embodiment Figure 1 A block diagram of exemplary functional components of a motion detector, face detector and / or sensor;

[0022] Figure 6 In one embodiment Figure 1 A block diagram of exemplary functional components of a monitoring station;

[0023] Figure 7 is a block diagram of a simplified monitoring system environment;

[0024] Fig. 8A and Figure 8B is a flow chart for handling real-time deviations in a video surveillance system in one embodiment;

[0025] Fig. 9A , Fig. 9B and Fig. 9C The diagram illustrates what an operator may see on a display before an event, during an event, and after an event in one embodiment;

[0026] Fig.10The graph illustrates the bit rate before an event, during an event, and after an event in one embodiment;

[0027] Fig.11 Graphs showing bit rates before an event, during an event, and after an event in another embodiment;

[0028] Fig.12 The diagram shows Fig.11 what an operator may see on a display during an event in an embodiment of; and

[0029] Fig.13 The graph illustrates the bit rate before an event, during an event, and after an event in another embodiment. DETAILED DESCRIPTION

[0030] The following detailed description refers to the accompanying drawings. The same reference numbers in different drawings identify the same or similar elements.

[0031] As noted above, an operator sits at a monitoring station to monitor the video streaming from a security camera. Problems may arise in the monitoring system when an "event" such as motion occurs in the monitored area. Motion may cause the video stream bit rate to temporarily increase because additional information needs to be encoded. This increase may result in lost packets because the increased bit rate may be higher than the network capacity. Therefore, the video may be compromised at the very moment that the operator is most interested. Some monitoring systems may reduce the quality of the video (e.g., by reducing the frame rate, image quality, or motion capture) to prevent the bit rate from increasing. This results in lower quality video at the very moment that the operator is most interested.

[0032] The embodiments disclosed below can enable a monitoring system to increase the video stream bit rate (e.g., including additional motion information) without unnecessarily overwhelming the network and dropping packets. Instead, the embodiments described below can enable an operator to watch an event, for example, in slow motion, at a higher, or at a bit rate.

[0033] Figure 1 1 is a block diagram illustrating an exemplary environment 100 in one embodiment. The environment 100 may be, for example, a surveillance system for a security area or providing public safety. Figure 1 As shown, environment 100 may include cameras 110-1 to 110-M, network 120, video management system (VMS) 150, monitoring stations 125-1 to 125-N, and / or displays 130-1 to 130-N. Environment 100 may also include motion detector 111, smoke detector 112, and / or sensor 113.

[0034] Cameras 110-1 through 110-M (a single "camera 110" or a plurality of "cameras 110") capture images and / or video of surveillance area 106. Area 106 may include a "live scene" or a "live event," meaning that the captured video stream represents the live scene or live event. Surveillance area 106 may be monitored by one or more cameras 110. For example, camera 110-1 and camera 110-2 monitor area 106-1, which includes object 102-1. Object 102 may include any object discernible in the video stream, such as an image, a door, a person, an animal, a vehicle, a license plate on a vehicle, and the like.

[0035] The camera 110 may capture image data using visible light, infrared light, and / or other non-visible electromagnetic radiation (e.g., ultraviolet light, far infrared light, megahertz radiation, microwave radiation, etc.). The camera 110 may include a thermal camera and / or a radar for radar imaging. The captured image data may include a continuous sequence of images (e.g., video), a limited sequence of images, still images, and / or combinations thereof. The camera 110 may include a digital camera for capturing and digitizing images and / or an analog camera for capturing images and storing the image data in an analog format.

[0036] The camera 110 may include a sensor that generates data (e.g., image data or video data) arranged in one or more two-dimensional arrays. As used herein, "video data," "video stream," and "video" may be more generally referred to as "image data" and "image," respectively. Therefore, "image data" or "image" is meant to include "video data," "video stream," and "video," unless otherwise stated. Similarly, "video data," "video stream," or "video" may include still images, unless otherwise stated.

[0037] Other monitoring devices or sensors can capture information from monitoring area 106. For example, motion detector 111 (e.g., which can include devices other than cameras) can detect motion in area 106-1. Motion detector 111 can include a proximity sensor, a magnetic sensor, an intrusion sensor, a pressure sensor, an infrared light sensor, a radar sensor, and / or a radiation sensor. Smoke detector 112 can detect smoke in area 106-1. Smoke detector 112 can also include a thermal sensor. Sensor 113 can include any type of sensor (including a proximity sensor) to determine when, for example, a door or window is opened or closed, or has been opened or closed.

[0038] Monitoring stations 125-1 through 125-N are coupled to displays 130-1 through 130-N (respectively, "monitoring stations 125" and "displays 130"). Monitoring stations 125 and displays 130 allow an operator (not present) to Figure 1100) can view images from camera 110. Each monitoring station 125 and display 130 can be a "client" for an operator to interact with the monitoring system shown in environment 100.

[0039] The display 130 may receive and display video streams from one or more cameras 110. A single display 130 may show images from a single camera 110 or from multiple cameras 110 (e.g., in multiple frames or windows on the display 130). A single display 130 may also show images from a single camera but in different frames. That is, a single camera may, for example, include a wide-angle or fisheye lens and provide images of multiple areas 106. Images from different areas 106 may be separated and shown separately on the display 130 in different windows and / or frames. The display 130 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, a cathode ray tube (CRT) display, a plasma display, a laser video display, an electrophoretic display, a quantum dot display, a video projector, and / or any other type of display.

[0040] The network 120 may include one or more circuit-switched networks and / or packet-switched networks. For example, the network 120 may include a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), an ad hoc network, an intranet, the Internet, a fiber-based network, a wireless network, and / or a combination of these or other types of networks.

[0041] The VMS 150 may include one or more computer devices, such as, for example, a server device, which coordinates the operation of the camera 110 and / or the display 130. The VMS 150 may receive and store image data from the camera 110. The VMS 150 may also provide a user interface that enables an operator of the monitoring station 125 to view image data stored in the VMS 150 or streamed from the camera 110. The VMS 150 may include a rule engine to receive notifications of events from the camera 110 and determine whether these notifications trigger a rule (e.g., satisfy a criterion) to perform an action (e.g., generate an alarm). For example, if the camera 110 detects motion in the area 106, the camera 110 may notify the VMS 150. As a result of the rule, the rule engine in the VMS 150 may send an alarm to the display 130 at the monitoring station 125 via the network 120. Events (and their notifications) may be triggered by devices and sensors in the environment 100 other than the camera 110. For example, the environment may include motion detectors, photodetectors, etc. that are capable of generating notifications of events.

[0042] In some embodiments, the environment 100 does not include a separate VMS 150. Instead, the services provided by the VMS 150 are provided by the monitoring stations 125 and / or the cameras 110 themselves and are provided in a manner distributed among the devices in the environment 100. For example, the cameras 110 may include a rules engine to determine when notification of an event triggers a rule to perform an action (e.g., to raise an alarm based on detected motion). Likewise, the VMS 150 may perform operations described as being performed by the cameras 110. For example, the VMS 150 may analyze image data to detect motion, rather than the cameras 110 doing so.

[0043] although Figure 1 Exemplary components of environment 100 are shown, but in other embodiments, environment 100 may include Figure 1 Fewer components, different components, differently arranged components, or additional components than those depicted in the drawings. Additionally or alternatively, any one device (or any group of devices) may perform functions described as being performed by one or more other devices.

[0044] Figure 2 is a block diagram illustrating exemplary components of camera 110 in one embodiment. Figure 2 As shown in , the camera 110 may include an optical chain 210, a sensor array 220, a bus 225, an image processor 230, a controller 240, a memory 245, a video encoder 250, and / or a communication interface 260. In one embodiment, the camera 110 may include one or more motor controllers 270 (e.g., three) and one or more motors 272 (e.g., three) for panning, tilting, and zooming the camera 110. The bus 225 is a communication path that enables the components in the camera 110 to communicate with each other.

[0045] The optical train 210 includes a housing that directs incident radiation (e.g., light, visible light, infrared light waves, millimeter waves, etc.) to the sensor array 220 to capture an image based on the incident radiation. The optical train 210 includes a lens 212 to collect and focus the incident radiation from the surveillance area onto the sensor array 220.

[0046] The sensor array 220 may include an array of sensors for aligning, sensing, and measuring radiation (e.g., light) incident on or falling on the sensor array 220. The radiation may be in the visible wavelength range, the infrared wavelength range, or other wavelength ranges. The sensor array 220 may include, for example, a charge coupled device (CCD) array and / or an array of activated pixels (e.g., a complementary metal oxide semiconductor (CMOS) sensor array). The sensor array 220 may also include a microbolometer (e.g., when the camera 110 includes a thermal camera or detector).

[0047] The sensor array 220 outputs data indicating radiation (e.g., light) incident on the sensor array 220 (e.g., describing properties or characteristics). For example, the data output from the sensor array 220 may include information such as the intensity (e.g., brightness), color, etc. of the light incident on one or more pixels in the sensor array 220. The light incident on the sensor array 220 may be an "image" because the light can be focused due to the lenses in the optical chain 210. The sensor array 220 can be considered an "image sensor" because it senses electromagnetic radiation (e.g., light) impinging on the sensor array 220 and converts the radiation into a multi-dimensional signal. As the term is used herein, an "image" includes data indicating radiation (e.g., describing properties or characteristics of light) incident on the sensor array 220. Therefore, the term "image" may also be used to mean "image sensor data" or any data or data set describing an image. In addition, a "pixel" may mean any area or region of the sensor array 220 for which a measurement of radiation (e.g., a measurement indicating light incident on the sensor array 220) is made. A pixel may correspond to one or more (or less than one) sensors in sensor array 220. In alternative embodiments, sensor 220 may be a linear array that may use scanning hardware (e.g., a rotating mirror) to form an image, or a non-array sensor that may rely on image processor 230 and / or controller 240 to generate image sensor data.

[0048] Video encoder 250 may encode image sensor data for transmission to other devices in environment 100, such as station 125 and / or VMS 150. Video encoder 250 may use video encoding techniques such as the video coding standards of ISO / MPEG or the ITU-H.26X series.

[0049] The image processor 230 and / or the controller 240 may perform signal processing operations on the image data captured by the sensor array 220. The image processor 230 and / or the controller 240 may include any type of single-core or multi-core processor, microprocessor, latch-based processor, and / or processing logic (or series of processors, microprocessors, and / or processing logic) that interprets and executes instructions. The image processor 230 and / or the controller 240 may include or be coupled to a hardware accelerator such as a graphics processing unit (GPU), a general purpose graphics processing unit (GPGPU), a unit, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or another type of integrated circuit or processing logic.

[0050] Controller 240 may also determine and control the desired focus and position (e.g., tilt and zoom) of camera 110. To do this, controller 240 sends commands to one or more motor controllers 270 to drive one or more motors 272 to tilt and / or translate camera 110 or optically zoom lens 212.

[0051] The memory 245 may include any type of volatile and / or non-volatile storage device that stores information and / or instructions. The memory 245 may include a random access memory (RAM) or any type of dynamic storage device, a read-only memory (ROM) device or any type of static storage device, a magnetic or optical recording memory device and its corresponding drive, or a removable memory device. The memory 245 may store information and instructions (e.g., applications and / or operating systems) and data (e.g., application data) for use by the processor camera 110.

[0052] The memory 245 may store instructions executed by the controller 240 and / or the image processor 230. The software instructions may be read into the memory 245 from another computer-readable medium or from another device. The software instructions may cause the controller 240 and / or the image processor 230 to perform the processing described herein. For example, the camera 110 may perform operations related to image processing (e.g., encoding, transcoding, detecting objects, etc.) in response to the controller 240 and / or the image processor 230 executing the software instructions stored in the memory 245. Alternatively, hard-wired circuits (e.g., logic) may be used in place of, or in combination with, the software instructions to implement the processing described herein.

[0053] The communication interface 260 includes circuits and logic circuits, which include input and / or output ports, input and / or output systems, and / or other input and output components that facilitate the transmission of data to another device. For example, the communication interface 260 may include a network interface card (e.g., an Ethernet card) for wired communication or a wireless network interface (e.g., WiFi) card for wireless communication.

[0054] The camera 110 (e.g., the memory 245, the image processor 230, the video encoder 250, and / or the controller 240) may implement a cache that acts as a buffer between the image pipeline and a streaming server that provides a video stream to a client (e.g., the monitoring station 125). The image pipeline operates in a steady state and pushes data into the cache. The streaming server may pull data from a cache (or buffer) that is used individually for each client. In one embodiment, the streaming server may pull data from the cache for eventual transmission to the client at a faster-than-real-time rate, in real-time, or slower than real-time.

[0055] although Figure 2 Exemplary components of camera 110 are shown, but in other embodiments, camera 110 may include Figure 2 10. Additionally or alternatively, one or more components of camera 110 may perform functions described as being performed by one or more other components of camera 110. For example, controller 240 may perform functions described as being performed by image processor 230 and vice versa. Alternatively or additionally, camera 110 may include the following: Figure 3 The computational module described.

[0056] Figure 3 1 is a block diagram illustrating exemplary components of a computing module 300 in one embodiment. Devices in environment 100 (e.g., VMS 150, display 130, motion detector 111, smoke detector 112, and / or sensor 113) may include one or more computing modules 300. Figure 3 As shown in , the computing module 300 may include a bus 310, a processor 320, a memory 330 and / or a communication interface 360. In some embodiments, the computing module 300 may also include an input device 340 and / or an output device 350.

[0057] Bus 310 includes a path that permits communication between components of computing module 300 or other devices. Processor 320 may include any type of single-core processor, multi-core processor, microprocessor, latch-based processor, and / or processing logic (or a series of processors, microprocessors, and / or processing logic) that interprets and executes instructions. Processor 320 may include an ASIC, an FPGA, and / or another type of integrated circuit or processing logic. Processor 320 may include or be coupled to a hardware accelerator such as a GPU, a GPGPU, a unit, an FPGA, an ASIC, and / or another type of integrated circuit or processing logic.

[0058] The memory 330 may include any type of volatile and / or non-volatile storage device that stores information and / or instructions. The memory 330 may include RAM or any type of dynamic storage device, ROM or any type of static storage device, a magnetic or optical recording memory device and its corresponding drive, or a removable memory device. The memory 330 may store information and instructions (e.g., applications and operating systems) and data (e.g., application data) for use by the processor 320.

[0059] Memory 330 may store instructions executed by processor 320. The software instructions may be read into memory 330 from another computer-readable medium or from another device. The software instructions may cause processor 320 to perform the processes described herein. Alternatively, hard-wired circuitry (e.g., logic) may be used in place of, or in combination with, software instructions to implement the processes described herein.

[0060] The operating system may include software instructions for managing the hardware and software resources of the computing module 300. For example, the operating system may include Linux, Windows, OS X, Android, embedded operating systems, etc. Depending on the device in which the particular computing module 300 is found, the applications and application data may provide network services or include applications.

[0061] The communication interface 360 ​​may include a transmitter and / or a receiver (e.g., a transceiver) that enables the computing module 300 to communicate with other components, devices, and / or systems. The communication interface 360 ​​may communicate via wireless communication (e.g., radio frequency, infrared, etc.), wired communication, or a combination thereof. The communication interface 360 ​​may include a transceiver that converts baseband signals to radio frequency (RF) signals or vice versa and may be coupled to an antenna.

[0062] The communication interface 360 ​​may include logic components including input and / or output ports, input and / or output systems, and / or other input and output components that facilitate the transmission of data to other devices. For example, the communication interface 360 ​​may include a network interface card (e.g., an Ethernet card) for wired communication or a wireless network interface (e.g., WiFi) card for wireless communication.

[0063] Some devices may also include input devices 340 and output devices 350. Input devices 340 may enable a user to input information into computing module 300. Input devices 370 may include a keyboard, mouse, pen, microphone, camera, touch screen display, and the like.

[0064] Output device 350 can output information to the user. Output device 350 can include a display, printer, speaker, etc. Input device 340 and output device 350 can enable a user to interact with applications executed by computing module 300. In the case of a "headless" device (such as a deployed remote camera), input and output are primarily performed through communication interface 360 ​​rather than input device 340 and output device 350.

[0065] The computing module 300 may include other components (not shown) that assist in receiving, transmitting, and / or processing data. Moreover, other configurations of components in the computing module 300 are possible. Figure 3300 may include fewer components, different components, additional components, or differently arranged components than depicted in . Additionally or alternatively, one or more components of computing module 300 may perform one or more tasks described as being performed by one or more other components of computing module 300.

[0066] Figure 4 4 is a block diagram of exemplary functional components of camera 110 and / or VMS 150 in one embodiment. Camera 110 and / or VMS 150 may include real-time deviation logic 402, channel capacity detector 404, encoder 406, buffer 408, rule table 410, rule engine 412, motion detection logic 414, and / or object detection logic 416.

[0067] The real-time deviation logic 402 determines when the camera 110 and / or VMS 150 may deviate from transmitting a video stream at a video stream bit rate (e.g., a real-time bit rate or speed). For example, if one or more qualifying conditions are met (e.g., motion in the video stream, a door opening, etc.), the real-time deviation logic 402 may determine to increase the bit rate of the video (or allow or enable an increase in the bit rate of the video), and to buffer the video to the extent that the video stream cannot be transmitted at the video stream bit rate, and transmit the video stream to the monitoring station 125 over a longer period of time (time stretching) for the operator to view on the display 130 (e.g., in slow motion).

[0068] The channel capacity detector 404 determines the capacity (e.g., the maximum transmission speed in bits per second) of the channel between the camera 110 and the monitoring station 125. A number of different methods may be used to determine the channel capacity. The channel capacity logic 404 may determine the channel capacity by transmitting data through the network 120 at an increasing speed to determine the transmission speed when packets are dropped. When packets begin to be dropped, the channel capacity has been exceeded. The data passed through the network 120 to measure its capacity may include a video stream. That is, when packets in the video stream begin to be dropped, the capacity of the network 120 has been reached. In one embodiment, the channel capacity may be measured or determined continuously. In one embodiment, the channel capacity may be assigned to the camera 110 or a particular video stream (e.g., without considering the actual capacity of the network 120). For example, during configuration, each camera 110 in the environment 100 may be provided with a channel capacity that each camera 110 should not exceed. In another embodiment, the channel capacity may be a dynamic threshold set by the camera 110 (e.g., by the encoder 406). For example, if the encoder 406 employs a variable bit rate (VBR) encoder, the channel capacity may be the maximum bit rate assigned to the VBR encoder. If encoder 406 employs a fixed bit rate (CBR) encoder, the channel capacity can be a fixed bit rate assigned to the CBR encoder. As described herein, during an event, encoder 406 can be enabled to provide a bit rate higher than the channel capacity, which can include enabling encoder 406 (e.g., during an event) to generate a stream with a bit rate that is higher than its assigned maximum bit rate (for a VBR encoder) or higher than its assigned fixed bit rate (for a CBR encoder). Channel capacity can also be an arbitrary bit rate threshold that the video stream must meet for any other reason.

[0069] The encoder 406 inputs image and / or video information and encodes it into a video stream for transmission to the monitoring station 125 via the network 120. The encoder 406 may use a variable bit rate encoding method. The encoder 406 may increase the bit rate according to the motion in the video. For example, if the camera 110 captures a video of a scene of a door and a person enters the scene, the encoder 406 may increase the bit rate to encode additional information about the person entering the scene. The encoder 406 may increase the bit rate (e.g., as a result of encoding motion) without having to change the quality of the video. Additional factors may cause the encoder 406 to increase the bit rate of the video stream. The encoder 406 may adjust the quality of the video by changing the "quality factor" of the video stream. The quality factor may include quantization parameters (for discrete cosine transform (DCT) coefficients), image resolution, frame rate, dynamic color gamut, no color (e.g., black and white), etc. Improved quality and / or additional information (such as motion) may result in a higher bit rate. Depending on the capacity of the network 120, the encoder 406 may encode the video stream at different bit rates. That is, higher channel capacity through network 120 can enable higher bit rates and higher quality video. On the other hand, lower channel capacity through network 120 can reduce the quality and bit rate of the video (e.g., to avoid lost packets during transmission).

[0070] When the video stream bit rate is higher than the transmission rate of the video stream through the network 120, the buffer 408 can store the video stream before transmission. For example, during events (such as sports events) that may require more information to be encoded, the video stream bit rate can be higher than the transmission rate. The memory 330 and / or the memory 245 may include a buffer 408. In one embodiment, the buffer 408 is an application layer buffer (e.g., relative to a network layer buffer). In this embodiment, the buffer 408 is an application layer buffer (which may be located in the kernel of the operating system) separated from the transmission control protocol (TCP) buffer. In addition, the buffer 408 can be used in a user datagram protocol (UDP), which may not be associated with a buffer in the kernel. In the application layer, the buffer 408 can be close to the capture of the image data, rather than being located in the kernel or the network (e.g., in a network device). In one embodiment, the buffer 408 can be between the image pipeline and the streaming server.

[0071] The rules table 410 stores actions that the rules engine 412 takes when the conditions of the rules are met. The rules engine 412 receives event notifications (e.g., from sensors), interprets the rules stored in the rules table 410, and performs actions when the conditions of the rules are met. For example, if motion is detected by the motion detection logic 414, the rules engine 412 can instruct the encoder 406 to allow a higher bit rate in the video stream, accept a real-time deviation mode, and / or increase the quality of the video stream to be transmitted to the monitoring station 125 and displayed on the display 130 (which may increase the bit rate). As another example, if the sensor determines that a door has been opened, the rules engine 412 can instruct the encoder 406 to allow a higher bit rate in the video stream (e.g., with or without changing the quality factor), accept a real-time deviation mode, and / or increase the quality of the video stream to be transmitted to the monitoring station 125 and displayed on the display 130 (which may increase the bit rate).

[0072] The motion detection logic 414 detects motion events and sends notifications of the motion events to other devices and / or the rules engine 412. For example, the motion detection logic 414 in the camera 110 may compare one image to the next image to detect motion. If motion (e.g., an event) is detected, the motion detection logic 414 may send an event notification to another device rules engine 412. In one embodiment, the motion detection logic 414 may determine a motion event by determining that the bit rate of the video stream has increased.

[0073] Object detection logic 416 may detect objects in a video stream. When a particular object (e.g., an event) is detected, object detection logic 416 may send an event notification to another device or to rules engine 412. Events may include: movement of an object, an object entering the camera's field of view, an object leaving the camera's field of view, realizing that an object belongs to a particular class of physical objects, or realizing that an object is the face of a particular person.

[0074] Figure 5 is a block diagram of exemplary functional components of motion detector 111, object detector 112, and / or sensor 113 in one embodiment. These devices may include object detection logic 502, motion detection logic 504, and / or sensor logic 506.

[0075] Object detection logic 502 may operate similarly to object detection logic 416, but may be separate from camera 110. Like object detection logic 416, object detection logic 502 may detect objects and / or faces in a video stream. When a particular object or face (e.g., an event) is detected, object detection logic 502 may send an event notification to another device or to rules engine 412, and rules engine 412 may instruct encoder 406 to allow a higher bit rate for the video stream (e.g., with or without changing the quality factor), accept a real-time deviation mode, and / or improve the quality of the video stream to be transmitted to monitoring station 125 and displayed on display 130.

[0076] The motion detection logic 504 may operate like the motion detection logic 414, but may be separate from the camera 110. The motion detection logic 504 detects motion events and sends notifications of the motion events to other devices and / or the rules engine 412. For example, motion (e.g., an event) is detected, the motion detection logic 504 may send an event notification to another device rules engine 412, and the rules engine 412 may instruct the encoder 406 to allow a higher bit rate for the video stream (e.g., without or without changing the quality factor), accept a deviation from real-time mode, and / or increase the quality of the video stream to be transmitted to the monitoring station 125 and displayed on the display 130.

[0077] The sensor logic 506 includes any type of sensor capable of generating an event. For example, if the sensor logic 506 determines that a door has been opened, the sensor logic 506 can send an event message to the rule engine 412, and the rule engine 412 can instruct the encoder 406 to allow a higher bit rate for the video stream, accept a real-time deviation mode, and / or improve the quality of the video stream to be transmitted to the monitoring station 125 and displayed on the display 130.

[0078] Figure 6 6 is a block diagram of exemplary functional components of monitoring station 125 in one embodiment. Monitoring station 125 may include decoder 602 and display logic 604. Decoder 602 decodes the video stream received via network 120 and displays the video on display 130. Display logic 604 notifies the operator when the video is displayed in slow motion or fast motion. In one embodiment, display logic 604 may also send a signal back to camera 110 and / or VMS 150 to request a stop of the deviation in real time.

[0079] Figure 7700 is a block diagram of a simplified monitoring system environment 700. The environment 700 includes a camera 110 for capturing images of an area 106 and encoding the images into a video stream for transmission over a network channel 702. Although the channel 702 is shown as a single line, the channel 702 may include a network (e.g., network 120). Thus, the channel 702 may be shared by multiple devices and may have multiple sections, each section being shared by a different device. The environment 700 also includes a monitoring station 125 having a display 130 for displaying the video stream to an operator 704 on the display 130. The environment 700 is simplified. Additional or different arrangements of components are possible, including the use of a VMS 150 for encoding and / or transcoding the video stream for transmission over the channel 702.

[0080] A video stream has a "bit rate" or "video stream bit rate". The video stream bit rate corresponds to the number of bits in the video stream per time unit in the captured scene. Thus, if a video stream uses 1 megabit (Mb) to encode 1 second of video, the bit rate you use for the video stream is 1 megabit per second (Mbps). The video stream is transmitted into the channel 702 at a "transmission speed" which can also be measured in bits per second. Thus, if a transmitter transmits 1 Mb of data in 1 second, the transmission speed is 1 Mbps. If the transmission speed of the video stream matches the video stream bit rate, then the transmission speed is considered to be the "real-time transmission speed". The rate at which the video stream is received (e.g., by a decoder) is roughly the same as the transmission speed (assuming that packets are not lost). The term "rate" is considered to be synonymous with "speed".

[0081] The transmission speed may be limited by the capacity of the channel 702 ("channel capacity"). Because the channel 702 is typically a network 120 with multiple segments shared by other devices (such as other cameras 110), the channel capacity may vary over time (e.g., as other devices come on and off-line or if another device temporarily requires a higher bit rate). The channel capacity may also be expressed in bits per second. If the transmission rate is higher than the channel capacity, packets may be dropped in the network and data may be lost.

[0082] If the video stream bit rate is higher than the channel capacity, the system can (1) attempt to transmit the video stream at a transmission rate higher than the channel capacity. In this case, the network 120 can drop packets and the decoder can present incomplete video. Alternatively, if the video stream bit rate is higher than the channel capacity, the system can (2) transmit at a rate lower than the video stream bit rate. In this case, the transmission speed is lower than the real-time speed, and the decoder can present a video that starts and stops abruptly because the decoder receives the video stream at a rate lower than the real-time rate. As another alternative, if the video stream bit rate is higher than the channel capacity, the system can (2) encode, transcode and / or re-encode the video to reduce the video stream bit rate to match the channel capacity. In this case, the quality of the video presented to the operator 704 may be lower.

[0083] Operator 704 typically has two criteria regarding viewing video occurring in area 106. First, operator 704 wants to view video of as high a quality as possible. Second, operator 704 wants to view what is happening "in real time" or "on the fly" with the understanding that there are transmission delays (in channel 702) and potential encoding and decoding delays. The ability of VMS 150 and / or camera 110 to meet these two criteria depends on the capacity of channel 702. Higher quality video typically means a higher video stream bit rate. Therefore, camera 110 and / or VMS 150 improves video quality until the video stream bit rate corresponds to the channel capacity (e.g., the video stream bit rate is the same as or lower than the channel capacity). Improving video quality even more will mean that packets will be lost or delayed, resulting in poor video quality being presented to operator 704.

[0084] Problems may arise in the system because the video stream bitrate may change (especially when the encoder uses a variable bitrate algorithm). For example, if a person enters area 106 (an "event") the video stream bitrate may temporarily increase because additional information (including motion) needs to be encoded. To prevent the video stream bitrate from increasing beyond the channel capacity, the system may encode the video to reduce the quality of the video at exactly the time when the operator 704 may wish not to reduce the quality of the video (but may wish to maintain and / or improve the quality). Disclosed herein are methods that may enable the system to maintain the quality of the video in a surveillance system in such circumstances (e.g., during an event).

[0085] Fig. 8A and Figure 8BFlowchart of a process for real-time deviation in a video surveillance system in one embodiment. Process 800A involves capturing, encoding, and transmitting a video stream. Process 800B involves receiving and displaying a video stream. Processes 800A and 800B (collectively, process 800) can be executed in or by camera 110 and / or VMS 150, monitoring station 125, and / or display 130. Process 800A can be executed in a device by a processor (e.g., processor 230, encoder 250, controller 240, and / or processor 320) executing instructions stored in a memory (e.g., memory 245 and / or memory 330). In combination Figure 7 , 9A to 9C ,and Fig.10 To describe process 800A. Fig. 9A , Fig. 9B and Fig. 9C The diagram illustrates what the operator 704 may see on the display 130 before an event, during an event, and after an event, respectively, in one embodiment. Fig.10 The graph illustrates the bit rate before an event, during an event, and after an event in one embodiment.

[0086] Process 800A begins with the capture and encoding of image and / or video data (block 802). In this example, camera 110 captures an image of area 106. In a surveillance system, camera 110 captures a "live" image of area 106 (e.g., a live scene or live event). To help determine the quality of a video stream, a channel capacity (e.g., a maximum transmission speed without packet loss) may be determined (block 804). A number of different techniques may be used to determine channel capacity. For example, channel capacity logic 404 may determine channel capacity by transmitting data through channel 702 at an increasing rate to determine the transmission speed at which packets are dropped. When packets begin to be dropped, the channel capacity has been exceeded. The data passed through channel 702 to measure its capacity may include a video stream. That is, when packets in the video stream begin to be dropped, the capacity of channel 702 has been reached. In one embodiment, the channel capacity may be measured or determined continuously. Fig.10The channel capacity 1002 is shown as a dotted line. In one embodiment, the channel capacity is determined when no event occurs (e.g., before the event). In other embodiments, the channel capacity logic 404 can assign channel capacity to the camera 110 or a specific video stream (box 804) (e.g., without considering the actual capacity of the network 120). For example, during configuration, each camera 110 in the environment 100 can be provided with a channel capacity that each camera 110 should not exceed. In another embodiment, the channel capacity can be a dynamic threshold set by the camera 110 (e.g., by the channel capacity logic 404 and / or the encoder 406). For example, if the encoder 406 adopts a variable bit rate (VBR) encoder, the channel capacity can be the maximum bit rate assigned to the VBR encoder. If the encoder 406 adopts a fixed bit rate (CBR) encoder, the channel capacity can be a fixed bit rate assigned to the CBR encoder.

[0087] The quality factor of the video stream can be adjusted based on the channel capacity (block 805). For example, if channel 702 is capable of carrying additional data, the resolution of the image can be increased (and the video stream bit rate increased). As another example, if channel 702 is capable of carrying additional data, the quantization parameters of the DCT coefficients can be changed to allow for a larger number of levels (and an increased video stream bit rate). On the other hand, if channel 702 is dropping packets, the resolution of the image can be reduced (and the video stream bit rate reduced). Alternatively, if channel 702 is dropping packets, the quantization parameters of the DCT coefficients can be changed to reduce the number of levels (and reduce the video stream bit rate). With the quality factor determined, process 800A can continue to capture and encode image and video data (block 806) and transmit the video stream (at real-time speed) over channel 702 (block 807). The process (e.g., blocks 804 to 807) can continue as the video is captured and transmitted. Process 800B (see Figure 8B ) may begin by receiving a video stream from channel 702 at real-time speed (block 852) and displaying the video at real-time speed (block 854).

[0088] Fig.10 Graphs of video stream bitrate 1004 (top) and transmission rate 1006 (bottom) are shown before an event, during an event, and after an event. In this example, before event start time 1010, the video stream bitrate (section 1014) closely follows the channel capacity 1002. At this time, because the video stream bitrate is less than the channel capacity, the video stream is transmitted at its real-time speed (box 807) (see Fig.10 Monitoring station 125 receives (block 852) the video stream (at real time rate) and Fig. 9A The video stream is displayed (block 854) on the display 130 as shown in FIG. Fig. 9A Display 130 is shown with window 920A indicating that area 106 (from camera 110) is shown "live". The video in window 920A may also be displayed (block 854) at normal motion speed (e.g., not slow motion or fast motion). This may continue until event start time 1010.

[0089] If an event is detected (block 808), process 800A (e.g., blocks 804 through 807) may be interrupted. The event of interest may include motion, object detection, and / or facial detection, for example, by motion detection logic 414 or 502, object detection logic 416 or 504, and / or sensor logic 506. Detecting an event may include determining that the bit rate of the video stream has increased, such as an increase caused by motion in an image of the video. During the event, it may be desirable to increase the bit rate of the video stream (e.g., due to motion) (block 810). For example, if Fig. 9B As shown in , person 910 has entered area 106, which corresponds to an event at event start time 1010. Fig.10 , the video stream bitrate 1004 increases to a rate (segment 1016) above the channel capacity 1002. In one embodiment mentioned above, during an event, the encoder 406 may be enabled to provide a bitrate above the channel capacity, which may include enabling the encoder 406 (e.g., during an event) to generate a stream having a bitrate above its assigned maximum bitrate (for a VBR encoder) or above its assigned fixed bitrate (for a CBR encoder).

[0090] Because the video stream bit rate is higher than the channel capacity, transmitting the video stream at the real-time rate may result in lost packets. Therefore, the video stream may be transmitted at a bit rate lower than the real-time bit rate (block 812). Because the video stream is transmitted at a rate lower than the real-time rate, more time is spent transmitting the video stream (compared to the corresponding video recording time). Fig.10 , during transmission, the time segment 1016 during the event is extended (to a corresponding segment 1018). At this time, the video stream can be buffered (frame 813) to avoid the loss of packets during the event. Transmitting the video stream at a rate lower than the real-time transmission rate introduces a time extension 1025. The real-time deviation logic 402 can use a buffer 408 to buffer the video stream. Because the buffer is an application layer buffer (rather than a network layer buffer), when the video stream bit rate is higher than the channel capacity, packets will not be discarded. In one embodiment, the transmitted video stream can include information indicating to the decoder 602 that the video stream is time-extended. In another embodiment, the decoder 602 can infer the time extension when there is no additional information added to the video stream.

[0091] Process 800B continues with receiving a video stream at a slower-than-real-time rate (block 856) and displaying the video in slow motion (block 858). Monitoring station 125 receives the video stream at a slower-than-real-time transmission rate. Rather than displaying a video that starts and stops abruptly, monitoring station 125 displays the video in slow motion. Fig. 9B , window 920B on display 130 indicates to operator 704 that the video is being shown in slow motion (e.g., at 90% of real time speed) and that the scenes being shown have been cumulatively delayed by an additional 1 second (e.g., to prepare for a higher bit rate). Slow motion at a higher bit rate may continue until the end of the event. In one embodiment, operator 704 may select Fig. 9B The video stream may be received at a rate lower than real time (e.g., causing the decoder 602 to display the video in slow motion). In another embodiment, the video stream may include information interpreted at the monitoring station 125 (e.g., added by the encoder 406) that causes the decoder 602 to display the video in slow motion.

[0092] When the end of the event is detected or determined (block 814), the bit rate of the video stream may be reduced (block 816). The end of the event may include, for example, a lack of motion, no object detected, and / or no facial recognition by motion detection logic 414 or 502, object detection logic 416 or 504, and / or sensor logic 506. For example, person 910 may have left the scene, such as Fig. 9C , which corresponds to after the event end time 1012. Determining that the event has ended may also include determining that the bit rate of the video stream has decreased, such as a decrease caused by a lack of motion in the image of the video. Fig.10 As shown in , the video stream bit rate is reduced to a range where there is additional channel capacity (e.g., the difference between the video stream bit rate 1004 and the channel capacity 1002 during the segment 1020). When the transmitter transmits this portion of the video stream (e.g., the section 1020), the video stream can be transmitted at a rate higher than the real-time transmission rate (block 818) until the buffer can be emptied (block 820). Transmitting at a rate higher than the real-time transmission rate compensates for the time stretch 1025 introduced earlier during the event. In one embodiment, the transmitted video stream can include information indicating to the decoder 602 that the video stream is transmitted to compensate for the stretched time 1025. In another embodiment, the decoder 602 can infer this without additional information added to the video stream.

[0093] Process 800B continues with receiving a video stream at a higher than real-time rate (block 860) and displaying the video in fast motion (block 862). Monitoring station 125 receives the video stream at a higher than real-time rate. Instead of displaying the video at the real-time rate, monitoring station 125 displays the video in fast motion. Fig. 9C , window 920C on display 130 indicates to operator 704 that the video is being shown in fast motion (e.g., at 110% of real time speed) and that the scenes being shown have been cumulatively delayed by an additional 0.5 seconds (e.g., to prepare for a higher bit rate). Fast motion at a lower bit rate may continue until the buffer is emptied (block 820) and / or monitoring station 125 begins receiving the video stream at a transmission rate corresponding to the real time rate.

[0094] After the buffer is empty, the process 800A can start again with a video stream bit rate corresponding to the real-time transmission rate. After determining that the event is over (block 814) and before the buffer is emptied (block 820), another event can be detected (block 808), and blocks 810-813 can be repeated (e.g., increasing the bit rate of the video stream (block 810), transmitting the video stream at a bit rate lower than the real-time rate (block 812), buffering the video stream (block 813), and playing the video in slow motion (block 858)).

[0095] like Fig.10 As shown in , the transmitted video stream has a first bit rate before the event (segment 1014) and a second bit rate during the event (segment 1016). In one embodiment, the second bit rate is higher than the first bit rate. The transmitted video stream has a third bit rate after the event (segment 1020). The third bit rate corresponds to a reduced bit rate, and in one embodiment the third bit rate is lower than the first bit rate. Reducing the bit rate of the video stream may include reducing the quality factor of the video stream. Reducing the quality factor of the video stream may include reducing the frame rate in the video stream, reducing the quantization parameter and / or reducing the resolution of the image.

[0096] Fig.11 The figure shows the bit rate before the event, during the event, and after the event in another embodiment. Fig.11As shown, during an event, the video stream may be split into two: a base layer 1102 and an enhancement layer 1104. The base layer 1102 is a lower quality version of the video that can be enhanced with the enhancement layer 1104 to achieve a high quality version of the video. The base layer 1102 may be transmitted at a real-time transmission rate, while the enhancement layer 1104 may be transmitted at a rate lower than the real-time transmission rate. In this case, the buffer 408 may be filled with the enhancement layer 1104. Thus, the monitoring station 125 may display both the higher quality (and slow motion) version of the video while displaying the lower quality (actual or normal motion speed) version of the video. After the event has ended, the buffer 408 may be empty and transmitted to the monitoring station 125 at the channel capacity. Fig.12 The diagram shows Fig.11 What the operator can see on the display during an event in an embodiment of Fig.12 106 , the display 130 includes a small window 1122 having a real-time (normal motion) lower quality view of the area 106 . The display 130 also includes a larger window 1122 having a slow motion, high quality view of the area 106 .

[0097] Fig.13 The figure shows the bit rate before the event, during the event, and after the event in another embodiment. Fig.13 As shown in (similar to Fig.11 ), during an event, the video stream may be split into two: a base layer 1102 and an enhancement layer 1104. Fig.13 In the case of , the base layer 1102 can be transmitted at the real-time transmission rate, while the enhancement layer 1104 can be transmitted at a rate lower than the real-time transmission rate without interrupting the low-quality transmission (such as in Fig.11 Thus, monitoring station 125 may display both the higher quality (and slow motion) version of the video while displaying the lower quality (actual or normal motion speed) version of the video.

[0098] In one embodiment, if the video stream includes audio, the audio may be time stretched (e.g., when the video is time stretched) or time compressed (e.g., when the video is time compressed) in a manner that the audio pitch does not change. That is, even though the video may be played in slow motion and / or fast motion by the monitoring station 125, the audio pitch is not shifted downward or upward. The adjustment of the audio (if implemented) may be performed by the decoder 602 in the monitoring station 125. Alternatively, the adjustment of the audio may be performed by the encoder 406 in the camera 110 and / or the VMS 150.

[0099] The monitoring system (such as operator 704 in environment 100) can exclude audio from the video stream. For example, operator 704 can view more than one video stream on display 130. In this case, playing more than one audio stream can be confusing to operator 704. Therefore, camera 110 and / or VMS 150 can include (e.g., at the source) the ability to omit audio from any video stream. The audio may also not be ideal for other reasons (e.g., the audio may not be relevant or to save bandwidth). The methods and systems described above can exclude audio from the video stream (in which case, pitch adjustment of the audio will not be necessary during slow motion or fast motion).

[0100] In another embodiment, the method and system as described above can be applied to an audio stream that is not accompanied by a video stream. That is, when the audio stream bit rate exceeds a threshold (or when an event is detected) (similar to box 808), the audio bit rate can then be transmitted at a rate lower than the real-time rate (similar to boxes 810 to 813) (e.g., when the buffer is used). When the event ends (similar to box 814), the audio bit rate can be reduced and transmitted at a rate higher than the real-time rate (similar to boxes 816 to 820).

[0101] As disclosed above, the methods and systems described herein can enable a surveillance system to increase the video stream bit rate (e.g., to include additional motion information or to improve quality) without unnecessarily overwhelming the network and dropping packets. Instead, the embodiments described below can enable an operator to watch an event, for example, in slow motion, at a higher quality, at a higher bit rate. These methods and systems can also reduce the quality of the video (e.g., by reducing the frame rate, image quality, or motion capture) to prevent the bit rate from increasing.

[0102] In the previous description, various preferred embodiments have been described with reference to the accompanying drawings. However, it will be apparent that various modifications and changes may be made thereto, and additional embodiments may be implemented without departing from the broad scope of the invention as set forth in the appended claims. Therefore, the description and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0103] For example, although a series of blocks have been described with respect to a diagram having a flow, in other implementations, the order of the blocks may be modified. In addition, non-dependent blocks may be executed in parallel.

[0104] It is apparent that the systems and / or methods described above may be implemented in a number of different forms of software, firmware, and hardware in the embodiments illustrated in the figures. The actual software code or specialized control hardware used to implement these systems and methods does not limit the embodiments. Thus, the operation and behavior of the systems and methods are described without reference to specific software code—it is understood that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0105] In addition, some parts as described above can be implemented as components that perform one or more functions.Components as used herein may include hardware such as a processor, ASIC, or FPGA, or a combination of hardware and software (e.g., a processor that executes software).

[0106] It should be emphasized that when the term "include / comprising" is used in this specification, it is adopted to specify the existence of stated features, integers, steps, or components, but does not exclude the existence or additional presence of one or more other features, integers, steps, components or their groups.

[0107] No element, action, or instruction used in this application should be understood as being critical or essential to the embodiments unless so explicitly described. Moreover, as used herein, the article "a" is intended to include one or more items. In addition, the phrase "based on" is intended to mean "based at least in part on," unless otherwise explicitly stated.

Claims

1. A method for encoding captured live video data for transmission, comprising: transmitting the encoded video bitstream via a network transmission channel, wherein the network transmission channel is associated with a maximum transmission bit rate; detecting an event associated with the live video data captured by a camera, wherein the event has a start time and an end time; Prior to the start time of the event, the method comprises: encoding the captured live video data into an encoded video bitstream set to have a first output video bitstream bitrate, the first output video bitstream bitrate being lower than the maximum transmission bitrate of the network transmission channel, and transmitting the encoded video bitstream having the first output video bitstream bitrate through the network transmission channel at a transmission bitrate lower than the maximum transmission bitrate, thereby achieving real-time communication; During the event, the method includes: encoding the captured live video data into an encoded video bitstream set to have a second output video bitstream bitrate, the second output video bitstream bitrate being higher than the maximum transmission bitrate of the network transmission channel, buffering the encoded video bitstream having the second output video bitstream bitrate, and transmitting the encoded video bitstream having the second output video bitstream bitrate over the network transmission channel at a transmission bitrate lower than the second output video bitstream bitrate, thereby introducing a temporal stretch into the encoded video bitstream having the second output video bitstream bitrate; and After the end time of the event, the method comprises: encoding, by an encoder, the captured live video data into an encoded video bitstream set to have a third output video bitstream bitrate, the third output video bitstream bitrate being lower than the maximum transmission bitrate of the network transmission channel, and The encoded video bitstream having the third output video bitstream bitrate is transmitted over the network transmission channel at a transmission bitrate higher than the third output video bitstream bitrate until a buffer is emptied, thereby catching up to compensate for the temporal stretch introduced during the event.

2. The method according to claim 1, wherein: The buffer is an application layer buffer.

3. The method according to claim 1, further comprising: The maximum transmission bit rate of the network transmission channel is determined.

4. The method according to claim 1, wherein: During the event, the method further comprises: receiving, at a monitoring station, said encoded video bitstream having said second output video bitstream bitrate; decoding the encoded video bitstream having the second output video bitstream bitrate into video; and The video is displayed on a display in slow motion corresponding to the time stretch.

5. The method according to claim 4, wherein: After the end time of the event, the method further comprises: receiving, at said monitoring station, said encoded video bitstream having said third output video bitstream bitrate; decoding said coded video bitstream having said third output video bitstream bitrate into said video, and The video is displayed on a display in fast motion.

6. The method according to claim 1, in, The second output video bitstream bit rate is higher than the first output video bitstream bit rate.

7. The method according to claim 1, in, encoding the captured live video data into the encoded video bitstream set to have the first output video bitstream bitrate prior to the start time of the event includes adjusting a quality factor based on the maximum transmission bitrate of the network transmission channel, The quality factor includes a frame rate, a quantization parameter or an image resolution.

8. The method according to claim 1, wherein: Detecting an event associated with the video bitstream comprises: detecting motion in video of the coded video bitstream, detecting a bit rate increase in the encoded video bitstream, Receive event messages in the control system, or An event message is received in a physical access control system.

9. An apparatus for encoding captured live video data, comprising: a processor for detecting events associated with live video data captured by the camera; an encoder configured to encode the live video data into an encoded video bitstream; a memory configured to buffer the encoded video bitstream; as well as a transmitter for transmitting the encoded video bitstream via a network transmission channel, wherein the network transmission channel is associated with a maximum transmission bit rate; Wherein, before the processor detects the event: The encoder is configured to encode the captured live video data into an encoded video bitstream having a first output video bitstream bitrate, the first output video bitstream bitrate being lower than the maximum transmission bitrate of the network transmission channel, and The transmitter is configured to transmit the encoded video bitstream having the first output video bitstream bitrate at a transmission bitrate through the network transmission channel, thereby achieving real-time communication; During the events described: The encoder is configured to encode the live video data into an encoded video bitstream having a second output video bitstream bitrate, the second output video bitstream bitrate being higher than the maximum transmission bitrate of the network transmission channel, The memory is configured to buffer the encoded video bitstream having the second output video bitstream bitrate, and The transmitter is configured to transmit the encoded video bitstream having the second output video bitstream bitrate at a transmission bitrate lower than the second output video bitstream bitrate, thereby introducing a temporal stretch into the encoded video bitstream having the second output video bitstream bitrate, and Wherein, after the event described: The encoder is configured to encode the captured live video data into an encoded video bitstream having a third output video bitstream bitrate, the third output video bitstream bitrate being lower than the maximum transmission bitrate of the network transmission channel, and The transmitter is configured to send the encoded video bitstream having the third output video bitstream bitrate over the network transmission channel at a transmission bitrate higher than the third output video bitstream bitrate until the memory is emptied, thereby catching up to compensate for the time stretch introduced during the event.

10. The device according to claim 9, wherein: The memory is configured as a buffer that is an application layer buffer.

11. The device according to claim 9, in, The processor is configured to determine a maximum transmission bit rate for the network transmission channel.

12. The apparatus according to claim 9, further comprising: a receiver for acquiring, at a monitoring station, said encoded video bitstream having said second output video bitstream bitrate; a decoder for decoding said encoded video bitstream having said second output video bitstream bitrate into video, and A display is configured to display the video in slow motion corresponding to the time stretch during the event.

13. The device according to claim 12, in, The display is configured to display the video corresponding to after the event on the display in fast motion.

14. The device according to claim 9, in, The second output video bitstream bit rate is higher than the first output video bitstream bit rate; or The bit rate of the third output video bit stream is lower than the bit rate of the first output video bit stream.

15. The device according to claim 9, in, The processor is configured to adjust the quality factor by reducing the frame rate, quantization parameter or resolution of the image; or The processor is configured to detect events associated with the live video data by detecting motion in the live video data, detecting a bit rate increase in the encoded video bitstream having the first output video bitstream bit rate, receiving an event message in a control system, or receiving an event message in a physical access control system.

Images