Method, system and program for detecting scene transitions in video footage

Abstract

A method and system for detecting scene transitions in video footage is presented. One example involves obtaining, for each of a plurality of distinct frames of the video footage, a confidence measure value associated with a detected object in the frame, and detecting a scene transition in the video footage based on the obtained confidence measure value.

Description

[Technical field]

[0001] The present invention relates generally to video processing, and more particularly to detecting scene transitions in video footage. [Background technology]

[0002] Scene detection is a subset of video processing that prepares video footage for indexing and retrieval. Transitions between scenes (such as "dissolves" or "cross fades") pose challenges to existing scene detection algorithms because the low-level information used to identify scene changes (e.g., color histograms, brightness, contrast) typically do not generalize well across a wide range of video footage. For example, dissolve transitions between scenes that have similar contrast, brightness, color, etc. can be difficult to detect.

[0003] In addition, existing scene detection algorithms are usually standalone and need to run independently of other video processing tasks. Thus, existing scene detection methods may exhibit significant processing requirements for large-scale videos. Thus, there is a need in the art to address the aforementioned problems. Summary of the Invention

[0004] According to a first aspect, the present invention provides a computer-implemented method for detecting scene transitions in video footage, the method comprising obtaining, for each of a plurality of different frames of the video footage, a value of a confidence measure associated with a detected object in the frame, and detecting a scene transition in the video footage based on the obtained value of the confidence measure.

[0005] According to a further aspect, the present invention provides a computer program product including a computer readable storage medium having program instructions embodied thereon, the program instructions being executable by a processing unit when executed on a computer network to cause the processing unit to perform a method for detecting scene transitions in video footage, the method comprising the steps of obtaining, for each of a plurality of different frames of the video footage, a value of a confidence measure associated with a detected object in the frame, and detecting a scene transition in the video footage based on the obtained value of the confidence measure.

[0006] According to a further aspect, the present invention provides a system for detecting scene transitions in video footage comprising one or more processors and one or more tangible storage media storing programming instructions for execution by the one or more processors, the programming instructions including instructions for obtaining, via an interface, for each of a plurality of distinct frames of the video footage, a value of a confidence measure associated with a detected object in the frame, and instructions for detecting a scene transition in the video footage based on the obtained value of the confidence measure.

[0007] According to a further aspect, the present invention provides a computer program product for detecting scene transitions in video footage, comprising a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method implementing steps of the present invention.

[0008] According to a further aspect, the present invention provides a computer program stored on a computer readable medium and loadable into an internal memory of a digital computer, the computer program comprising software code portions for performing the steps of the present invention when said program is executed on a computer.

[0009] The invention also relates to a computer program product comprising computer readable program code enabling a processor of a system or a plurality of processors of a network to implement such a method. The invention further relates to a computer system comprising at least one processor and such a computer program product, the at least one processor being configured to execute the computer readable program code of said computer program product. The invention also relates to a system for detecting scene transitions in video footage. The invention seeks to provide a computer implemented method for detecting scene transitions in video footage.

[0010] The invention further seeks to provide a computer program product comprising computer program code for implementing the proposed method when executed by a processing unit.

[0011] The present invention also seeks to provide a processing system configured to execute this computer program code.

[0012] The present invention also seeks to provide a system for detecting scene transitions in video footage.

[0013] According to one aspect of the invention, there is provided a computer-implemented method for detecting scene transitions in video footage, the method including obtaining, for each of a plurality of distinct frames of the video footage, a value of a confidence measure associated with a detected object in the frame, the method then further including detecting a scene transition in the video footage based on the obtained value of the confidence measure.

[0014] According to yet another aspect of the present invention, there is provided a system for detecting scene transitions in video footage, the system including an interface component configured to obtain, for each of a plurality of distinct frames of the video footage, a value of a confidence measure associated with a detected object in the frame, the system further including a detection component configured to detect a scene transition in the video footage based on the obtained value of the confidence measure.

[0015] According to another aspect of the present invention, there is provided a computer program product for detecting scene transitions in video footage, the computer program product comprising a computer readable storage medium having program instructions embodied thereon, the program instructions being executable by a processing unit to cause the processing unit to perform a method according to the proposed embodiments.

[0016] According to another aspect of the present invention, there is provided, according to one embodiment, a processing system comprising at least one processor and a computer program product, the at least one processor configured to execute computer program code of said computer program product.

[0017] Preferred embodiments of the invention will now be described, by way of example only, with reference to the following drawings: [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram of an example system in which aspects of the example embodiments may be implemented. [Diagram 2] 1 is a simplified block diagram of an example embodiment of a system for detecting scene transitions in video footage. [Diagram 3] 1 is an example diagram of how a confidence associated with a detected object varies between frames of video footage, according to an illustrative embodiment. [Figure 4]1 is a simplified block diagram of an example embodiment of a system for detecting scene transitions in video footage. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] It should be understood that the drawings are only schematic and are not drawn to scale, and that the same reference numbers are used throughout the drawings to denote the same or similar parts.

[0020] In the context of the present application, where embodiments of the invention constitute methods, it is to be understood that such methods may be processes for execution by a computer, i.e. computer-implemented methods, and thus the various steps of the method may reflect various portions of a computer program, e.g. portions of one or more algorithms.

[0021] Also, in the context of this application, a system may be a single device or a collection of distributed devices configured to perform one or more embodiments of the method of the present invention. For example, a system may be a personal computer (PC), a server or a collection of PCs or servers connected via a network such as a local area network, the Internet, or any combination thereof, for cooperatively performing at least one embodiment of the method of the present invention.

[0022] A concept is proposed for detecting scene transitions in video footage. In particular, it is proposed that scene transitions in video footage are detected based on the values ​​of a confidence measure associated with detected objects for different frames of the video footage. In other words, the change in confidence for detected objects between frames of the video can be used to identify the presence of a scene transition in the video.

[0023] The inventors propose the concept of analyzing the confidence of detected objects between frames of a video in order to infer or identify the presence of a transition in the video. Hence, the proposed embodiment can be provided as an extension to existing video processing pipelines that detects and indicates the confidence associated with an object in a video.

[0024] Such a proposal may eliminate the need for manual human tuning, which is currently a requirement in many conventional methods. As an example, the proposed embodiments may be integrated into known cost function approaches for detecting scene transitions to increase the robustness of the methods for detecting scene transitions. The embodiments may also be implemented with existing frame-wise object detectors.

[0025] To detect scene transitions in a video footage, it is proposed that values ​​of confidence measures associated with detected objects for different frames of the video footage are analyzed. For example, confidence scores associated with detected objects between frames can be compared to identify scene transitions in a relatively simple manner. In an efficient and robust manner, the embodiment can reliably detect dissolve type scene transitions, unlike conventional approaches that do not consider dissolve type scene transitions.

[0026] The proposed concept is able to exploit high-level semantic information to provide an efficient scene transition detection approach, which allows integration with existing video processing methods / algorithms.

[0027] By way of example, we propose that for each of a number of different frames of video footage, a value of a confidence measure is obtained that is associated with a detected object in the frame. Scene transitions in the video footage can then be detected based on the obtained values ​​of the confidence measure.

[0028] Thus, embodiments may provide the advantage that they are robust to dissolving transitions between scenes that have similar low-level pixel information.

[0029] Another advantage provided by the proposed concept is that raw video frames may not need to be inspected by the embodiments. Instead, the embodiments may employ the output of conventional object detection. As a result, the embodiments may be employed in conjunction with existing video processing algorithms that are already configured to detect objects in video footage and provide a confidence level associated with the detected objects. Such embodiments may thus leverage information provided by object detection algorithms / components and thus complement existing / conventional video processing algorithms / systems.

[0030] Thus, a simple integration and dynamic use of existing and / or new object detection algorithms during video processing can be realized by the proposed embodiments.

[0031] Thus, the proposed embodiments may provide concepts for modifying and / or supplementing the functionality of video processing algorithms. For example, the embodiments may be used to integrate new or additional scene transition algorithms into existing video processing systems. Thus, the embodiments may help provide improved or extended capabilities to existing video processing implementations. Thus, the embodiments may enhance the value of video processing systems by leveraging information about detected objects in video footage to provide modified and / or extended scene transition detection capabilities. Thus, the embodiments may help provide improved or extended capabilities to existing scene detection and / or video processing implementations.

[0032] In one embodiment, detecting scene transitions based on the obtained values ​​of the confidence measure may include analyzing the obtained values ​​of the confidence measure to identify trends or patterns in the values ​​of the confidence measure across multiple different frames of the video footage, and identifying scene transitions in the video footage based on the identified trends or patterns in the confidence values. For example, identifying scene transitions may include comparing the identified trends or patterns in the confidence values ​​to a predefined trend or pattern, and identifying scene transitions in the video footage based on the results of the comparison. Thus, relatively simple data / trend analysis techniques may be employed by the proposed embodiments, thereby minimizing the cost and / or complexity of the implementation.

[0033] Moreover, embodiments may also classify identified scene transitions based on identified trends or patterns in the confidence values. Thus, embodiments may not only detect the presence of a scene transition, but may also identify the type or classification of the detected scene transition. For example, a smooth, gradual decrease in the confidence associated with a detected object across successive frames of the video may be used to infer that the scene transition is a "dissolve" transition, whereas a sudden step-like change in the confidence associated with a detected object across successive frames of the video may be used to infer that the scene transition is a "cut" or "fast fade" transition.

[0034] In some embodiments, detecting a scene transition based on the obtained value of the confidence measure may include determining the presence of a scene transition in the video footage in response to the obtained value of the confidence measure decreasing across successive frames of the video footage. In other words, it is proposed that when a scene slowly fades out or dissolves out as part of a transition, the value of the confidence measure associated with detected objects in the scene also decreases along with the fading / dissolving of the scene. Thus, relatively simple data value analysis techniques may be implemented to determine the presence of a scene transition in the video footage.

[0035] In one embodiment, detecting a scene transition based on the obtained value of the confidence measure may include determining the presence of a scene transition in the video footage in response to the obtained value of the confidence measure increasing across successive frames of the video footage. It is proposed that when a scene slowly fades in or dissolves in as part of the transition, the value of the confidence measure associated with detected objects in the scene will also increase along with the fading / dissolving in of the scene. Thus, again, relatively simple data value analysis techniques may be implemented to determine the presence of a scene transition in the video footage.

[0036] Some embodiments may further include comparing the obtained value of the confidence measure to a threshold value and identifying at least one of a start and an end of a scene transition based on a result of the comparison. By way of example, identifying the end of the scene transition may include determining a frame of the video footage at which the obtained value of the confidence measure decreases from a value above the threshold value toward a value below the threshold value and determining the end of the scene transition based on the determined frame. Similarly, identifying the start of the scene transition may include determining a frame of the video footage at which the obtained value of the confidence measure increases from a value below the threshold value toward a value above the threshold value and determining the start of the scene transition based on the determined frame. Thus, simple data value comparison techniques may be employed to accurately infer the start and / or end of a scene transition.

[0037] In some embodiments, obtaining the confidence measure value may include obtaining a first confidence measure value associated with the detected object in a first frame of the video footage and obtaining a second confidence measure value associated with the detected object in a subsequent second frame of the video footage, where the first and second frames of the video footage may be, for example, consecutive frames of the video footage.

[0038] 1 is a block diagram of an example system 200 in which aspects of the example embodiments may be implemented. System 200 is an example of a computer, such as a client in a distributed processing system, in which computer usable code or instructions implementing processes for example embodiments of the present invention may be located. For example, system 200 may be configured to implement an interface component and a detection component in accordance with an embodiment.

[0039] In the illustrated example, system 200 employs a hub architecture including a north bridge and memory controller hub (NB / MCH) 202 and a south bridge and input / output (I / O) controller hub (SB / ICH) 204. A processing unit 206, a main memory 208, and a graphics processor 210 are connected to the NB / MCH 202. The graphics processor 210 may be connected to the NB / MCH 202 through an accelerated graphics port (AGP).

[0040] In the illustrated example, a local area network (LAN) adapter 212 connects to the SB / ICH 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read only memory (ROM) 224, a hard disk drive (HDD) 226, a CD-ROM drive 230, a universal serial bus (USB) port and other communication ports 232, and a PCI / PCIe device 234 connect to the SB / ICH 204 through a first bus 238 and a second bus 240. PCI / PCIe devices can include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. PCI uses a card bus controller, whereas PCIe does not. The ROM 224 can be, for example, a flash basic input / output system (BIOS).

[0041] The HDD 226 and the CD-ROM drive 230 are connected to the SB / ICH 204 through a second bus 240. The HDD 226 and the CD-ROM drive 230 may use, for example, an integrated drive electronics (IDE) or a serial advanced technology attachment (SATA) interface. A super I / O (SIO) device 236 may be connected to the SB / ICH 204.

[0042] An operating system executes on processing unit 206. The operating system coordinates and controls various components in system 200 (or system 300 in FIG. 2). As a client, the operating system may be a commercially available operating system. An object-oriented programming system, such as a Java™ programming system, may execute in conjunction with the operating system to make calls to the operating system from Java™ programs or applications executing on system 200. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and / or its subsidiaries.

[0043] As a server, system 200 may be, for example, an IBM® eServer™ System P5p® or System P5® computer system running the AIX® (Advanced Interactive Executive) operating system or the LINUX® operating system. System 200 may be a symmetric multiprocessor (SMP) system including multiple processors in processing unit 206. Alternatively, a single processor system may be employed. IBM eserver, System p5, and AIX® are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. The registered trademark Linux® is used pursuant to a sublicense from the Linux Foundation, the exclusive licensee Linus Torvalds, owner of the trademark worldwide. Instructions for the operating system, programming system, and application or program may be located in a storage device, such as HDD 226, and loaded into main memory 208 for execution by processing unit 206. Similarly, according to one embodiment, one or more message processing programs may be configured to be stored by the storage device and / or main memory 208.

[0044] Processing for the exemplary embodiments of the present invention may be performed by processing unit 206 using computer usable program code located in a memory, such as main memory 208, ROM 224, or one or more peripheral devices 226 and 230.

[0045] A bus system, such as first bus 238 or second bus 240 as shown in FIG. 2, may include one or more buses. Of course, a bus system may be implemented using any type of communications fabric or architecture that provides for a transfer of data between different components or devices attached to the communications fabric or architecture. A communications unit, such as modem 222 or network adapter 212 of FIG. 1, may include one or more devices used to transmit and receive data. A memory may include, for example, main memory 208, ROM 224, or a cache such as found in NB / MCH 202 of FIG. 1.

[0046] Those skilled in the art will appreciate that the hardware in Figure 1 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives, may be used in addition to or in place of the hardware depicted in Figure 1. Additionally, the processes of the illustrative embodiments may be applied to microprocessor data processing systems other than those described above without departing from the scope of the present invention.

[0047] Moreover, system 200 may take the form of any of a number of different data processing systems, including a client computing device, a server computing device, a tablet computer, a laptop computer, a telephone or other communications device, a personal digital assistant (PDA), etc. In some illustrative examples, system 200 may be a portable computing device configured with flash memory to implement non-volatile memory for storing operating system files and / or user-generated data. Thus, system 200 may be, in essence, any known or later developed data processing system without architectural limitation.

[0048] Referring now to FIG. 2, a simplified block diagram of an example embodiment of a system 200 (labeled system 300) for detecting scene transitions in video footage 310 is shown.

[0049] The system 200 includes an interface component 320 that is configured to obtain, for each of a plurality of distinct frames of the video footage, a confidence measure value (i.e., a confidence level) associated with a detected object in the frame.

[0050] In this example, the interface component 320 is configured to obtain a first value of a confidence measure associated with a detected object in a first frame of the video footage and obtain a second value of a confidence measure associated with the detected object in a subsequent second frame of the video footage (in other words, the interface component 320 obtains multiple confidence measure values ​​associated with the detected object in multiple consecutive frames of the video footage, respectively). Here, the interface component 320 obtains the confidence values ​​from the video processor 330. The video processor 330 in this example is a conventional video processing system configured to process received video footage and, as a result of such processing, detect objects in the video footage and determine an associated confidence value. In such processing, the video processor 330 may employ any one or more of known and available object detection algorithms to determine a confidence value associated with a detected object.

[0051] The system 200 for detecting scene transitions in the video footage 310 also includes a detection component 340. The detection component 340 is configured to detect a scene transition in the video footage based on the obtained value of the confidence measure.

[0052] 2, detection component 340 includes analysis component 350 and processor 360. Analysis component 350 is configured to analyze the obtained values ​​of the confidence measure to identify trends or patterns in the values ​​of the confidence measure across multiple different frames of the video footage. Processor 360 is then configured to identify scene transitions in the video footage based on the identified trends or patterns in the confidence values.

[0053] Purely by way of example, in response to the obtained value of the confidence measure decreasing across successive frames of the video footage, the processor 360 of the detection component 340 may determine the presence of a scene transition in the video footage 310. In response to the obtained value of the confidence measure increasing across successive frames of the video footage, the processor 360 of the detection component 340 may determine the presence of a scene transition in the video footage 310.

[0054] In the example of FIG. 2, the detection component 340 also includes a comparison unit 370 configured to compare the obtained value of the confidence measure with a threshold value. The detection component 340 is configured to identify at least one of a start and an end of a scene transition based on the comparison result obtained by the comparison unit 370. For example, to identify the start of a scene transition, the detection component 340 is configured to determine a frame of the video footage in which the obtained value of the confidence measure increases from a value that does not exceed the threshold value toward a value that exceeds the threshold value. Similarly, to identify the end of a scene transition, the detection component 340 is configured to determine a frame of the video footage in which the obtained value of the confidence measure decreases from a value that exceeds the threshold value toward a value that does not exceed the threshold value.

[0055] By way of further explanation, the concept of analyzing obtained values ​​of a confidence measure for detecting scene transitions in video footage according to the proposed embodiment will now be explained with reference to another example.

[0056] Steps (i) to (viii) below describe an example of video footage, where a video clip of a dog and a cat on a couch (i.e., a first scene) transitions to footage of people outdoors beside a car and a motorbike (i.e., a second scene), and the proposed embodiment is configured to detect the transition between these two scenes. (i) Employ an existing frame-wise convolutional neural network (CNN) to detect confidences associated with objects in the video footage. As an example, confidences associated with detected objects for the first and second frames of the video footage are determined as detailed in Table 1 below. [Table 1] (ii) Using a sliding window technique, iterate over all object detector frame results in a batch, e.g., with a batch size of 100 and a window size of 5. Figure 3 illustrates an example of how the confidence associated with a detected object changes across frames of video footage. In the illustrated example, the percentage values ​​indicate the confidence (i.e., the value of the confidence measure) associated with a detected object in a frame of video footage. To represent the change in confidence, the arrows are drawn using varying gradient shades of grey, with lighter shading representing lower confidence. As an example, the confidence associated with a detected dog decreases from a value of 90% in frame #1 to 47% around frame #57. This decrease in confidence as the video frames progress is represented by the arrow's fill changing from a darker shade (frame #1) to a lighter shade (frame #57). (iii) Using the location information provided by the output of the object detector, objects in the scene are given a unique ID and tracked frame by frame. Note that if multiple objects of the same type appear, a different ID will be issued for each object. Using a clever estimate of the expected motion of each object type, it is possible to track individual objects (e.g. according to the class label predicted by the object detector). This can be computed prior to iterating through all frames. (iv) The confidence associated with each unique object is monitored from frame to frame as shown by the gradient shading in Figure 3. (v) When a new set of objects appears (e.g., objects have associated confidences above a 30% threshold) and this coincides with a simultaneous decrease in the confidence of previous objects currently on the screen, this is inferred to indicate the start of a dissolve transition (e.g., from frame #45 in Figure 3). (vi) When the confidence of the new object continues to increase and the confidence of the old object continues to decrease past the 30% threshold, this point marks the end of the dissolve transition (frame #60 in FIG. 3). (vii) If at any point, none of the above conditions (v) & (vi) are true, then there is no dissolve transition and the start and end positions are removed from the temporary memory. Note that the assigned unique object ID information may be used to determine whether the same object is visible or not. (viii) This process is repeated for every batch of frames in the video footage.

[0057] From the above examples, it will be appreciated that the proposed embodiment can infer a transition from the obtained values ​​of the confidence measure crossing a threshold across consecutive frames of the video. For example, a decrease in confidence associated with a first detected object coinciding or corresponding with the timing of an increase in confidence associated with a second detected object may indicate a dissolve transition (as illustrated by the description of FIG. 3). Furthermore, the start and / or end points of the transition can be identified based on the frames where the confidence crosses a predefined threshold (which is configurable).

[0058] 4, an embodiment may include a computer system 70, which may form part of a networked system 7. For example, a system for detecting scene transitions in video footage may be implemented by computer system 70. Components of computer system / server 70 may include, but are not limited to, one or more processing devices, including, for example, a processor or processing unit 71, a system memory 74, and a bus 90 coupling various system components including the system memory 74 to the processing unit 71.

[0059] The system memory 74 may include computer system readable media in the form of volatile memory such as random access memory (RAM) 75 and / or cache memory 76. The computer system / server 70 may further include other removable / non-removable, volatile / non-volatile computer system storage media. In such cases, each may be connected to the bus 90 by one or more data media interfaces. The memory 74 may include at least one program product having a set (at least one) program module configured to perform the functions of the proposed embodiment. For example, the memory 74 may include a computer program product having a program executable by the processing unit 71 to cause the system to perform a method for detecting scene transitions in video footage according to the proposed embodiment.

[0060] A set of (at least one) program modules 79 programs / utilities 78 may be stored in memory 74. Program modules 79 generally perform the functions and / or methods of the proposed embodiment for detecting scene transitions in video footage.

[0061] The computer system / server 70 may also communicate with one or more external devices 80, such as a keyboard, a pointing device, a display 85, one or more devices that allow a user to interact with the computer system / server 70, or any device (e.g., a network card, a modem, etc.) that allows the computer system / server 70 to communicate with one or more other computing devices, or a combination thereof. Such communication may occur via an input / output (I / O) interface 72. Furthermore, the computer system / server 70 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter 73 (e.g., to communicate recreated content to a system or a user).

[0062] It will be appreciated that in the context of this application, where embodiments of the invention constitute methods, such methods are processes for execution by a computer, i.e. computer-implemented methods, and thus the various steps of the method reflect various portions of a computer program, e.g. portions of one or more algorithms.

[0063] The present invention may be a system, a method, and / or a computer program product, which may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0064] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), storage class memory (SCM), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or grooved structures having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over electrical wires.

[0065] The computer readable program instructions described herein may be downloaded from a computer readable storage medium to an individual computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may comprise copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the individual computing / processing device.

[0066] The computer readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or object oriented programming languages ​​such as Smalltalk®, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer as a standalone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information of the computer readable program instructions to perform aspects of the present invention.

[0067] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0068] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium, such that the computer readable storage medium on which the instructions are stored comprises an article of manufacture including instructions for implementing aspects of the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams, and can direct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner.

[0069] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operable steps to produce a computer-implemented process that implements the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0070] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block of the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, are implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0071] The description of various embodiments of the present invention has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used in this specification is chosen to best explain the principles of the embodiments, practical applications or technical improvements over the art found in the market, or to allow others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for detecting scene transitions in video footage, comprising: obtaining, for each of a plurality of distinct frames of the video footage, a confidence measure value associated with a detected object in said frame; detecting a scene transition in the video footage based on the obtained value of the confidence measure; detecting the scene transition based on the obtained value of the confidence measure; analyzing the obtained values ​​of the confidence measure to identify trends or patterns in the values ​​of the confidence measure across multiple different frames of the video footage; identifying the scene transitions in the video footage based on the identified trends or patterns in the values ​​of the confidence measure; classifying the identified scene transitions based on the identified trends or patterns in the values ​​of the confidence measures. The computer-implemented method further comprising:

2. Identifying scene transitions comparing the identified trends or patterns in the values ​​of the confidence measure with predetermined trends or patterns; identifying the scene transition in the video footage based on a result of the comparison; and The method of claim 1 , comprising:

3. detecting a scene transition based on the obtained value of the confidence measure; determining a presence of a scene transition in the video footage in response to the obtained value of the confidence measure decreasing over successive frames of the video footage; The method of claim 1 or 2, comprising:

4. detecting a scene transition based on the obtained value of the confidence measure; determining a presence of a scene transition in the video footage in response to the obtained value of the confidence measure increasing over successive frames of the video footage; The method of claim 1 or 2, comprising:

5. comparing the obtained value of the confidence measure with a threshold value; identifying at least one of a beginning and an end of the scene transition based on a result of the comparing; The method of any one of claims 1 to 4, further comprising:

6. Identifying an end of the scene transition determining frames of the video footage where the obtained value of the confidence measure decreases from a value that exceeds the threshold towards a value that does not exceed the threshold; determining the end of the scene transition based on the determined frame; and The method of claim 5 , comprising:

7. identifying a start of the scene transition, determining frames of the video footage for which the obtained value of the confidence measure increases from a value not exceeding the threshold towards a value exceeding the threshold; determining the start of the scene transition based on the determined frame; and The method of claim 5 , comprising:

8. Obtaining the value of the reliability scale is obtaining a first value of the confidence measure associated with the detected object in a first frame of the video footage; obtaining a second value of the confidence measure associated with the detected object in a subsequent second frame of the video footage; The method of any one of claims 1 to 7, comprising:

9. The method of claim 8 , wherein the first and second frames of the video footage are consecutive frames of the video footage.

10. A computer program product for causing a computer to execute the method according to any one of claims 1 to 9.

11. A computer-readable recording medium storing the computer program according to claim 10.

12. 1. A system for detecting scene transitions in video footage, comprising: one or more processors; and one or more tangible storage media storing programming instructions for execution by the one or more processors, the programming instructions comprising: instructions for obtaining, via the interface, for each of a plurality of distinct frames of the video footage, a confidence measure value associated with a detected object in the frame; and instructions for detecting a scene transition in the video footage based on the obtained value of the confidence measure; the instructions for detecting include: comparing the obtained value of the confidence measure with a threshold value; identifying at least one of a start and an end of the scene transition based on a result of the comparison; identifying a start of the scene transition, determining frames of the video footage for which the obtained value of the confidence measure increases from a value not exceeding the threshold towards a value exceeding the threshold; determining the start of the scene transition based on the determined frame; and Including, the system.

13. The instructions for detecting instructions for analyzing the obtained values ​​of the confidence measure to identify trends or patterns in the values ​​of the confidence measure across multiple different frames of the video footage; instructions for identifying scene transitions in the video footage based on the identified trends or patterns in the values ​​of the confidence measure; The system of claim 12 , comprising:

14. The detecting step comprises: determining a presence of a scene transition in the video footage in response to the obtained value of the confidence measure decreasing over successive frames of the video footage; The system according to claim 12 or 13, comprising:

15. The detecting step comprises: determining a presence of a scene transition in the video footage in response to the obtained value of the confidence measure increasing over successive frames of the video footage; The system according to claim 12 or 13, comprising:

16. Obtaining the value of the reliability scale is obtaining a first value of the confidence measure associated with the detected object in a first frame of the video footage; obtaining a second value of the confidence measure associated with the detected object in a subsequent second frame of the video footage; 16. The system of claim 12, comprising:

Images