Apparatus and Method for Extracting Scene Change Points in a Video

Abstract

A device and method for extracting scene transition points within a video are disclosed. According to one aspect of the present embodiment, a scene transition recognition device is provided, characterized by comprising a region image separation unit that separates each frame of an input image into n*m region images, a histogram extraction unit that extracts an HSV (color, saturation, brightness) histogram for each region image within each frame, and a similarity detection unit that detects similarity between each region image and the corresponding frame based on the histogram extracted from region images at corresponding locations within two adjacent frames.

Description

Technology Field

[0001] The present invention relates to an apparatus and method for automatically extracting the point in time when a scene changes within an input image. Background Technology

[0002] The content described in this section merely provides background information regarding the present embodiment and does not constitute prior art.

[0003] In the field of video editing and analysis, scene transition detection is considered a technically critical challenge. Scene transitions are a decisive factor in breaking video continuity and play a key role in the processes of editing, summarizing, searching, and search optimization.

[0004] Conventionally, there was a method for detecting scene transitions based on pixels. This method analyzes changes in pixel values ​​between consecutive frames of video. If a significant change is detected, it is considered that a scene transition has occurred. However, this method is sensitive to even subtle movements of the background or changes in lighting, which can lead to errors.

[0005] Due to these critical problems, a method for detecting scene transitions based on feature points has emerged. This method extracts feature points from the image, such as edges or textures, and detects scene transitions by analyzing changes in these points. While effective for complex scenes, it has the disadvantages of high computational cost and slow processing speed.

[0006] While these traditional methods may be effective for simple scene transitions, they are prone to errors due to various factors such as dynamic video content, complex backgrounds, and rapid camera movements. In particular, detecting scene transitions in low-light environments is very difficult. Furthermore, these methods are limited in terms of processing speed and efficiency when handling large amounts of video data in real time.

[0007] Therefore, there is a demand for a new method to detect whether a scene transition occurs within a video. The problem to be solved

[0008] One objective of the present invention is to provide an apparatus and method for automatically extracting the point in time when a scene changes within an input image while minimizing error. means of solving the problem

[0009] According to one aspect of the present invention, a scene transition recognition device is provided, characterized by comprising: a local image separation unit that separates each frame of an input image into n*m local images; a histogram extraction unit that extracts an HSV (color, saturation, brightness) histogram for each local image within each frame; and a similarity detection unit that detects similarity between each local image and the corresponding frame based on the histogram extracted from local images at corresponding locations within two adjacent frames.

[0010] According to one aspect of the present invention, the histogram extraction unit is characterized by converting RGB values ​​within each frame of an input image into HSV space.

[0011] According to one aspect of the present invention, the histogram extraction unit is characterized by converting each frame and each region image into an HSV (hue, saturation, brightness) space and then extracting a histogram for each of H, S, and V.

[0012] According to one aspect of the present invention, the similarity detection unit is characterized by determining whether the similarity of region images at corresponding locations within two adjacent frames is greater than or equal to a preset first threshold.

[0013] According to one aspect of the present invention, the similarity detection unit is characterized by determining that local images of corresponding locations within adjacent frames are similar to each other when the similarity is greater than or equal to a preset first threshold.

[0014] According to one aspect of the present invention, the similarity detection unit is characterized by determining that local images of corresponding locations within adjacent frames are not similar to each other when the similarity is less than a preset first threshold.

[0015] According to one aspect of the present invention, a scene transition point extraction device is provided, comprising a scene transition recognition unit that receives an image and recognizes whether a scene transition occurs within the image, and a point extraction unit that outputs a point in time when a scene transition occurs within the image based on the analysis result of the scene transition recognition unit, wherein the scene transition recognition unit comprises a region image separation unit that separates each frame of the input image into n*m region images, a histogram extraction unit that extracts an HSV (hue, saturation, brightness) histogram for each region image within each frame, and a similarity detection unit that detects similarity between each region image and the corresponding frame based on the histogram extracted from region images at corresponding locations within two adjacent frames.

[0016] According to one aspect of the present invention, a scene transition recognition device is provided, characterized by comprising: a region image separation unit that separates each frame of an input image into n*m region images; a histogram extraction unit that extracts an HSV (color, saturation, brightness) histogram for each region image within each frame; a similarity detection unit that detects similarity between each region image and the corresponding frame based on a histogram extracted from region images at corresponding locations within two adjacent frames; and a memory unit that stores the viewpoint of a frame determined by the similarity detection unit to have different similarity to an adjacent frame.

[0017] According to one aspect of the present invention, the similarity detection unit is characterized by determining whether the similarity of region images at corresponding locations within two adjacent frames is greater than or equal to a preset first threshold.

[0018] According to one aspect of the present invention, the similarity detection unit is characterized by determining that local images of corresponding locations within adjacent frames are similar to each other when the similarity is greater than or equal to a preset first threshold.

[0019] According to one aspect of the present invention, the similarity detection unit is characterized by determining that local images of corresponding locations within adjacent frames are not similar to each other when the similarity is less than a preset first threshold.

[0020] According to one aspect of the present invention, the similarity detection unit is characterized by determining whether the number of region images determined to be similar to each other within both frames is less than or equal to a preset second threshold, relative to the total number of region images within each frame.

[0021] According to one aspect of the present invention, the similarity detection unit is characterized by detecting that a scene has been switched between two frames when the number of region images determined to be similar to each other within both frames relative to the total number of region images within each frame is less than or equal to a preset second threshold.

[0022] According to one aspect of the present invention, a scene transition point extraction device is provided, comprising a scene transition recognition unit that receives an image and recognizes whether a scene transition occurs within the image, and a point extraction unit that outputs a point in time when a scene transition occurs within the image based on the analysis result of the scene transition recognition unit. The scene transition recognition unit comprises a region image separation unit that separates each frame of the input image into n*m region images, a histogram extraction unit that extracts an HSV (hue, saturation, brightness) histogram for each region image within each frame, a similarity detection unit that detects similarity between each region image and the corresponding frame based on histograms extracted from region images at corresponding locations within two adjacent frames, and a memory unit that stores a point in time of a frame determined by the similarity detection unit to have different similarity from an adjacent frame.

[0023] According to one aspect of the present invention, a scene transition point extraction device extracts a point in time when a scene in an image is transitioned, and the method comprises: a separation process of separating each frame in the image into a predetermined number of local images; an extraction process of extracting HSV histograms of each local image; a calculation process of calculating the similarity between corresponding local images between adjacent frames based on the extracted HSV histograms; a determination process of determining the number of local images whose similarity is greater than or equal to a predetermined first threshold; a determination process of determining whether the number of local images whose similarity is greater than or equal to a predetermined first threshold is less than or equal to a predetermined second threshold; and a detection process of detecting whether a scene transition has occurred according to the result of the determination process. Effects of the invention

[0024] As described above, according to one aspect of the present invention, there is an advantage of being able to automatically extract the point at which a scene changes within an input image while minimizing error. Brief explanation of the drawing

[0025] FIG. 1 is a diagram illustrating the configuration of a scene transition point extraction device according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a scene transition recognition unit according to an embodiment of the present invention. FIGS. 3 and 4 are diagrams illustrating the process of a scene transition recognition unit recognizing a scene transition point according to an embodiment of the present invention. FIGS. 5 and 6 are drawings illustrating the process of a memory unit storing a switched scene according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating a method for extracting a major scene by a scene transition point extraction device according to an embodiment of the present invention. Specific details for implementing the invention

[0026] The present invention is susceptible to various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0027] Terms such as first, second, A, B, etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0028] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0029] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" should be understood as not precluding the existence or addition of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification.

[0030] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.

[0031] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0032] In addition, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory to one another.

[0033] FIG. 1 is a diagram illustrating the configuration of a scene transition point extraction device according to one embodiment of the present invention.

[0034] Referring to FIG. 1, a scene transition point extraction device (100, hereinafter abbreviated as 'device') according to one embodiment of the present invention includes a scene transition recognition unit (110) and a point extraction unit (120).

[0035] The device (100) receives an arbitrary video input and analyzes and extracts the point in time when a scene changes within the video. The device (100) recognizes the point in time when a scene changes within the video, extracts the point in time, and notifies the outside, thereby allowing the device user to check only the point in time when a scene changes within the video without making any separate effort.

[0036] The scene transition recognition unit (110) receives an image and recognizes whether a scene transition occurs within the image. The scene transition recognition unit (110) divides each frame within the image into multiple local images and extracts a histogram of each local image to detect similarity between adjacent frames. If the similarity between adjacent frames is below a certain level, the scene transition recognition unit (110) determines that a scene transition has occurred between adjacent frames. The scene transition recognition unit (110) stores the time (frame) at which the scene transition occurred.

[0037] The viewpoint extraction unit (120) outputs the viewpoint at which a scene is switched within the video to the outside based on the analysis results of the scene transition recognition unit (110). The viewpoint extraction unit (120) extracts the viewpoints of the frames at which the scene is switched that are stored within the scene transition recognition unit (110). The viewpoint extraction unit (120) extracts only the viewpoints of the corresponding frames within the video and notifies the outside (user), thereby enabling the user to more conveniently recognize the viewpoint at which a scene is switched in the video.

[0038] The device (100) extracts only the scene transition point of the input video, thereby enabling the user of the device to easily recognize the scene transition point within the video without having to check the video one by one.

[0039] FIG. 2 is a diagram illustrating the configuration of a scene transition recognition unit according to an embodiment of the present invention, FIG. 3 and 4 are diagrams illustrating the process of a scene transition recognition unit recognizing a scene transition point according to an embodiment of the present invention, and FIG. 5 and 6 are diagrams illustrating the process of a memory unit storing a transitioned scene according to an embodiment of the present invention.

[0040] Referring to FIG. 2, a scene transition recognition unit (110) according to one embodiment of the present invention includes a region image separation unit (210), a histogram extraction unit (220), a similarity detection unit (230), and a memory unit (240).

[0041] The local image separation unit (210) separates each frame of the input image into n*m local images. The local image separation unit (210) separates each frame into multiple local images as shown in FIG. 3. The local image separation unit (210) separates the i-th frame into multiple local images, and the separated local images can be defined as follows.

[0042]

[0043] The defined local image refers to the local image at the (x, y) coordinate within the i-th frame. In this way, the local image separation unit (210) separates each frame into an equal number of local images. Accordingly, local images of corresponding locations between adjacent frames can be generated.

[0044] The histogram extraction unit (220) extracts an HSV (hue, saturation, brightness) histogram for each region image within each frame. The input image has RGB values ​​for each pixel. The histogram extraction unit (220) converts the RGB values ​​within each frame into an HSV (hue, saturation, brightness) space in this manner. After converting each frame and each region image into an HSV (hue, saturation, brightness) space, the histogram extraction unit (220) extracts a histogram for each of H, S, and V. The histogram is defined as follows.

[0045]

[0046] The defined histogram refers to the histogram for the local image at the (x, y) coordinate within the i-th frame. As illustrated in FIG. 4, it can be implemented by accumulating the H·S·V values ​​of each local image converted into HSV spatial values. The histogram extraction unit (220) extracts a histogram for each H·S·V of each local image in this manner.

[0047] The similarity detection unit (230) detects the similarity between each local image and the corresponding frame based on a histogram extracted from local images of corresponding locations within adjacent frames (the i-th frame and the i+1-th frame).

[0048] The similarity detection unit (230) detects the similarity of local images of corresponding locations using the following formula.

[0049]

[0050] Here, H1 and H2 represent the histograms of local images at corresponding locations within two adjacent frames, and Sl(H1, H2) represents the similarity between those local images, respectively.

[0051] The similarity detection unit (230) determines whether the similarity of the corresponding region images calculated through the aforementioned formula is greater than or equal to a preset first threshold. If the similarity of the corresponding region images is greater than or equal to the preset first threshold, the similarity detection unit (230) determines that the two region images are similar to each other, and if not, the similarity detection unit (230) determines that the two region images are not similar to each other.

[0052] The similarity detection unit (230) determines the aforementioned similarity for all region images of corresponding locations within adjacent frames (i-th frame and i+1-th frame).

[0053] When the number of region images (of corresponding locations) determined to be similar to each other within both frames relative to the total number of region images within each frame is less than or equal to a preset second threshold, the similarity detection unit (230) detects that the scene has been switched between the two frames. On the other hand, when the number of region images (of corresponding locations) determined to be similar to each other within both frames exceeds a preset second threshold, the similarity detection unit (230) detects that the scene is continuing.

[0054] The memory unit (240) stores the time of a frame that is determined by the similarity detection unit (230) to have a different similarity to an adjacent frame (that the scene has been switched). As illustrated in FIG. 5, the memory unit (240) may store the time of a frame that is determined to have a different similarity to an adjacent frame (that the scene has been switched). Alternatively, as illustrated in FIG. 6, the memory unit (240) may store metadata for a frame that is determined to have a different similarity to an adjacent frame (that the scene has been switched). As the memory unit (240) stores the time of the frame or metadata in this manner, the time extraction unit (120) can recognize at which time (frame) the scene was switched within the video based on the information stored in the memory unit (240) and transmit or output this to the outside.

[0055] FIG. 7 is a flowchart illustrating a method for extracting a major scene by a scene transition point extraction device according to an embodiment of the present invention.

[0056] The local image separation unit (210) separates each frame in the video into a preset number of local images (S710).

[0057] The histogram extraction unit (220) extracts the HSV histograms of each region image (S720).

[0058] The similarity detection unit (230) calculates the similarity of corresponding local images between adjacent frames based on the extracted HSV histogram (S730). The similarity detection unit (230) determines whether the similarity of corresponding local images between adjacent frames is greater than or equal to a preset first threshold.

[0059] The similarity detection unit (230) identifies the number of region images whose similarity is greater than or equal to a first threshold value that has been set (S740).

[0060] The similarity detection unit (230) determines whether the number of images of the corresponding region is less than or equal to a pre-set second threshold (S750).

[0061] If the number of images of the area is less than or equal to a second threshold set beforehand, the similarity detection unit (230) determines that a scene change has occurred (S760).

[0062] If the number of images of the area exceeds a pre-set second threshold, the similarity detection unit (230) determines that there is no scene change (S770).

[0063] Although FIG. 7 describes each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present invention. In other words, a person skilled in the art to which one embodiment of the present invention belongs can modify and adapt it in various ways, such as changing the order described in each figure or executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present invention; therefore, FIG. 7 is not limited to a chronological order.

[0064] Meanwhile, the processes illustrated in FIG. 7 can be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. That is, a computer-readable recording medium includes storage media such as magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.). In addition, computer-readable recording media can be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner.

[0065] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment. Explanation of the symbols

[0066] 100: Scene transition point extraction device 110: Scene transition recognition unit 120: Time point extraction unit 210: Region image separator 220: Histogram Extraction Unit 230: Similarity detection unit 240: Memory section

Claims

Claim 1 A scene transition recognition device characterized by comprising: a region image separation unit that separates each frame of an input video into n*m region images; a histogram extraction unit that extracts HSV (hue, saturation, brightness) histograms for each region image within each frame; and a similarity detection unit that detects the similarity of each region image at a corresponding location using the following formula based on the histograms extracted from region images at corresponding locations within two adjacent frames, determines that the two region images are similar to each other if the similarity of the detected region images is greater than or equal to a preset first threshold, and determines that the similarity of all region images at corresponding locations within two adjacent frames is similar to each other, and detects that the scene has transitioned between the two frames if the number of region images determined to be similar to each other within the two frames relative to the total number of region images within each frame is less than or equal to a preset second threshold. Here, H1 and H2 represent the histograms of local images at corresponding locations within two adjacent frames, and Sl(H1, H2) represents the similarity between those local images, respectively. Claim 2 A scene change recognition device according to claim 1, wherein the histogram extraction unit converts RGB values ​​within each frame of an input image into HSV space. Claim 3 A scene change recognition device according to claim 2, wherein the histogram extraction unit converts each frame and each region image into HSV (hue, saturation, brightness) space and then extracts a histogram for each of H, S, and V. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A scene transition point extraction device characterized by comprising: a scene transition recognition unit that receives video input and recognizes whether a scene transition occurs within the video; and a point extraction unit that outputs a point in time when a scene transition occurs within the video based on the analysis result of the scene transition recognition unit, wherein the scene transition recognition unit comprises: a local image separation unit that separates each frame of the input video into n*m local images; a histogram extraction unit that extracts HSV (hue, saturation, brightness) histograms for each local image within each frame; and a similarity detection unit that detects the similarity of each local image at a corresponding location using the following formula based on the histograms extracted from local images at corresponding locations within two adjacent frames, determines that the two local images are similar to each other if the similarity of the detected local images is greater than or equal to a preset first threshold, and determines that the similarity of all local images at corresponding locations within two adjacent frames is determined to be similar to each other if the number of local images determined to be similar to each other within both frames relative to the total number of local images within each frame is less than or equal to a preset second threshold. Here, H1 and H2 represent the histograms of local images at corresponding locations within two adjacent frames, and Sl(H1, H2) represents the similarity between those local images, respectively. Claim 8 A scene transition recognition device characterized by comprising: a region image separation unit that separates each frame of an input video into n*m region images; a histogram extraction unit that extracts an HSV (hue, saturation, brightness) histogram for each region image within each frame; a similarity detection unit that detects the similarity of each region image at a corresponding location using the following formula based on the histogram extracted from the region images at a corresponding location within two adjacent frames, and determines that the two region images are similar to each other if the similarity of the detected region images is greater than or equal to a preset first threshold, and determines that the similarity of all region images at corresponding locations within two adjacent frames is determined to be similar to each other if the number of region images determined to be similar to each other within both frames relative to the total number of region images within each frame is less than or equal to a preset second threshold; and a memory unit that stores the viewpoint of a frame determined by the similarity detection unit to have a different similarity from an adjacent frame. Here, H1 and H2 represent the histograms of local images at corresponding locations within two adjacent frames, and Sl(H1, H2) represents the similarity between those local images, respectively. Claim 9 delete Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 A scene transition recognition unit that receives video input and recognizes whether a scene transition occurs within the video; and a point in time extraction unit that outputs a point in time when a scene transition occurs within the video based on the analysis result of the scene transition recognition unit, wherein the scene transition recognition unit includes: a local image separation unit that separates each frame of the input video into n*m local images; a histogram extraction unit that extracts HSV (hue, saturation, brightness) histograms for each local image within each frame; and a similarity detection unit that detects the similarity of each local image at a corresponding location using the following formula based on the histograms extracted from local images at corresponding locations within two adjacent frames, determines that the two local images are similar to each other if the similarity of the detected local images is greater than or equal to a preset first threshold, and determines that the similarity of all local images at corresponding locations within two adjacent frames is determined to be similar to each other if the number of local images determined to be similar within both frames relative to the total number of local images within each frame is less than or equal to a preset second threshold, thereby detecting that a scene transition has occurred between the two frames. A scene transition point extraction device characterized by including a memory unit that stores the point of view of a frame determined by the similarity detection unit to have a different similarity to an adjacent frame. Here, H1 and H2 represent the histograms of local images at corresponding locations within two adjacent frames, and Sl(H1, H2) represents the similarity between those local images, respectively. Claim 15 delete

Images