Film detection system and film detection method

Abstract

The present invention discloses a film detection system and a film detection method. The method includes: capturing a current image of an input video source; performing a pre-processing process on the current image, performing image scaling on the current image, cutting and retaining the top and bottom of the current image, and splicing to generate a spliced ​​current image; inputting the spliced ​​image into a trained machine learning model to classify the spliced ​​current image as a film image or a non-film image. The trained machine learning model is generated by performing a training process on the machine learning model, and the training process is performed based on multiple spliced ​​training images. The spliced ​​training image is generated by performing a pre-processing process on multiple film images with black borders and multiple non-film images without black borders.

Description

Technical Field

[0001] The present invention relates to a detection system and a detection method, and in particular to a movie detection system and a movie detection method. Background Art

[0002] As people's expectations for movie viewing quality continue to rise, existing multimedia products often adjust the video or audio when playing movies to enhance the user's viewing experience, even allowing users to perceive the difference between movies and non-movies. In other words, it is necessary to detect and distinguish between movie and non-movie video sources.

[0003] However, as display resolution increases, the resolution of video sources also increases. Consequently, the cost of capturing and analyzing high-resolution video sources is increasing. In addition to increasing demands on computing resources, the time required to analyze high-resolution video is also increasing.

[0004] Therefore, there is a need for a detection system and a detection method that can reduce the cost of analyzing images. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a film detection system and a film detection method in view of the deficiencies in the prior art.

[0006] In order to solve the above-mentioned technical problems, one of the technical solutions adopted by the present invention is to provide a movie detection method, comprising: configuring an electronic device to play an input video source; configuring a processor of the electronic device to capture a current image of the input video source and store it in a memory, wherein the current image has a first image scale; performing a pre-processing process on the current image, which includes configuring the processor to: perform image scaling processing on the current image to generate a scaled current image with a second image scale, wherein the second image scale is smaller than the first image scale; performing a cropping process on the scaled current image to retain only the top and bottom of the scaled current image; and The top and the bottom are joined to generate a joined current image; and the processor is configured to input the joined image into a trained machine learning model to classify the joined current image as a movie image or a non-movie image, wherein the trained machine learning model is generated by executing a training process on the machine learning model, and the training process is based on a plurality of joined training images, wherein the plurality of joined training images are generated by executing the pre-processing process on a plurality of movie images with black edges and a plurality of non-movie images without black edges, and the plurality of joined training images are respectively labeled as movie images and non-movie images to serve as the expected output of the machine learning model in the training process.

[0007] To address the aforementioned technical issues, another technical solution employed by the present invention is to provide a film detection system, comprising an input video source and an electronic device. The electronic device is configured to receive and play the input video source, and includes a memory and a processor. The memory stores a trained machine learning model. The processor is configured to capture a current image of the input video source and store it in the memory, wherein the current image has a first image scale. The processor is configured to perform a pre-processing process on the current image, comprising: performing image scaling on the current image to generate a scaled current image having a second image scale, wherein the second image scale is smaller than the first image scale; performing a cropping process on the scaled current image to retain only the top and bottom portions of the scaled current image; and splicing the top and bottom portions to generate a spliced ​​current image. The processor is configured to input the spliced ​​current image into the trained machine learning model to classify the spliced ​​current image as a film image or a non-film image. The trained machine learning model is generated by performing a training process on the machine learning model, and the training process is performed based on multiple spliced ​​training images. Among them, the multiple spliced ​​training images are generated by executing the pre-processing process on multiple movie images with black borders and multiple non-movie images without black borders, and the multiple spliced ​​training images are respectively labeled as movie images and non-movie images to serve as the expected output of the machine learning model in the training process.

[0008] One of the benefits of the present invention is that the film detection system and method provided herein utilizes the unique black borders produced by the specific proportions of film images during playback. Pre-processing the input image extracts specific features for detection, and also trains a machine learning model based on these black border features. Given the significantly reduced image size required for detection, a lightweight deep learning model can be employed, significantly reducing computational cost and time.

[0009] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and illustration and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 FIG. 4 is a functional block diagram of a movie detection system according to an embodiment of the present invention.

[0011] Figure 2 FIG. 4 is a flow chart of a movie detection method according to an embodiment of the present invention.

[0012] Figures 3A to 3C FIG. 1 is a schematic diagram of a pre-processing process according to an embodiment of the present invention.

[0013] Figure 4 FIG. 4 is a flow chart illustrating a training process according to an embodiment of the present invention.

[0014] Figure 5 FIG2 is a schematic diagram of the architecture of a convolutional neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following is an explanation of the implementation methods of the "movie detection system and movie detection method" disclosed in the present invention through specific specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual dimensions. It is stated in advance. The following embodiments will further explain the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention. In addition, the term "or" used in this article may include any one or more combinations of the associated listed items depending on the actual situation.

[0016] Figure 1 FIG. 4 is a functional block diagram of a movie detection system according to an embodiment of the present invention.

[0017] See Figure 1 As shown, a first embodiment of the present invention provides a movie detection system 1 , which includes an input video source 10 and an electronic device 12 .

[0018] The electronic device 12 is, for example, a desktop computer, a laptop computer, a smartphone, a tablet computer, a game console, an e-book, a set-top box, or a smart TV, and is configured to receive and play the input video source 10. The electronic device 12 includes a processor 120 and a memory 122. The electronic device 12 may have a display, such as a liquid crystal display (LCD), a light-emitting diode (LED), a field emission display (FED), an organic light-emitting diode (OLED), or other types of displays.

[0019] The input video source 10 may be a file source in the electronic device 12 or another independent electronic device. The input video source 10 may provide multimedia data, for example, a local video of the electronic device 12. In some embodiments, the input video source 10 may be streaming data transmitted based on a protocol such as User Datagram Protocol (UDP), Transmission Control Protocol (TCP), or Real-Time Streaming Protocol (RTSP), but the present invention is not limited thereto.

[0020] Memory 122 can be used to store data such as images, program code, software modules, and the like. It can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, a hard disk, or other similar devices, integrated circuits, or combinations thereof. In this embodiment, memory 122 stores a trained machine learning model M1.

[0021] The processor 120 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), programmable logic device (PLD), graphics processing unit (GPU), or other similar devices or a combination of these devices. The processor 120 can execute program codes, software modules, instructions, etc. recorded in the memory 122 to implement the movie detection method according to the embodiment of the present invention.

[0022] Additionally, the electronic device 12 may include hardware decoding circuitry to perform video decoding operations on a hardware basis. In some embodiments, the hardware decoding circuitry may be implemented by at least one image / audio processing engine circuit, at least one display chip, at least one audio processing chip, and / or at least one application-specific integrated circuit, but the present invention is not limited thereto. The aforementioned hardware decoding circuitry may be included in the processor 120.

[0023] Figure 2 This is a flow chart of a movie detection method according to an embodiment of the present invention. Figure 2The movie detection method mentioned in this embodiment is applicable to the movie detection system 1 in the above embodiment. The following is a detailed description of the processing method for the user interface of this embodiment in combination with the various components in the movie detection system 1.

[0024] Step S20: Configuring the electronic device 12 to play the input video source.

[0025] Step S21: The processor 120 of the electronic device 12 is configured to capture a current image of an input video source and store the captured image in the memory 122. The current image has a first image scale.

[0026] Generally speaking, the current standard specifications for film images usually have an aspect ratio of 2.39:1 or 1.85:1. However, existing monitors that support specifications such as 4K, 2K, 1080P, and 720P mostly use 16:9 or 16:10, which inevitably results in black bars on the upper and lower sides when playing movies. This can also be used to distinguish between film images and non-film images.

[0027] Step S22: Execute pre-processing process on the current image, which can be referred to Figures 3A to 3C , Figures 3A to 3C FIG. 1 is a schematic diagram of a pre-processing process according to an embodiment of the present invention. The pre-processing process includes configuring the processor 120 to perform the following steps:

[0028] Step S23: Perform image scaling on the current image to generate a scaled current image with a second image scale. For example, the image scaling includes nearest neighbor sampling interpolation, bilinear interpolation, bicubic interpolation, and bitmap scaling algorithms, and detail loss caused by the scaling process can be ignored. Figures 3A to 3B shown.

[0029] On the other hand, the second image scale is smaller than the first image scale. For example, the first image scale includes 4K, 2K, 1080P, and 720P, and the second image scale can be smaller than 1 / 16 of the first image scale. In this step, the current image can be scaled to the second image scale of 320*180 using a bilinear interpolation method. Although this method will lose details in the current image, since the present invention focuses on the black borders on the upper and lower sides of the current image, the impact of the loss of image details is negligible.

[0030] Step S24: Crop the zoomed current image to retain only the top TOP and bottom BOT of the zoomed current image. Figures 3B to 3C As shown, only the top TOP and bottom BOT of the scaled current image are retained.

[0031] Step S25: Join the top TOP and the bottom BOT to generate a joined current image, such as Figure 3C shown.

[0032] Step S26: configuring the processor 120 to input the joined image into the trained machine learning model to classify the joined current image as a movie image or a non-movie image.

[0033] In the above steps, the trained machine learning model is generated by executing the training process on the machine learning model. Figure 4 , Figure 4 FIG. 1 is a flow chart illustrating a training process according to an embodiment of the present invention. The training process is performed based on multiple joined training images and includes the following steps:

[0034] Step S40: Pre-processing is performed on the plurality of film images with black borders and the plurality of non-film images without black borders in the training dataset. Similarly, the plurality of film images with black borders and the plurality of non-film images without black borders can be scaled to a second image scale of 320*180 using a bilinear interpolation method, cropped, and then spliced ​​to obtain a plurality of spliced ​​training images.

[0035] Step S41: Label the plurality of joined training images as movie images and non-movie images, respectively, to serve as expected outputs of the machine learning model.

[0036] Step S42: Performing data augmentation processing on the plurality of joined training images. In this step, the data augmentation processing is to randomly adjust the RGB values ​​of the plurality of joined training images within a predetermined range.

[0037] Specifically, since different TV boxes or different video sources may have different responses to the RGB values ​​of black edges, this step can perform random brightness adjustment on the copied multiple spliced ​​training images to randomly increase the RGB values ​​by approximately 10 to 20, so that the training data is closer to real-world usage.

[0038] Please refer to Table 1, which is the training dataset used in the above training process, and the training dataset is divided into a training set and a test set:

[0039] Table 1:

[0040]

[0041]

[0042] Step S43: Input the multiple joined training images that have undergone data enhancement processing into the machine learning model, and evaluate the performance of the machine learning model at the same time, and use the machine learning model that passes the performance test as the trained machine learning model.

[0043] In the above embodiment, the machine learning model may be, for example, a convolutional neural network. Figure 5 , Figure 5 FIG2 is a schematic diagram of the architecture of a convolutional neural network according to an embodiment of the present invention.

[0044] like Figure 5 As shown, the convolutional neural network includes an input convolution layer 50, a plurality of depthwise separable convolution layers 51 and a linear output layer 52.

[0045] The input convolution layer 50 may be a 2D convolution layer, for example. Each of the plurality of depthwise separable convolution layers 51 includes a depthwise convolution architecture 510 and a pointwise convolution architecture 512 .

[0046] In detail, the depth-wise separable convolution layer 51 comes from MobileNet, a high-efficiency network designed for mobile and embedded vision applications. Compared with networks that use the same depth for regular / standard convolution calculations, MobileNet uses a mixture of depth-wise convolutions and a large number of 1*1*M point-by-point convolutions to significantly reduce the convolution calculation burden (loading), thereby achieving a lightweight deep neural network.

[0047] [Beneficial Effects of Embodiments]

[0048] One of the benefits of the present invention is that the film detection system and method provided herein utilizes the unique black borders produced by the specific proportions of film images during playback. Pre-processing the input image extracts specific features for detection, and also trains a machine learning model based on these black border features. Given the significantly reduced image size required for detection, a lightweight deep learning model can be employed, significantly reducing computational cost and time.

[0049] The contents disclosed above are only preferred feasible embodiments of the present invention and do not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the scope of the patent application of the present invention.

[0050]

Explanation of symbols

[0051] 1: Movie detection system

[0052] 10: Input video source

[0053] 12: Electronic devices

[0054] 50: Input convolution layer

[0055] 51: Depthwise Separable Convolutional Layer

[0056] 52: Linear output layer

[0057] 120: Processor

[0058] 122: Memory

[0059] 510: Depthwise Convolutional Architecture

[0060] 512: Point-by-point convolutional architecture

[0061] BOT: bottom

[0062] M1: trained machine learning model

[0063] TOP: Top

Claims

1. A movie detection method, comprising: configuring the electronic device to play the input video source; configuring the processor of the electronic device to capture a current image of the input video source and store the image in a memory, wherein the current image has a first image scale; Performing a pre-processing process on the current image includes configuring the processor to: performing an image scaling process on the current image to generate a scaled current image having a second image scale, wherein the second image scale is smaller than the first image scale; performing a cropping process on the scaled current image to retain only the top and bottom of the scaled current image; and Joining the top portion and the bottom portion to generate a joined current image; and configuring the processor to input the joined current image into a trained machine learning model to classify the joined current image as a movie image or a non-movie image, In which, the trained machine learning model is generated by executing a training process on the machine learning model, and the training process is based on multiple spliced ​​training images, wherein the multiple spliced ​​training images are generated by executing the pre-processing process on multiple movie images with black borders and multiple non-movie images without black borders, and the multiple spliced ​​training images are respectively labeled as movie images and non-movie images to serve as the expected output of the machine learning model in the training process.

2. The movie detection method according to claim 1, wherein the training process further comprises: performing data augmentation processing on the plurality of joined training images; and The multiple joined training images that have undergone data augmentation processing are input into the machine learning model, and the performance of the machine learning model is evaluated at the same time, and the machine learning model that passes the performance test is used as the trained machine learning model. 3 . The movie detection method according to claim 2 , wherein the data augmentation process is to randomly adjust the RGB values ​​of the plurality of joined training images within a predetermined range. 4 . The movie detection method according to claim 1 , wherein the first image scale includes 4K, 2K, 1080P and 720P, and the second image scale is smaller than 1 / 16 of the first image scale. 5 . The movie detection method according to claim 1 , wherein the image scaling process comprises a nearest neighbor sampling interpolation method, a bilinear interpolation method, a bicubic interpolation method, and a bitmap scaling algorithm.

6. The movie detection method according to claim 1, wherein the machine learning model is a convolutional neural network.

7. The movie detection method according to claim 6, wherein the convolutional neural network comprises: Input convolutional layer; Multiple depthwise separable convolutional layers; as well as Linear output layer, Each of the plurality of depth-wise separable convolutional layers includes a depth-wise convolutional architecture and a point-wise convolutional architecture.

8. A film detection system comprising: Input video source; as well as An electronic device configured to receive and play the input video source, and comprising: a memory storing a trained machine learning model; and A processor configured to capture a current image of the input video source and store it in a memory, wherein the current image has a first image scale, The processor is configured to perform a pre-processing process on the current image, which includes: performing an image scaling process on the current image to generate a scaled current image having a second image scale, wherein the second image scale is smaller than the first image scale; performing a cropping process on the scaled current image to retain only the top and bottom of the scaled current image; and Joining the top portion and the bottom portion to generate a joined current image; and Wherein, the processor is configured to input the joined current image into a trained machine learning model to classify the joined current image as a movie image or a non-movie image, wherein the trained machine learning model is generated by executing a training process on the machine learning model, and the training process is based on multiple joined training images, wherein the multiple joined training images are generated by executing the pre-processing process on multiple movie images with black borders and multiple non-movie images without black borders, and the multiple joined training images are respectively labeled as movie images and non-movie images to serve as the expected output of the machine learning model in the training process.

9. The movie detection system according to claim 8, wherein the training process further comprises: performing data augmentation processing on the plurality of joined training images; and The multiple joined training images that have undergone data augmentation processing are input into the machine learning model, and the performance of the machine learning model is evaluated at the same time, and the machine learning model that passes the performance test is used as the trained machine learning model. 10 . The movie detection system according to claim 9 , wherein the data augmentation process is to randomly adjust the RGB values ​​of the plurality of stitched training images within a predetermined range.

Images