Video and audio content processing through machine learning models for delivery to aircraft inflight entertainment systems

Abstract

A method is performed by a computer system of a content service provider for executing a quality assurance and / or quality control (QA / QC) system module. The method includes performing at least one media verification procedure on media content. The method generates a QA / QC score based on results of the performed at least one media verification procedure. The method determines if the media content satisfies a QA / QC standard based on the QA / QC score.

Description

VIDEO AND AUDIO CONTENT PROCESSING THROUGH MACHINE LEARNING MODELS FOR DELIVERY TO AIRCRAFT INFLIGHT ENTERTAINMENT SYSTEMSTECHNICAL FIELD

[0001] Embodiments described herein relate generally to aircraft inflight entertainment systems for passengers and, more particularly, to preparing content for delivery to aircraft inflight entertainment systems.BACKGROUND

[0002] Aircraft inflight entertainment (IFE) systems are deployed onboard aircraft to provide entertainment services for passengers in a passenger cabin. The IFE systems typically distribute on-demand and / or streaming video and / or audio content, including movie, television, audio, electronic publications, and gaming programming to passenger-operated electronic communication devices. Content can be provided from onboard content servers and / or ground-based content servers. Passenger-operated electronic communication devices can include, without limitation, seat video units and portable passenger electronic devices (PEDs).

[0003] Content undergoes quality assurance (QA) and quality control (QC) processes before being delivered to IFE systems. Such QA and QC processes have been human labor intensive, which increases content costs and delays delivery of new content to the IFE systems. As a result, frequent flyers may perceive on-board content offerings to be relatively static.Passenger satisfaction with a flight experience and, ultimately, with an airline can be significantly impacted by the variety of content made available through IFE systems and its freshness relative to when first released.SUMMARY

[0004] In various embodiments of the present disclosure, a method performed by a media content service provider for executing a quality assurance and / or quality control (QA / QC) system module. The method includes performing at least one media verification procedure on a mediacontent. The method also includes generating a QA / QC score based on results of the performed at least one media verification procedure. The method also includes determining if the media content satisfies a QA / QC standard based on the QA / QC score.

[0005] Other computer systems of content service providers and related methods and computer program products according to embodiments of the present disclosure will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such additional computer systems of content service providers and related methods and computer program products be included within this description and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and / or combination.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of the invention. In the drawings:

[0007] Figure 1 illustrates a block diagram of process activities performed by entities to prepare content for delivery to IFE systems in accordance with some embodiments;

[0008] Figure 2 illustrates a block diagram of a content quality assurance (“QA”) and / or quality control (“QC”) ("QA / QC") process pipeline for performing some process activities in accordance with some embodiments;

[0009] Figure 3 illustrates a block diagram of operational processes in an artificial intelligence and / or machine learning pipeline for performing a first set of IFE content QA / QC processes shown in Figure 2 in accordance with some embodiments;

[0010] Figure 4 illustrates a block diagram of operational processes for watermark / trademark detection in a video stream component of content according to one of the first set of IFE content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments;

[0011] Figure 5 illustrates a block diagram of operational processes for end credit detection in a video stream component of content according to another of the first set of IFE content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments;

[0012] Figure 6 illustrates a block diagram of operational processes for spoken language detection in an audio stream component of content according to another of the first set of IFE content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments;

[0013] Figure 7 illustrates a simplified system block diagram of an IFE content provider platform that processes IFE content for delivery to components of an IFE system according to some embodiments; and

[0014] Figure 8 illustrates a simplified system block diagram of components of an IFE content provider platform which operate according to some embodiments.DETAILED DESCRIPTION

[0015] The following detailed description discloses various non-limiting example embodiments of the invention. The invention can be embodied in many different forms and is not to be construed as limited to the embodiments set forth herein.

[0016] Embodiments of the present disclosure are directed to innovative hardware processing architectures and processes which can automate and improve quality assurance (“QA”) and / or quality control (“QC”) ("QA / QC") activities performed to make content from content owners and / or content service providers available for delivery to aircraft IFE systems. Figure 1 illustrates a block diagram of example process activities performed by entities to prepare content for delivery to IFE systems in accordance with some embodiments.

[0017] Referring to Figure 1 , content owners 100 can include movie studios (e.g., Warner Bros., Disney, Universal Pictures, etc.), broadcast networks (e.g., ABC, NBC, CBS, BBC, etc.), streaming platforms (e.g., Netflix, Hulu, Amazon Prime Video, etc.), production companies (e.g., create and own content for distribution), music labels (e.g., Sony MusicEntertainment, Universal Music Group, and Warner Music Group, etc.), and independent content creators (e.g., individual artists, YouTubers, vloggers, influencers, etc.).

[0018] Content service providers (CSPs) 102 perform various processing activities on content from the content owners 100 that can include video encoding, metadata creation / editing, localization, programming, and content QA / QC. Localization can include adapting content to suit the cultural, language (e.g., embedding multi-language components for written and / or spoken content, and adapting content to meet local laws, regulations, and industry standards, etc.), and regional preferences in a specific geographic location.

[0019] In various embodiments of the present disclosure, a method performed by a content service provider for executing a quality assurance and / or quality control (QA / QC) system module. The method includes performing at least one procedure to a content. The method also includes generating a QA / QC score based on results of the performed at least one procedure. The method also includes determining if the content satisfies a QA / QC standard based on the QA / QC score.

[0020] IFE content supplier 104 performs various processing activities 105 on content from CSPs 102 that can include performing digital rights management (DRM), forensic watermarking, content QA / QC, packaging of content (e.g., preparing for digital distribution), and rack testing. Forensic watermarking can include embedding visible / invisible and traceable information within content, such as images, videos, or audio, in order to identify the source of creation, distribution or use. Rack testing can include testing and verifying the functionality, quality, and readiness of content to be distributed and played through components of an aircraft IFE system. Example content QA / QC processes are described further below with reference to Figures 2-6.

[0021] Content from the IFE content supplier 104 can be distributed to aircraft IFE systems 108, e.g., in one or more airline aircraft fleets, through physical data downloader devices 106 (e.g., portable mass-memory storage devices) and / or through communications networks that can include publicnetworks (e.g., Internet) and / or private networks and wireless networks that can include satellite networks, cellular networks, and / or WiFi networks.

[0022] Although various embodiments are described in the example context of preparing content for distribution to IFE systems, these and other embodiments are not limited thereto and may be used to prepare content for distribution to any type of system. Moreover, although some QA / QC operational activities are described as being performed by a content supplier, these and other operational activities may be performed by a content owner, a CSP, etc. in accordance with some other embodiments.

[0023] Some embodiments are directed to automating the content QA / QC operational activities 105 in Figure 1 through a computer-implemented content QA / QC process pipeline(s). Figure 2 illustrates a block diagram of a content QA / QC process pipeline 200 for performing at least some of the QA / QC operational activities 105 in accordance with some embodiments.

[0024] Referring to Figure 2, a movie with metadata or other content is QA / QC processed to confirm one or more of: 1 ) the movie is complete; 2) airline watermark(s) and / or other trademark(s) is / are present in the video component of the movie, e.g., in a defined range of the movie timeline, occur concurrently or within a defined time of each other, and / or are displayed for at least a threshold time; 3) which languages (e.g., among a defined set of languages) are present in corresponding audio component tracks of the movie; 4) synchronization between video and audio components and / or other synchronization checks; 5) movie corresponds to a defined asset identifier; 6) audio packet identifiers (PIDs); 7) defined subtitle track(s) are available; 8) subtitle(s) is / are properly synchronized to corresponding parts of the audio track(s); and 9) synopsis is accurate / appropriate.

[0025] As used herein, the term "language" can include variations in languages associated with dialects, such as Portuguese, Brazilian Portuguese, French, French Canadian, Chinese Mandarin, Chinese Taiwanese, Chinese Cantonese, Traditional Chinese, Simplified Chinese, etc.

[0026] The first three content QA / QC processes of the QA / QC operational activities 105 in Figure 2 are referred to as a first set ofoperations which are described in further detail below. These and other embodiments can also be applied to the other five content QA / QC processes of the QA / QC operational activities 105 in Figure 2, which are also referred to as a second set of operations. Accordingly, the embodiments disclosed herein can be applied to perform any of the first and second set of operations shown in Figure 2.

[0027] As will be explained in further detail below, the operation to confirm that the movie is complete may include performing movie (video) frame extraction, processing the extracted frames through at least an artificial intelligence (Al) / machine learning end-credit classifier, and filtering the output thereof through logical operations that confirm that the start of the movie and the expected end of the movie (e.g., end-credits) are present.

[0028] The operation to confirm that the airline watermark(s) and / or other trademark(s) is / are present may include performing movie (video) frame extraction, processing the extracted frames through an optical character recognition (OCR) module and / or an Al / machine learning watermark or other trademark identifier (e.g., based on AIResNet convolutional neural network product), and aggregating the output thereof through logical operations that confirm that the airline watermark(s) and / or other trademark(s) is / are present in the movie.

[0029] The operation to confirm which languages (e.g., among a defined set of languages) are present in corresponding audio component tracks of the movie, may include performing audio track extraction, processing the extracted audio using an Al / machine learning spoken language model adapted to identify defined types spoken languages or using a set of Al / machine spoken language models each adapted to identify different types of spoken languages, and aggregating the output thereof through logical operations that confirm, e.g., which languages among the defined set of languages are present.

[0030] Figure 3 illustrates a block diagram of operational processes in an Al and / or machine learning (ML) pipeline 300 for performing the first set of content QA / QC processes shown in Figure 2 in accordance with some embodiments.

[0031] Referring to Figure 3, the AI / ML pipeline 300 can include a file preprocessing step 310 that logs which content files have been processed, success or failure outcomes of the processing of the content files, and metrics associated with the content files (e.g., size, playback time, etc.) and / or metrics associated with the system utilization when processing the content files.

[0032] An audio and / or video check processing step 312 extracts chunks of the audio track(s), video component, subtitle track(s), etc., for further processing analysis. The duration of the chunks may be static (e.g., 20 seconds) or may be adapted based on prior processing of the content resulted in one or more failure determination outcomes, e.g., failed the movie completion check, watermark / trademark check, and / or languages check. The number of chunks extracted from spaced apart locations in the content may be static (e.g., 4 chunks) or the number may be adapted (e.g., increased) based on whether prior processing of the content resulted in one or more failure determination outcomes. The spaced apart locations in the content where chunks are extracted may be static or may be adapted (e.g., shifted to other time windows) based on whether prior processing of the content resulted in one or more failure determination outcomes.

[0033] For example, failure determination output(s) from one cycle of processing through the AI / ML pipeline 300 may cause a new set of chunks to be selected and extracted with each chunk having a longer duration, the number of chunks may be increased, and / or the locations where the chunks are extracted from the movie may be time shifted. Time shifting the locations where chunks are extracted for another cycle of processing through the pipeline 200 may be particularly beneficial in avoiding portions of a movie where high background noise occurred, where a non-dominant language is being temporarily spoken, or where another scenario occurs that prevents proper detection of the languages present in the audio tracks of the content. Similarly, time shifting the locations where chunks are extracted for another cycle of processing through the pipeline 200 may be particularly beneficial in identifying presence of watermarks / trademarks which are only temporarily displayed in the video content and / or avoiding portions of thevideo content where insufficient contrast is present between a watermark / trademark being displayed and bright background images in the video content and which prevents proper detection of presence of the watermarks / trademarks.

[0034] Output of audio and / or video chunking 312 is then input to one or more AI / ML models 314. In some embodiments, a plurality of models 314 process the chunks in parallel. One model 314 is trained / configured to confirm a movie is complete, e.g., adapted to confirm length of the movie corresponds to a movie length indicated by metadata, and / or adapted to identify indicia associated with the start and end of a movie, e.g., title and / or name of content owner and / or CSP near start of the movie, copyright notification near start of movie, credits near end of movie, etc. Another model 314 is trained / configured to confirm a watermark or trademark is present in the movie, e.g., video component. Another model 314 is trained / configured to confirm a correct set of languages are present on corresponding audio tracks of the movie.

[0035] In another embodiment, the same chunks or different chunks can be processed through a plurality of processing cycles of the pipeline 300, with the model 314 being reconfigured between processing cycles to be adapted to perform different defined QA / QC tests.

[0036] Output of the model(s) 314 is processed to perform scoring 316. The processing can include providing confidence indications output by the models 314 to a ruleset to make decisions and / or to algorithms that generate metrics (e.g., data drift over time). Results of the scoring 316 can be aggregated 318 for output to operators and / or to a reporting / management system, and may be used to adapt the chunking operations 312 as described above to perform one or more processing cycles of the pipeline 300 when one or more QC / QA tests failed. Results of the scoring 316 may be fed-back for training of the AI / ML model(s) 314.

[0037] Figure 4 illustrates a block diagram of operational processes for watermark / trademark detection in a video stream component of content according to one of the first set of content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments.

[0038] Referring to Figure 4, operations may determine expected locations of a watermark(s) or other trademarks ("mark(s)") using metadata from a metadata repository 408 which may be generated in part using metadata extracted from the content and / or through machine learning from earlier processing of other content. Video from the content is extracted and then clipped and / or cropped 400, e.g., using the expected locations of the marks to determine where to start looking for marks. Results are further processed to extract 402 video frames to be used for testing. The extracted frames may be replicated and provided in parallel as inputs to a plurality of optical character recognition (OCR) algorithms and / or neural network models 404, such as the ResNet50 product or other convolutional neural network.

[0039] The plurality of OCR algorithms and / or neural network models 404 can be adapted / trained to identify different frame content features which are indicative of watermarks or other trademarks ("marks"). Metadata from repository 408 can be indicative of what marks are expected to be present in content, and which may be provided as inputs to the algorithms / models 404. The metadata repository 408 may include a listing of all marks that may be present in content deliverable to IFE systems, and which can be used for training or configuring the algorithms / models 404 and / or the logical rules 406. Output of the algorithms / models 404 may be textual representations of the detected marks and / or may be binary indicators of which marks, if any, among a set of defined marks is present. The output can be processed 406 through one or more rules that define, for example a minimum time duration that one or more marks must be present to pass the testing, may define two or more marks (e.g., owner mark, CSP mark and airline mark) that must be concurrently present and that must be present within a threshold time of each other to pass the testing, or may define other conditions that need to be satisfied to pass the testing.

[0040] The output may be used for scoring 410 which indicates, for example, which conditions are satisfied or failed or more simply provides a binary indication of pass or fail. The scoring 410 may use metadata from repository 408 indicating what marks are expected to be present and / or indicating what rules must be satisfied to pass the testing.

[0041] The output can be provided to operators and / or a reporting / management system, and may be used to adapt the chunking operations 312 as described above to perform one or more processing cycles of the pipeline 300 when one or more QC / QA tests failed. Results of the processing 404, 406, and / or 410 may be fed-back for training of the algorithms / models 404.

[0042] Figure 5 illustrates a block diagram of operational processes for end credit detection in a video stream component of content according to another of the first set of IFE content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments.

[0043] Referring to Figure 5, operations may determine expected locations of end credits based on a defined duration from the end of the movie (e.g., last 10 minutes of a movie) or using metadata from a metadata repository 508 (may be same as 408) indicating where end credits are expected to be located. Metadata may be used to determine an expected style of the end credits, e.g., background characteristics (e.g., color, pattern, etc.) and end credit text characteristics (e.g., color, font, scrolling / static, duration of display, side-by-side configuration, etc. Video from the content is extracted and then clipped and / or cropped 500, e.g., using the expected locations of the end credits. Results are further processed to extract 502 video frames to be used for testing. The extracted frames may be replicated and provided in parallel as inputs to (or different frames are provided to different ones of) a plurality of end credit classifiers 504, e.g., optical character recognition (OCR) algorithms and / or neural network models 504 adapted for identify end credits. As shown in Figure 5, the end credit classifier 504 may include a plurality of neural network models (1 ...n), such as the ResNet50 product or other convolutional neural network models.

[0044] The end credit classifier 504 can be adapted / trained to identify different frame content characteristics which are indicative of end credits. Metadata from repository 408 which is indicative of end credits expected to be present may also be provided as inputs to the end credit classifier 504. The characteristics indicative of end credits may include sequences of text displayed for time duration(s) that within a defined duration range that istypical of displayed pages of end credits, presence of scrolling text, presence of titles that are indicative of end credits (e.g., "director", "editor", "designer", "producer", "writer", "associate producer", "composer", "cinematographer", "cast", "actor", "role", etc.). The characteristics may be based on metadata that indicates names of persons associated with one or more of the above titles.

[0045] Output of the end credit classifier 504 may include a confidence indication (e.g., likelihood score) that end credits are present, and may further indicate what characteristics were identified in the frames, e.g., list of titles and names, how long identified titles / names remained present, whether scrolling is present, etc.). The output can be processed 506 through filter operations and end-credit detection logic, which may include one or more rules that define, for example, the defined time duration range(s) that sequences of end-credit text are expected to be displayed, whether endcredits are expected to be scrolling, what titles and / or person / character names are expected to be present, etc.

[0046] The output may be used for scoring 510 which indicates, for example, which conditions are satisfied or more simply indicates pass / fair using a binary value. The scoring 510 may use metadata from repository 508 which is indicative of the defined time duration range(s) that sequences of end-credit text are expected to be displayed, whether end-credits are expected to be scrolling, what titles and / or person / character names are expected to be present, etc.

[0047] Metrics may also be output which indicate the computer resources used and duration for processing per hour of movie, error between expected and actual start of end credit time, accuracy of detecting existence of end credits, etc.

[0048] The output can be provided to operators and / or a reporting / management system, and may be used to adapt the chunking operations 312 as described above to perform one or more processing cycles of the pipeline 300 when one or more QC / QA tests failed. Results of the processing 504, 506, and / or 510 may be fed-back for training of the end credit classifier 504.

[0049] The operations of Figures 4 and 5 may be used to identify and log other types of marks, credits, etc. that are identifiable in the content and which are associated with organizations, such as whether the movie includes an indication of approval by the People for the Ethical Treatment of Animals (PETA) organization, attribution to one or more trade unions (e.g., Screen Actors Guild, National Writers Union, etc.), copyright notifications, governmental notifications (e.g., government copyright warning), etc. Such logs can be output to other systems.

[0050] Figure 6 illustrates a block diagram of operational processes for spoken language detection in an audio stream component of content according to another of the first set of IFE content QA / QC processes shown in Figures 2 and 3 in accordance with some embodiments.

[0051] Referring to Figure 6, operations perform chunking 600 to extract one or more defined length (e.g., 20 seconds) chunks of each of the different language audio tracks of the content. The extracted chunk(s) from each of the different language audio tracks may be provided in parallel as inputs to a plurality of spoken language identification models 604, where each model 604 is trained / adapted to identify a different one of a set of languages.Alternatively, different language chunks can be sequentially provided to a single model 604 that is modified between sequence cycle to be adapted to identify a different one of the set of languages. The spoken language identification models 604 may be based on the SpeechBrain model open- source conversational Al toolkit based on PyTorch. In one example embodiment, the SpeechBrain model is trained using VoxLingua107 dataset which contains 107 languages. The model inputs an audio track and outputs a prediction of the top language and most dominant language identified in the input audio track.

[0052] Output of the spoken language identification models 604 may include confidence scores indicating likelihood that associated ones of the languages in the set is present in the audio tracks, and may indicate which languages are present in which tracks. Metadata from a metadata repository 608 (may be same as 408 / 508) which is indicative or which set of languages are expected to be present in the audio tracks of content, may be used toselect which of a set of spoken language identification models 604 are to be used for processing the multi-language audio chunks.

[0053] Output of the spoken language identification models 604 may be provided to aggregation logic 606 that determines the dominate language or set of more dominate languages present in each of the audio tracks, and which may be used for scoring 610 which indicates, for example, which conditions are satisfied. The scoring 610 may use the metadata from repository 608 to determine, for example, whether one or more rules are satisfied, e.g., are each of the languages in an expected set defined by the metadata actually present in the audio tracks.

[0054] Metrics may also be output which indicate the computer resources used and duration for processing per hour of movie, error between expected and actual spoken languages, accuracy of detecting spoken languages, etc.

[0055] The output can be provided to operators and / or a reporting / management system, and may be used to adapt the chunking operations 312 as described above to perform one or more processing cycles of the pipeline 300 when one or more QC / QA tests failed. Results of the processing 604, 606, and / or 610 may be fed-back for training of the spoken language identification models 604.

[0056] For example as describe above, failure to detect one expected language in chunks extracted from the language track of a movie may be due to high background noise, a non-dominant language being temporarily spoken instead of or more loudly than the dominate (expected) language for the language audio track, or another scenario occurring that prevents proper detection of the expected language. That scenario is therefore likely to affect and prevent accurate detection of the other expected languages in the existing audio chunks from the other language audio tracks. The failure to detect one language or a threshold number of languages can be used as a condition to initiate sliding of the window(s) where chucking is again performed on each of the language audio tracks, increase the number and / or duration of the chunks, and trigger re-processing of the new audio chunks for each of the language audio tracks through the spoken language identification models 604.

[0057] Figure 7 illustrates a simplified system block diagram of an IFE content provider platform 104 that processes IFE content for delivery to components of an IFE system 700 according to some embodiments.

[0058] Referring to Figure 7, the IFE system 700 includes wired-network video display units (VDUs) 720 and / or wireless-networked VDUs / PEDs 722 which communicate through respective seat electronics boxes 712 and wireless router(s) 714 (e.g., WiFi access point and / or cellular access point (e.g., a pico-cell radio base station) of distribution components 710. The VDUs can include, without limitation, seat video units which are mounted to individual seats, e.g., seatbacks or armrests. Example PEDs transportable onto the aircraft can include, without limitation, cellular phones, tablet computers, and laptop computers. The distribution components 210 may communicate through networks 740, e.g., Ethernet, with a content server 750 and / or one or more wireless communication interface(s) 730. The content server 750 can provide broadcast and / or on-demand distribution of passenger-requested content to passengers. The wireless communication interface(s) 730 communicate with ground-based radio access network(s) 772 through satellite networks and / or terrestrial (e.g., cellular (e.g., NR 4G, 5G, etc.), WiFi, etc.). In some optional embodiments, the IFE system 700 communicates through the wireless networks with an IFE system operations platform 105 which controls access to content from an IFE content provider platform 104 and / or other networked content servers such as Internet content servers 774.

[0059] Figure 8 illustrates a simplified system block diagram of components of an IFE content provider platform 104 which operate according to some embodiments. The content provider platform 104 may include at least one processor (processor) 800, at least one memory (memory) 810 containing program code 812, and at least one network interface (network interface) 824. The processor 800 includes one or more data processing circuits, such as a general purpose and / or special purpose processor (e.g., microprocessor and / or digital signal processor), and may include one or more neural network circuits which are configured based on one or more neural network models. The processor 800 is configured toexecute the program code 812 in memory 810, described below as a computer readable medium, to perform some or all of the operations and methods that are described herein for one or more of the embodiments. Memory 810 may include an internal or external content buffer 814 where content can be temporarily or more permanently stored for further processing as described above and distribution to the IFE system 700. Platform 104 may include machine learning models 830 which may include neural network models 840 and / or rule-based models 850.

[0060] The network interface 824 may connect through the ground networks 770 with the various CSPs 102 and / or content owners 100, and provide processed content to the IFE system 700 on various aircraft through portable physical mass memory content devices and / or one or wireless communication networks, e.g., satellite, cellular, etc. Platform 104 may include a display device 822 and a user input interface 820.Further Embodiments and Definitions

[0061] In the above-description of various embodiments of the present invention, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense expressly so defined herein.

[0062] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another node, it can be directly connected, coupled, or responsive to the other element or intervening element may be present. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elementpresent. Like numbers refer to like element throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0063] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id est," may be used to specify a particular item from a more general recitation.

[0064] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor of a general purpose computer circuit, special purpose computer circuit, and / or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components withinsuch circuitry to implement the functions / acts specified in the block diagrams and / or flowchart block or blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0065] These computer program instructions may also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks.

[0066] A tangible, non-transitory computer-readable medium may include an electronic, magnetic, optical, electromagnetic, or semiconductor data storage system, apparatus, or device. More specific examples of the computer-readable medium would include the following: a portable computer diskette, a random access memory (RAM) circuit, a read-only memory (ROM) circuit, an erasable programmable read-only memory (EPROM or Flash memory) circuit, a portable compact disc read-only memory (CD- ROM), and a portable digital video disc read-only memory.

[0067] The computer program instructions may also be loaded onto a computer and / or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer and / or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of the present invention may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.

[0068] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact beexecuted substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0069] Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, the present specification, including the drawings, shall be construed to constitute a complete written description of various example combinations and subcombinations of embodiments and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

[0070] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present invention. All such variations and modifications are intended to be included herein within the scope of the present invention.

Claims

Claims:1 . A method performed by a computer system of a content service provider for executing a quality assurance and / or quality control (QA / QC) system module, the method comprising: performing at least one media verification procedure on a media content; and generating a QA / QC score based on results of the performed at least one media verification procedure; and determining if the media content satisfies a QA / QC standard based on the QA / QC score.

2. The method of Claim 1 , wherein media content comprises at least one of the following: digital video media; digital image media; and digital audio media.

3. The method of Claim 1 , wherein the performing of the at least one media verification procedure on the media content, comprises at least one of: confirming the media content contains a defined indication of a start of the media content and further contains a defined indication of an end of the media content; confirming the media content contains a forensic mark occurring in a defined time range when the media content is played out; confirming a set of language tracks contained in the media content satisfies a required set of language tracks; confirming a video component and an audio component of the media content are synchronized; confirming the media content contains a subtitle track synchronized to a corresponding audio track; and confirming the media content synopsis satisfies a defined rule.

4. The method of Claim 3, wherein the confirming that the media content contains the defined indication of the start of the media content and further contains the defined indication of the end of the media content, comprises: performing video frame extraction on the media content; processing the extracted frames through at least a start-content classifier and an end-credit classifier; and filtering the output of the processing through logical operations that confirm that the start of the media content and the end of the media content are present.

5. The method of Claim 4, wherein the confirming that the media content contains the defined indication of the start of the media content and further contains the defined indication of the end of the media content, further comprises: obtaining metadata identifying at least one of: indication of where end credits are expected to be located in the media content; background image characteristics expected in video frames of the media content containing end credits; and / or end credit text characteristics in video frames of the media content containing end credits; and adjusting parameter of the end-credit classifier based on the metadata before processing the extracted frames through at least a startcontent classifier and an end-credit classifier.

6. The method of Claim 5, further comprising: determining from the metadata, characteristics of how text is expected to move in video frames of the media content containing end credits and what text font is expected to be used in video frames of the media content containing end credits; and adjusting parameter of the end-credit classifier based on the metadata before processing the extracted frames through at least a startcontent classifier and an end-credit classifier.

7. The method of Claim 3, wherein the confirming the media content contains the forensic mark occurring in the defined time range when the media content is played out, comprises: performing video frame extraction on a portion of the media content corresponding to the defined time range; processing the extracted video frames through an optical character recognition (OCR) module and / or a machine learning mark identifier; and aggregating the output of the processing through logical operations that confirm that the forensic mark is present in the media content in the defined time range.

8. The method of Claim 3, wherein the confirming the media content contains the forensic mark occurring in the defined time range when the media content is played out, comprises: confirming the forensic mark remains displayed for at least a minimum time duration during playout of the media content.

9. The method of Claim 3, wherein the performing of the at least one media verification procedure on the media content, comprises: confirming the media content contains a first forensic mark and further contains a second forensic mark; and confirming that the first forensic mark is displayed concurrently with or within a threshold time of when the second forensic mark is displayed during playout of the media content.

10. The method of Claim 3, wherein the confirming the set of language tracks contained in the media content satisfies the required set of language tracks, comprises: extracting a set of audio tracks from the media content;processing each of the extracted audio tracks in the set using a spoken language machine learning model adapted to identify defined types spoken languages; and aggregating the output of the processing through logical operations that confirm the set of language tracks contained in the media content satisfies the required set of language tracks.11 . The method of Claim 10, further comprising: obtaining metadata identifying a set of expected spoken languages for the media content; and adjusting parameters of the spoken language machine learning model based on the metadata before processing each of the extracted audio tracks in the set using a spoken language machine learning model adapted to identify defined types spoken languages.

12. The method of Claim 3, wherein the confirming the set of language tracks contained in the media content satisfies the required set of language tracks, comprises: extracting a set of audio tracks from the media content; processing the extracted audio tracks in the set through a plurality of spoken language machine learning models each adapted to identify different types of spoken languages; and aggregating the output of the processing through logical operations that confirm the set of language tracks contained in the media content satisfies the required set of language tracks.

13. The method of Claim 1 , further comprising: extracting from the media content a first chunk having a first duration; performing the at least one media verification procedure on only the first chunk of the media content;generating a first QA / QC score based on results of the performed at least one media verification procedure on only the first chunk of the media content; determining if the media content satisfies a QA / QC standard based on the first QA / QC score; and responsive to determining the media content does not satisfy the QA / QC standard based on the first QA / QC score, further performing extracting from the media content a second chunk having a second duration that is greater than the first duration, performing the at least one media verification procedure on only the second chunk of the media content, generating a second QA / QC score based on results of the performed at least one media verification procedure on only the second chunk of the media content, and determining if the media content satisfies the QA / QC standard based on the second QA / QC score.

14. The method of Claim 13, responsive to determining the media content does not satisfy the QA / QC standard based on the second QA / QC score, further comprising: extracting from the media content a third chunk having a third duration greater than the second duration; performing the at least one media verification procedure on only the third chunk of the media content; generating a third QA / QC score based on results of the performed at least one media verification procedure on only the third chunk of the media content; and determining if the media content satisfies the QA / QC standard based on the third QA / QC score.

15. The method of Claim 1 , further comprising:extracting from the media content a first chunk corresponding to a first window location within the media content; performing the at least one media verification procedure on only the first chunk of the media content; generating a first QA / QC score based on results of the performed at least one media verification procedure on only the first chunk of the media content; determining if the media content satisfies a QA / QC standard based on the first QA / QC score; and responsive to determining the media content does not satisfy the QA / QC standard based on the first QA / QC score, further performing extracting from the media content a second chunk corresponding to a second window location within the media content that at least partially does not overlap the first window location within the media content, performing the at least one media verification procedure on only the second chunk of the media content, generating a second QA / QC score based on results of the performed at least one media verification procedure on only the second chunk of the media content, and determining if the media content satisfies the QA / QC standard based on the second QA / QC score.

16. The method of Claim 15, responsive to determining the media content does not satisfy the QA / QC standard based on the second QA / QC score, further comprising: extracting from the media content a third chunk corresponding to a third window location within the media content that is time shifted by a threshold time duration away from the second window location and which does not overlap either of the first and second window locations within the media content;performing the at least one media verification procedure on only the third chunk of the media content; generating a third QA / QC score based on results of the performed at least one media verification procedure on only the third chunk of the media content; determining if the media content satisfies the QA / QC standard based on the third QA / QC score.

Images