Method and system for enriching audiovisual content

Through artificial intelligence-driven enrichment methods, we can identify and associate elements in video content and create containers that can be accessed in video, solving the problem of difficulty in real-time access to products or services in audio-visual content, and realizing real-time purchases and brand promotion for users Effect.

CN119096237APending Publication Date: 2024-12-06SKOLEOM PLATFORM SAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380020395.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-02-04
Filing Date
2023-02-02
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art cannot realize real-time access to products or services in audio-visual content, and it is difficult for video creators to provide simple, direct purchase or service access methods in video content.

Method used

Through an AI-driven enrichment method, identify elements in video content, classify and associate them with relevant digital resources or services, create containers, integrate them into video content, allowing users to graphically select and perform and perform video content while watching Visit relevant additional videos, online stores, or media articles.

Benefits of technology

It realizes real-time access and purchase of products or services in video content by users, improves the ability of video creators to provide audiences with interactiveness and business opportunities, and enhances the visibility and accessibility of brands and commercial provisions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096237A_ABST
    Figure CN119096237A_ABST
Patent Text Reader

Abstract

The invention relates to a method for enriching initial video content, said method comprising:-a step of playing said initial video content, designed to identify predetermined elements therein; a step of classifying the identified elements; a step of searching and selecting digital resources and / or services associated with the identified and classified elements; a step of creating one or more containers, associated with each of the identified and classified elements, intended to contain the selected digital resource and / or service; a step of integrating the one or more containers thus created into the initial video content in order to deliver the enriched video content.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for enriching audiovisual content. The present invention also relates to a system, in particular a digital platform, for implementing the method.

[0002] The fields of the invention include digital communications, digital marketing, video and e-commerce. Background Art

[0003] A dramatic growth in the provision of audiovisual content is currently being observed. More and more of this audiovisual content is associated with brands and commercial offerings.

[0004] Today, if a person viewing such an audiovisual content wishes to purchase a product or service seen therein, they must be redirected via a digital communications network to a sales site on the Web and use a payment instrument. Examples include "Click and and Google ”.

[0006] A disadvantage of current methods is that they do not give users real-time access to product or service purchasing services.

[0007] Furthermore, it is difficult for video content creators to include in their video content the technical means to provide viewers of their video creations with simple, direct purchasing access or service access.

[0008] One of the main objectives of the present invention is to remedy this drawback by proposing a novel video content enhancement method, which is robust and easy to use.

[0009] Disclosure of the Invention

[0010] This object is achieved by a method for enriching initial video content, the method comprising the following steps:

[0011] - playing the initial video content, designed to identify predetermined elements therein,

[0012] - classifying said elements thus identified,

[0013] - search and select digital resources and / or services associated with the identified and classified elements,

[0014] - creating one or more containers associated with each identified and classified element intended to contain the selected digital resources and / or services,

[0015] - Integrating the container thus created into said initial video content to deliver the enriched video content.

[0016] The steps of identifying and classifying can advantageously implement artificial intelligence techniques.

[0017] The step of integrating into the original video content may be arranged to graphically represent the container when viewing the enriched video content.

[0018] The enrichment method according to the present invention may also include: creating tactile selection areas in the enriched video content, the tactile selection areas being associated with one or more selected elements to which one or more containers have been associated, so that a tactile action on one of the selection areas causes the container associated therewith to be displayed.

[0019] The container creation step may be arranged to include additional videos and / or online stores and / or searchable documents, such as media articles, in the container. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be better understood with reference to the following drawings:

[0021] [ Figure 1 ] Figure 1 is a block diagram of an embodiment of a method for enriching video content according to the present invention;

[0022] [ Figure 2 ] Figure 2 is an example of a block diagram of a prediction and recognition algorithm implemented in a method for enriching video content according to the present invention;

[0023] [ Figure 3 ] Figure 3 The first four steps of an example of an enrichment method according to the present invention are shown;

[0024] [ Figure 4 ] Figure 4 Shows Figure 3 Three further steps in the exemplary embodiment shown in .

[0025] definition

[0026] iFrame: The name given to an HTML tag used in computer languages ​​to integrate the contents of another HTML page into an HTML page.

[0027] API: stands for "Application Programming Interface": an application programming interface that includes a set of definitions and protocols that facilitate the creation and integration of application software.

[0028] Bounding box: A rectangle aligned with the axis lines. This is the simplest type of closed planar shape, represented by two points containing the minimum and maximum coordinates of each axis line.

[0029] PyTorc: An open source machine learning framework that accelerates the transition from research prototyping to production deployment.

[0030] SAAS: stands for "Software as a Service": an application software solution hosted in the cloud and operated by a third party (also called a service provider) outside an organization or company. SAAS solutions can be accessed on demand via an Internet connection.

[0031] Confidence score: The classification confidence score is designed to measure the accuracy of the model in predicting class assignments.

[0032] YOLOv5: A family of composite scaled object detection models trained on the COCO dataset, with simple features for test time augmentation (TTA), model assembly, hyperparameter evolution, and export to ONNX, CoreML, and TFLite.

[0033] Detailed Description of Exemplary Embodiments

[0034] Reference Figure 1 , the enrichment method according to the present invention can be implemented on a service platform 30, which is intended for video content creators and can be obtained in the form of a SAAS application. On this platform 30, the video content creator can enter the initial video creation content 31. The video player 32 is equipped with an identification / authentication device 33 and is linked to a database 34. The video player implements a set of artificial intelligence algorithms 35 that will perform the following operations:

[0035] - a series 36 of recognitions of elements in the initial video content 31 (such as products and / or services, people, places, objects, plants or animals, dates and many other elements) using known artificial intelligence (AI) techniques,

[0036] - classifying these identified elements 37 with reference to predetermined container categories using known learning techniques,

[0037] - creating 38 containers associated with various elements identified and selected by the user of the method of the platform 30,

[0038] - Inserting 39 these containers into the original video stream to deliver the enriched video content 42 from the original video content 31, now comprising 43, 44 a plurality of containers C1, ..., Cn.

[0039] The container contents created in this way are made available by access, for example in the cloud (digital cloud) or via a digital network or the Web, to the marketplaces PM1, ..., Pmn and to the content platforms (in particular the video content platforms PC1, ..., PCk).

[0040] Reference Figure 2 , we will now describe an example of an artificial intelligence algorithm implemented to detect elements (people) in video content.

[0041] The algorithm 100 is created using PyTorch coding tools commonly used in machine learning. In the first step 101, the YOLOv5 compound scaled object detection model is implemented on COCO data. From this model, a bounding box 102 (or B.Box) is used to identify the cropped person image. The next step 103 is the bounding box prediction from the style data 107.

[0042] The prediction 104 is then tested for a confidence score, a known technique in machine learning. If the confidence score is less than a predetermined value X, the bounding box B.Box is ignored 108. On the other hand, if the confidence score is greater than X, the next step 105 is to identify the class confidence, followed by step 109 to detect images containing the expected object.

[0043] Reference Figure 3 and Figure 4 , an example of using the enriched video content by a person viewing or consulting the content on an electronic communication device 1 such as a smartphone, tablet or personal computer will first be described.

[0044] In a first step I, the person consults the video content and locates an element such as a known actor 2. In a second step II, the person then selects this element 2 by touching the screen 1 with a finger of his hand 3.

[0045] In a third step III, the selection causes a container icon 4 to be displayed in a corner of the screen 1 , for example in the upper right corner.

[0046] The person then selects (step IV) the container icon 4 , which causes a window 5 to appear on the screen 1 , which is integrated into the video and comprises three areas 50 , 51 , 52 corresponding respectively to additional videos, an online store and a media article, all three resources being related to the selected element 2 .

[0047] Thus, if the user selects (step V) area 50, a window 20 for viewing the additional video appears and the user can view the additional video directly within the enriched video content.

[0048] If the user selects area 51 (step VI), the online store 21 integrated in the rich video content appears.

[0049] If the user selects (step VII) area 52 , a window 22 for reading a news article appears, which is integrated into the enriched video content.

[0050] The method for generating rich audiovisual content can be implemented in a video creation platform. The method enables video content creators to provide their viewers with opportunities to interact in their created content.

[0051] The video player is available as a SAAS application for both tech novices and video professionals, and allows embedding of branding that creators want to ensure is integrated into your audiovisual content.

[0052] The video player used in the method according to the invention may have the following features:

[0053] - Product / Service Recognition: AI can identify products and services embedded in videos,

[0054] -People Recognition: AI can identify all people in video content,

[0055] - Location recognition: AI can identify the most similar locations in the video and give a precise address in the form of geolocation (GPS),

[0056] - Object, Plant and Animal Recognition: AI can recognize most types of objects, plants and animals in videos.

[0057] - Date recognition: AI can recognize specific dates (mostly,

[0058] -Analyze the clothes the protagonist is wearing, the type of objects, etc...

[0059] - Food and Beverage Recognition: AI can recognize dishes and their ingredients and extract the ingredients of the dish.

[0060] Classification of elements and containers

[0061] The enrichment method is designed to classify the containers created as a result of the recognition process. The artificial intelligence classifies all the recognized elements in the container integrated into the upper right corner of the video.

[0062] This will then yield three supplies for this element:

[0063] - Additional Video: This video is extracted from the YouTube catalog or other videos from the public API, the video closest to the product for sale is integrated into the container. This additional content can be a report, promotional video or instructional video,

[0064] - An offer to purchase a product or service: the product or service can be drawn from millions of products available on the Partner's international marketplace.

[0065] -Article: The article describing the product is automatically generated by artificial intelligence from the information available on the website.

[0066] The text describing these offers can be translated into multiple languages ​​simultaneously.

[0067] database

[0068] For product recognition, the AI ​​selects products available on global e-commerce platforms. If the product is not recognized on one of the platforms, the AI ​​sends a "want to create product" message to the product creation department, which will create said product: size, color, material, etc.

[0069] Artificial intelligence is designed to do the following:

[0070] - "crop out" products from video content to extract photos, and

[0071] Place it on a white background and extract the 3D video rotating on the white background.

[0072] - assign a sales price to a product after benchmarking it against similar products internationally, automatically start the production machines for that product in the production plant,

[0073] - reference available inventory in video content,

[0074] - Publish the product in an online video store and make it available to the public.

[0075] The user of the communication device can touch the screen at any time. This action causes the container to appear on the screen. Beforehand, the user will verify the conditions for use of the product prediction algorithm.

[0076] The video player can be equipped with a biometric fingerprint reader and provide the customer with the option to register data such as banking information, taste, height, weight, measurements, identity, address, functions, etc.

[0077] Once customers verify the elements on their screen with their finger, the ordered products are shipped directly to their homes.

[0078] Examples of containers

[0079] A variety of types of containers can be envisioned. The following is a non-exhaustive list of containers that can be created as a result of the method for identifying in video content.

[0080] 1. People

[0081] 2. Location

[0082] 3. Events

[0083] 4. Objects

[0084] 5. Plants

[0085] 6. Animals

[0086] 7. Cosmetics

[0087] 8. Game Program

[0088] 9. Spirituality and religion

[0089] 10. Personal Development

[0090] 11. Date

[0091] 12. Food and Beverages

[0092] 13. Clothing and accessories

[0093] 14. Automobiles and accessories

[0094] 15. Furniture

[0095] 16. Competition

[0096] 17. Real Estate

[0097] 18. Beauty

[0098] 19. Driving Road

[0099] 20. Voting

[0100] 21. Music

[0101] 22. Animation

[0102] 23. Children

[0103] 24. Technology

[0104] 25. Books

[0105] 26. Banking

[0106] 27. Science

[0107] 28. Education

[0108] 29. Training

[0109] 30. Methods

[0110] 31. Government

[0111] As non-limiting examples, one may contemplate 200 containers for a 3 minute video, 600 containers for a 6 minute video, and over 1000 containers for a video longer than 6 minutes.

[0112] According to the enrichment method of the present invention, the user integrates the video content into the Web platform or SAAS application in the form of a file or a link.

[0113] The artificial intelligence then processes the video content over a period of time equal to or less than (depending on the availability of a queue for this processing) its duration, referencing all the information in a dashboard assigned to the user.

[0114] The platform for implementing the enrichment method according to the invention can be designed to group together multiple services available to creators and named with the term "Transmedia Universe":

[0115] 1. Watch the video

[0116] 2. Create an account

[0117] 3. Download videos

[0118] 4. Retweet from social media accounts

[0119] 5. Call (such as )

[0120] 6. Chat (such as )

[0121] 7. Purchase products and services (such as )

[0122] 8. Monetize video content (e.g. )

[0123] 9. Conference call (such as )

[0124] 10. Payment (similar to Apple )

[0125] 11. Banks (such as )

[0126] 12. Real estate (such as real estate agency)

[0127] 13. Production (similar to )

[0128] 14. Engine (such as )

[0129] 15. Academic (e.g. the school offering the course)

[0130] 16. Voting (e.g. via text)

[0131] 17. Events (such as Live )

[0132] 18. Products (such as )

[0133] 19. Games (similar to )

[0134] 20. And more features and services to be added in the future...

[0135] 1. Video viewing: Each user can watch any video on the platform (music, movies, documentaries, etc.).

[0136] 2. Account creation: Users will have an account that will allow them to manage all their actions on the various social networks (Facebook, Instagram, Twitter).

[0137] 3. Upload: Users can upload their own videos and photos.

[0138] 4. Reposting from social media accounts: The enrichment method allows users of any social network to register by creating an account. It also allows users to easily migrate their accounts, that is, directly use their or or other account signing.

[0139] 5. Call (such as ): Uses a similar method to WhatsApp, allowing for audio and video calls.

[0140] 6. Chat (such as ): Features of this method include messengers that allow you to chat with your contacts, such as

[0141]

[0142] 7. Purchase products and services (such as ): Makes purchasing possible by simply clicking on the selected product on the screen.

[0143] 8. Monetize video content (e.g. ): Each content owner has the opportunity to earn money by posting videos whenever people buy their products. All sales information is recorded directly and in detail in their account.

[0144] 9. Conference call (such as )

[0145] 10. Payment (similar to Apple ): Users can pay for their purchases using this payment application.

[0146] 11. Banks (such as ): Users can open bank accounts and receive bank statements like any other online bank.

[0147] 12. Real Estate: Dedicated service that enables users to sell or buy their properties as in a real estate agency.

[0148] 13. Production (such as ): This method allows you to produce your own videos, series and movies, or any other program that you plan to distribute directly on the platform.

[0149] 14. Engine (such as ): This method develops its own search engine that allows for certain options that others do not offer. For example, the "Search Bar" option enables you to copy a link from a video to extract all products and offer them for purchase, requiring authorization from the video content owner to access that content.

[0150] 15. Academic (e.g., school classes): This option could be developed for all online courses. In the COVID-19 era, teaching can be conducted remotely, allowing the student / teacher relationship to be maintained.

[0151] 16. Voting (eg via SMS): Voting enables the free exchange of messages within the community, as well as voting on competitions, TV shows, etc.

[0152] 17. Events (such as Live ): Live streaming and concerts will be possible on the platform.

[0153] 18. Products (such as ): Design and manufacture a wide range of products in different sectors. This allows the platform to significantly increase its surplus.

[0154] 19. Games (such as ): Online games for people playing on their smartphones and computers.

[0155] 20. More functions and services need to be added in the future.

[0156] Thanks to partnerships with e-commerce, fashion, real estate and other industries, it is possible to promise customers a wide variety of products. For video creators, they now have the opportunity to create short and interesting videos about the listed brands.

[0157] The use of AI helps in revenue optimization and visibility for brands and designers.

[0158] If the video recognition is successful, the predicted number of bins will be displayed in the bin bar on the right side of the screen. The bin bar remains frozen during the observation period. You can touch the screen at any time. This action turns on the bin function.

[0159] If video recognition is not enabled, the container does not automatically appear on the screen. The container bar is therefore deactivated. However, you can touch the screen at any time. This action evokes the container bar, which becomes active on the screen. You can interact with the "container" with three options.

[0160] The artificial intelligence implemented in the enrichment method according to the invention is designed to:

[0161] - Identify all the elements that make up a video.

[0162] - Categorize elements by container under category-based indexing.

[0163] -Determining whether the container was generated by a prospect, customer or advertiser.

[0164] Two bars can be displayed below the video:

[0165] - "Similar Products" entries: the product / service has been identified in the video but not by the customer or advertiser.

[0166] - “Specific Product” bar: The product / service has been identified in the video and is identified as the customer’s or advertiser’s product / service.

[0167] Search Bar

[0168] Users of the enrichment method according to the present invention are also provided with a search bar to enable them to refine the relevance of image or video content that would be associated with a product or service that is desired to encourage purchase or user engagement.

[0169] An analytics API is provided to perform a search function that improves the visibility of a query and determines keywords for displaying relevant videos or images from the Internet.

[0170] It also involves defining query rules for problematic queries, or adjusting search attributes / parameters to address relevance and systemic issues.

[0171] Filters are also provided to establish optimized conversion paths by pre-defining filters on specific keywords based on the most popular filters for video or image searches.

[0172] APIs from online stores, videos or websites (photos, videos, texts) can be stored in the algorithm.

[0173] A link is established between the registered API element and the replicated element in the algorithm.

[0174] Search its database for elements copied by artificial intelligence.

[0175] When an e-commerce platform makes available a product or service whose iFrame cannot be directly integrated in the video content, the product or service can be integrated with this type of iFrame in a dedicated storage so that the iFrame can be fully played without leaving the video (which could otherwise be considered a click and purchase).

[0176] It should be noted that the data integrated and thus replicated in the algorithm (e.g. more than 500 million products - images, texts, videos) may require huge servers and huge computing time (identification, linking, recording). An algorithm can then be designed to link the release date of the video with the market release date of the product. This has the effect of removing a large number of searches performed by the algorithm for products hosted by the e-commerce platform.

[0177] Of course, the invention is not limited to the examples just described, and many other embodiments may be envisaged without departing from the scope of the invention. In particular, the number of identifiable elements and containers that can be integrated into the enriched video content is limited only by the capacity and power of the computer servers used.

Claims

1. A method for enriching initial video content, the method comprising the following steps: - playing the initial video content, designed to identify predetermined elements therein, - classifying said elements thus identified, - search and select digital resources and / or services associated with the identified and classified elements, - creating one or more containers associated with each identified and classified element intended to contain the selected digital resources and / or services, - Integrating the container thus created into said initial video content to deliver the enriched video content.

2. The method according to the preceding claim, characterized in that The recognition step and the classification step implement artificial intelligence technology.

3. The method according to the preceding claim, characterized in that Said integrating step in said initial video content is arranged to graphically represent the container when viewing the enriched video content.

4. The method according to the preceding claim, characterized in that The method further includes the step of creating tactile selection areas in the enriched video content, the tactile selection areas being associated with one or more selected elements to which one or more containers are associated, such that a tactile action on one of the selection areas causes the container associated therewith to be displayed.

5. The method according to any one of the preceding claims, characterized in that The step of creating a container is arranged to include the additional video in the container.

6. The method according to any one of the preceding claims, characterized in that The step of creating a container is arranged to include the online store in the container.

7. The method according to any one of the preceding claims, characterized in that The step of creating a container is arranged to include the reviewable document in the container.