A system for placing dynamic advertisements and content in video streams

Abstract

The invention relates to a system and method for dynamically placing advertisements or content on all kinds of video broadcasts. The invention redefines how ads can be integrated into video content. Providing a solution that is both viewer-friendly and contextually integrated, the invention aims to improve the overall viewing experience while providing a more effective platform for advertisers. The invention achieves all this by using computer vision, artificial intelligence, and augmented reality techniques.

Description

[0001] A SYSTEM FOR PLACING DYNAMIC ADVERTISEMENTS AND CONTENT IN VIDEO STREAMS

[0002] Technological Field:

[0003] The invention relates to a system and method for dynamically placing advertisements or content on all kinds of video broadcasts.

[0004] State of art:

[0005] Today, there are hundreds of digital platforms where video broadcasts are served for end-user viewing. These video broadcasts can be live broadcasts or taped broadcasts. Platforms such as Netflix and Prime Video can be given as examples. The invention that is the subject of the description defines a system that allows integrating advertisements and content using artificial intelligence and augmented reality technologies on all kinds of videos, but not limited to those mentioned here.

[0006] In state of the art, advertising contents are included in the video while it is being prepared. In this case, advertisements cannot be revised according to audience, geographical location or time. However, in today's world, advertisements must vary according to different parameters in order to be successful.

[0007] In the state of the art, advertising contents in videos interfere with video streams. For example, the video is paused and the user is shown an advertisement. This situation causes advertisements to be perceived as annoying by users and causes the viewer to become disconnected from the context. In the state of the art, advertisements in videos are not integrated with the video content. The main reason for this is ad selections that are disconnected from the video, not positioned correctly, and far from the subject of the video, as well as the failure to position the ad on the video as if it were a part of the video content.

[0008] In the literature research, the PCT patent numbered WO0237828 (A2) was encountered. Although it is mentioned in said patent that advertising content is placed within the video stream, advertisements cannot be placed within the video in a moving, scaled and 3D aligned manner. For this reason, advertisements disturb the visual flow within said video content and negatively affect the user experience.

[0009] The digital advertising space is witnessing a transformative era with the emergence of artificial intelligence (Al), machine learning and augmented reality (AR). As can be seen, traditional advertising methods in video content often result in a disjointed viewer experience, with ads appearing intrusive or out of context. The invention that is the subject of the description aims to solve this problem.

[0010] As a result, a new system that goes beyond the state of the art eliminating its disadvantages is needed.

[0011] Brief Description of the Invention:

[0012] The invention is a system that goes beyond the state of the art, eliminates its disadvantages and includes additional advantages in order to integrate advertisements and content into all kinds of videos.

[0013] In the current digital advertising environment, the integration of ads into video content is largely based on predetermined, static placements that lack personalisation and context awareness. This traditional approach often leads to a suboptimal viewer experience and reduced engagement with advertised content. The rapid development of artificial intelligence and AR technologies offers the opportunity to revolutionise this field. The invention that is the subject of this description takes advantage of the aforementioned technological developments to dynamically insert contextually relevant 3D ads into video streams, increase viewer engagement and provide a more fluent advertising experience.

[0014] The technology is based on advanced artificial intelligence algorithms, machine learning techniques and computer vision capabilities. The system that is the subject of the invention intelligently processes video streams and determines the most suitable moments and locations for ad placement. The use of augmented reality also allows ads to be seamlessly incorporated into video content, making them appear natural and in harmony with the surrounding environment.

[0015] The system that is the subject of the invention differs from traditional methods by using advanced techniques in computer vision, especially in ArUco marker detection. By means of ArUco markers, the system recognises predefined markers within video frames. In this way, areas where advertisements can be placed are determined.

[0016] Ads are selected dynamically, not only based on the viewer's profile, but also taking into account geographical location and contextual relevance of the video content. In the invention, artificial intelligence technology is used for this purpose. This ensures that ads are not only tailored to the viewer, but also fit seamlessly into the narrative and aesthetic of the video stream.

[0017] The main aim of the invention is to redefine how ads are integrated into video content. Providing a solution that is both viewer-friendly and contextually integrated, the invention aims to improve the overall viewing experience while providing a more effective platform for advertisers. The invention paves the way for a future of personalised and context-aware digital advertising.

[0018] Description of Drawings

[0019] The invention will be explained with reference to the attached figures, so that the features of the invention will be understood more clearly. However, this is not intended to limit the invention to these particular embodiments. On the contrary, it is intended to cover all alternatives, modifications and equivalences that can be included within the scope of the invention as defined by the attached claims. It should be understood that the details shown are for the sole purpose of illustrating preferred embodiments of the present invention and are intended to provide the most useful and easily understandable description of both the embodiment of the methods and the rules and conceptual features of the invention. In these drawings;

[0020] Figure - 1 Schematic view of the algorithm explaining the operation of the system that is the subject of the invention.

[0021] The figures that will help understand this invention are numbered as indicated in the attached picture and are given below with their names.

[0022] Description of the Invention:

[0023] In this detailed explanation, the system that is the subject of the invention is explained only for a better understanding of the subject, with examples that will not create any limiting effect.

[0024] The system that is the subject of the invention is used to integrate advertisements into all kinds of videos. Each step in this system is carried out through a processor hardware. Additionally, the system may include a server for publishing and a database for storing advertisement-integrated videos. As seen in Figure 1 , the process begins with the entry of the video to be integrated into the system. Subsequently, each frame in the video is processed with image analysis techniques. During the processing of the video, ArUco markers are used to determine the areas where the advertisement will be placed. Then, using artificial intelligence and augmented reality technologies, advertisement selection is made by taking into account the area determined by ArUco markers, the audience of the video, the geographical region where the video will be broadcast and the context of the video content. The selected advertisement is placed in the area determined by ArUco markers, taking into account the 3 dimensions of the video content. The scale, angle and perspective of the advertisement are taken into consideration during placement. Finally, the colour and lighting of the ad are adjusted using histogram matching techniques and the ad is integrated into the video.

[0025] In the invention, each frame of the video content is analysed to determine potential moments and places on the video suitable for adding ads, using computer vision technique running on a processor. This process involves complex algorithms that can distinguish between scene composition, ongoing action, and available space and various elements in the video, ensuring ad placements are contextually relevant and do not interfere with the viewer experience.

[0026] One of the most important innovations of the invention is its use of ArUco pointers. The ArUco marker is a synthetic square marker consisting of a wide black border and an inner binary matrix that determines its identifier. These markers contain two vectors per marker. These vectors enable determining the position and rotation of the area where the advertisement will be placed in a frame of the video scene. One of the main reasons for using the ArUco marker in the invention is to ensure that the advertisement to be placed in the video is suitable for the perspective of the scene in the video. In this way, the viewer can continue his / her viewing activity without noticing anything abnormal in the advertisement. The invention comprises a processor that selects advertisements using artificial intelligence and augmented reality technologies in accordance with the location determined by the Arllco marker, the context of the content in the video scene, the audience and the geographical area. To give an example of the advertising choices here, if there are restaurants in the scene in the video, the sign advertisement of a restaurant receiving advertising can be placed appropriately in the scene. Likewise, if the person in the video is carrying a beverage in his / her hand, an advertisement for a beverage product that is likely to be consumed by the audience and consumed in the geographical region where the audience is located can be placed on this beverage.

[0027] The most important step after ad selection is to physically place the ad at the appropriate angle, scale and rotation in the area determined by ArUco pointers. To do this, the ad goes through a transformation process to fit into the 3D space of the video. This process is achieved by using homography and perspective distortion techniques, which allow 2D advertising images or messages to be aligned to 3D coordinates determined by ArUco markers. Homography and perspective distortion techniques are applied to the advertising image with a processor and adapted to the 3D coordinates of the location determined by ArUco pointers. This allows the ad to adapt to the scale, rotation and perspective of the scene and make the ad appear like a natural part of the original environment. In this way, the advertisement creates a perfect harmony with the video content or scene.

[0028] In the invention, colour and light adjustments are made on the advertising visual to complete the perfect harmony between the advertisement and video content. For this purpose, one processor detects the colour and light values in the video scene where the advertisement will be placed, and another processor applies the detected colour and light value to the advertisement image. Histogram matching techniques are used in the processor to ensure that the ad image is consistent with the lighting and colour palette of the video scene in which the ad will be placed. This attention to detail in colour and lighting setting is key to maintaining the illusion that the ad is an integral part of the scene. Histogram matching technique is the application of the necessary operations to map the histogram values of the input image with these histogram values after defining a histogram values. By means of this method, more effective improvements can be achieved on images.

[0029] The system for placing advertisements and content in videos that is the subject of the invention offers a wide range of applications in various industries, demonstrating its versatility and adaptability. Said areas of use are not limited to the following, these are given only as an example.

[0030] • Advertising on video streaming content: With the invention, it is possible to adapt advertisements to different target audiences in live broadcasts or recorded video content, increase audience interaction and offer a more personalised viewing experience.

[0031] • Content Injection into Movies and TV Series: With the invention, it is possible to provide post-production integration of product placements or information content in films and television series. This could revolutionise the way content creators and advertisers collaborate, offering new revenue streams and marketing opportunities.

[0032] • Advanced Sports Broadcasting: With the invention, it is possible to add dynamic ads to live and recorded sports events. The invention allows for non-intrusive ad placements that do not disrupt the viewing experience, such as integrating ads into the playing field or arena so that they appear natural and part of the event.

[0033] • Interactive Training Content: With the invention, it is possible to integrate educational material advertising or additional content into educational videos. This can make educational content more engaging and customised based on the audience's learning patterns and preferences.

[0034] • Advanced Ad Placement with Generative Artificial Intelligence: In addition to computer vision and augmented reality, the inventive system uses generative artificial intelligence to improve the ad placement process. This advanced technology allows dynamic replacement of objects in the video stream with predefined ads. For example, a generative Al algorithm can identify a generic object in a scene, such as a beverage can or billboard, and seamlessly replace it with a specific branded product or message in real time. This capability not only increases the relevance of ads, but also opens up new possibilities for creative and context-aware ad integrations.

[0035] • Advanced Content Change Advertising with Image Processing, AR and Generative Al: The integration of generative artificial intelligence in the inventive system opens up numerous possibilities for context modification in various applications. This technology can meet different needs by dynamically changing or enhancing elements in a video stream: o Education and Training Programs: With invention, it is possible to customise training content by changing visual elements in instructional videos to fit specific learning objectives or cultural contexts. o Personalising the Entertainment: With the invention, it is possible to personalise movies or TV series for different audiences by changing background elements, product placements and even the language of signs within the scene. o Corporate and Marketing Presentations: With the invention, it is possible to adapt corporate videos to different regional markets by changing certain products, logos or environmental elements to resonate more strongly with local audiences. o Interactive Gaming and Virtual Reality: With the invention, it is possible to improve the gaming and VR experience by dynamically changing in-game visuals or environments and offer users a more fluent and personalised experience. There is no limitation regarding the area of use of the invention. The above fields are given as examples only. With the invention, it is possible to integrate advertising visuals, advertising messages and various contents into any video.

[0036] One of the alternative features used in the invention is the ability to change the sound in the video. For this, there is a processor that processes the sounds in the video with artificial intelligence. Said processor can be used to change the narrative in educational content to suit different language preferences or change the dialogue in entertainment media for regional localisation, as well as to advertise a slogan of a brand. While watching the video, the viewer can listen to the audio integrated as part of the video. For example, if a restaurant is mentioned in the original video scene, a specific restaurant name can be added as audio.

[0037] Invention expands its capabilities beyond traditional boundaries by making use of generative Al for versatile content modification and audio changes in both 2D and 3D formats. This flexibility enables a wide range of applications in different sectors.

[0038] Whether it's changing the 2D image on a billboard in a video or transforming the 3D layout of a scene, our technology meets a variety of content modification needs. This includes adjusting aesthetic elements in entertainment media or updating product designs in real time for advertising purposes.

[0039] The invention can be used in everything from advertising where products and messages are dynamically integrated to non-commercial applications such as customisation of educational content, entertainment, corporate presentations and interactive games. Our technology offers a transformative approach to how content is experienced and interacted with.

[0040] The inclusion of both 2D and 3D content modification as well as voice changing capabilities in the system of the invention is a significant change in digital content interaction. This technology is not limited to advertising but extends to any application that requires dynamic content changes. Generative Al is known to be not only an advertising tool but also a versatile tool for creative expression and personalised experiences in various fields.

[0041] This advanced technology enables a new content creation and consumption experience that is more adaptable, interactive and tailored to the specific needs and preferences of different audiences and applications.

[0042] The breakthrough not only revolutionises digital advertising, but also marks a significant advance in the broader field of video content creation and customisation. This technology transcends traditional boundaries, providing versatile solutions for educational content, entertainment, corporate marketing and interactive gaming.

Claims

CLAIMS1. A method to dynamically place ads or content on any type of video, comprising the process steps of:• analysing each frame in the video content using computer vision technique to identify potential moments and places on the video suitable for adding ads,• determining the areas where the advertisement will be placed in the video frames by using ArUco markers operated by a processor,• selecting advertisements using augmented reality and artificial intelligence, taking into account the area determined by ArUco markers, the audience of the video, the geographical region where the video will be broadcast and the context of the video content,• aligning the visual or message of the selected advertisement to the 3D coordinates determined by ArUco markers by processing it on a processor with homography and perspective distortion techniques,• adjusting the colour and light of the advertising image or message using histogram matching techniques on a processor to ensure that it is consistent with the lighting and colour palette in the video scene where the advertisement will be placed, and• placing the advertisement or message, the colour and light adjustment of which is completed by a processor, into the area determined with ArUco markers.

2. Method according to Claim 1 , wherein scene composition, ongoing action and available space and various elements in the video are distinguished using computer vision technique.

3. Method according to Claim 1 , wherein the position and angle value of the appropriate advertising area in the video scene is determined with the two vectors of an ArUco marker.

4. Method according to Claim 1 , wherein the advertising visual or message is adapted to the three-dimensional coordinates of the location determined by ArUco markers using homography and perspective distortion techniques, so that the advertisement adapts to the scale, rotation, and perspective of the scene.

5. Method according to Claim 1 , wherein the video with the integrated advertising image or message is broadcast on a server and saved in a database.

6. Method according to Claim 1 , wherein, at a certain moment in the video, a processor changes the audio data for advertising purposes using artificial intelligence techniques.

Images