Method for extracting and inserting picture or video advertisement information

Abstract

The invention discloses a method for extracting and inserting picture or video advertisement information, which relates to the technical field of electronic information, and is characterized in that real-time detection and multi-modal feature extraction are performed on a target object in a picture or video, and accurate matching is realized in combination with an advertisement information database; according to the method and the device, the advertisement data is embedded into a picture or video coding protocol in a structured format, so that the problems of low advertisement and content association degree and poor display effect in the prior art are effectively solved, the consistency of the inter-frame advertisement data is ensured by embedding the complete advertisement data in the key frame and predicting the frame inheritance index, and the advertisement data is more uniform. Meanwhile, two scenes of real-time shooting and later editing are supported, the flexibility and applicability of advertisement putting are greatly improved, a priority management mechanism and a delay trigger strategy are adopted, the user interaction experience is optimized, and excessive interference of advertisements is avoided.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and more specifically, to a method for extracting and inserting picture or video advertising information. Background Art

[0002] In the era of rapid development of digital information, pictures and videos have become the main media for information dissemination and are widely used in social media, advertising marketing, e-commerce, navigation systems and other fields. However, the current technology for embedding specific information (such as advertising data or other interactive information) in picture and video content has the following significant shortcomings: Currently, advertising information in pictures or videos is usually displayed in the form of independent windows or pop-ups, which have a low correlation with the actual playback content and are easy to interrupt the user experience and cause disgust. Many advertising information cannot be deeply integrated with the picture or video content, resulting in low advertising display effects and user conversion rates. Traditional advertising delivery relies on fixed templates or predefined rules and cannot dynamically generate personalized advertisements based on the actual situation of the video or picture content.

[0003] Therefore, in order to solve the above technical problems, the present application proposes a method for extracting and inserting picture or video advertising information. Summary of the invention

[0004] In view of the deficiencies in the prior art, an object of the present invention is to provide a method for extracting and inserting picture or video advertisement information.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for extracting and inserting picture or video advertising information, comprising the following steps: S1: Perform image recognition on the target object in the picture or video; the specific steps are: S1.1: Target area acquisition: using a camera to acquire images or video frames of the target area; S1.2: Target object recognition: Detect the collected images or video frames and identify the target objects contained therein, including places, people and commodities. S1.3: Target feature extraction, extracting multimodal features of the target object, including: Image features, including the shape, color, and texture of the target object; Semantic features, including the category and associated description of the target object; S2: Match the recognition result with the preset advertisement information database. If the match is successful, extract the corresponding advertisement data, including advertisement identifier, advertisement prompt, advertisement URL and priority information; S3: In the encoding protocol of the picture or video, embedding the advertisement data through the extension bit; embedding the advertisement data through the extension bit is achieved through the following steps: S3.1: Determine the extension bit in the encoding protocol of the picture or video; S3.2: Write the advertisement data into the extension bit in a structured format; S3.3: Embed corresponding advertisement data into each frame of the video to ensure inter-frame consistency; S3.4: Generate a checksum for the advertisement data in the extension bit and append it to the file, completing the advertisement embedding operation; S4: The terminal device plays the image or video and parses the advertising data in the extended position, and displays the advertising prompt or jumps to the advertising content when triggered by user interaction.

[0006] Preferably, the extension bit is a data space reserved in a file format or protocol for storing additional information, and the specific steps of writing the advertisement data into the extension bit in a structured format are: structuring the advertisement data into a fixed binary format according to the extension bit determined in S3.1, and directly writing it into the extension bit; The advertisement data structure E is represented as:

[0007] Among them, H is the advertisement identifier, D is the advertisement ID, U is the advertisement URL, the advertisement URL is the link address that jumps after the advertisement is clicked, and P is the advertisement priority. The advertisement data is encoded in the format specified by the protocol.

[0008] Preferably, the method further includes priority management of advertisement data, specifically including: In the case where a single frame contains multiple advertisement data, they are sorted according to the preset priority field; The number of advertisement data displayed in a single frame is limited to no more than 3.

[0009] Preferably, when the expansion bit is limited to store 3 advertisement data, the advertisement data to be displayed is selected according to the priority calculation, and the priority calculation formula is as follows:

[0010] in, is the priority of the tth advertisement, is the original weight of the t-th advertisement, is the sum of all ad weights, is the original weight of the ith advertisement, and n is the total number of advertisements.

[0011] Preferably, in S3.3, corresponding advertisement data is embedded in each frame of the video, and the specific steps are as follows: a. Arrange and serialize the advertising data: Encode the advertising data into binary form according to the established format and write it into the extension bit of the video frame; b. Embedding advertisement data in key frames: the key frame is the first frame of each group of pictures, and the serialized advertisement data is directly written into the extended bit area of ​​the key frame. The key frame stores complete advertisement data, and the prediction frame inherits the advertisement data; c. Predicted frame inherits advertising data: The predicted frame directly references the nearest key frame. Its reference inheritance logic is to record the advertising data index or pointer pointing to the key frame in the extension bit of each predicted frame; d. Advertisement data analysis and display: The decoder extracts advertisement data from the extended bit of the current frame. When the user pauses or clicks on any frame, the advertisement data in the extended bit is called and a menu is generated for the user to select and view advertisement details. e. After the advertisement data is embedded in each group of pictures, the above process is repeated to embed the advertisement data into the key frames of the next group of pictures, and reference inheritance settings are performed on the prediction frames.

[0012] Preferably, the method supports embedding other information besides advertisement data, including: Geographic location information, used for navigation; Background information of the characters, for social media links; Product information, used for e-commerce redirection.

[0013] Preferably, when the user pauses or clicks on the video, the advertisement data in the extended position is parsed and a menu containing advertisement prompts is generated, and the user can choose whether to view the advertisement details according to needs.

[0014] Preferably, the extraction and embedding of advertising data is applicable to the following scenarios: The camera device shoots in real time, extracts the target object through image recognition and embeds the advertising data, or embeds the advertising data in combination with the advertising information database during the post-editing process of the picture or video.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention performs real-time detection and multimodal feature extraction of target objects in images or videos, combines with an advertising information database to achieve precise matching, and embeds advertising data in a structured format into the coding protocol of images or videos, effectively solving the problems of low correlation between advertising and content and poor display effects in the prior art. By embedding complete advertising data in key frames and predicting frame inheritance indexes, the consistency of advertising data between frames is ensured, while supporting both real-time shooting and post-editing scenarios, greatly improving the flexibility and applicability of advertising delivery. Priority management mechanisms and delayed triggering strategies are used to optimize user interaction experience, avoid excessive advertising interference, and achieve deep integration and efficient delivery of advertising information and video or image content. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 This is a schematic diagram of a method flow chart of Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the method flow of Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0017] Embodiment 1: like Figure 1 As shown, this embodiment provides a method for extracting and inserting picture or video advertisement information, including: 1. Extraction and embedding of advertising data in real-time scenarios 1. Data collection and target detection in real-time scenarios Use a camera device (such as a smartphone or camcorder) to capture video frames or pictures in real time and input the captured data into a deep learning model.

[0018] The deep learning model extracts the target object through the following steps: Target object recognition: Use the collected video frames and pictures to identify the target objects (such as people, places or products) in the picture.

[0019] The detection results will contain the location information, category label and feature description of the target object for subsequent matching.

[0020] 2. Target feature extraction and advertising data matching Target feature extraction: Extract multimodal features of the target object, including image features (shape, color, texture) and semantic features (category, associated description).

[0021] Match the detected target features with the advertising information database: Matching logic: Retrieve advertising data related to the target through target categories or features (such as detecting a certain brand of product).

[0022] Ad data content: ad identifier (H), ad prompt (such as "Buy this product and enjoy 20% discount"), ad URL (such as a jump link to an e-commerce website), and ad priority (P).

[0023] If multiple target objects are detected in a single frame, the ads are sorted according to the ad priority field, and only the three ads with the highest priority are retained.

[0024] 3. Advertising data is embedded in pictures and video files Expansion bit allocation: In the video encoding protocol (such as MP4 file), a udta box (the prior art is not described here) is selected as an extension bit to store advertisement data.

[0025] Data structuring and writing: Serialize the advertisement data into binary form according to the following structure:

[0026] Write the extension bit of each frame to ensure that all frames containing the target object have the advertising data embedded.

[0027] 4. Advertising data is embedded in key frames and predicted frames Inter-frame consistency: A video file consists of multiple frames, including key frames (I frames), predicted frames (P frames), and bidirectional predicted frames (B frames); Embed complete advertising data in key frames (I frames); The advertisement data index of the key frame is recorded in the prediction frame (P frame, B frame) to realize the inheritance and reference of the advertisement data.

[0028] 5.Terminal device advertising analysis and user interaction When the terminal device plays the video, the decoder parses the advertising data in the extension bit of each frame: Pause or click interaction: If the user pauses the video or clicks on a frame, the ad identifier in the current frame is detected and a dynamic menu is generated; The menu displays the ad prompt, and after the user clicks it, they are redirected to the page corresponding to the ad URL (such as a browser or e-commerce application).

[0029] 6. Data security and privacy protection In the process of embedding advertisement data, the advertisement data is encrypted by an encryption algorithm (such as existing technologies such as AES) before embedding to ensure that only authorized devices can decode it.

[0030] Users can choose to turn on or off the ad embedding function to protect privacy data.

[0031] 2. Ad embedding in post-editing scenarios 1. Content editing and object detection Load the video or image file in the post-editing software and perform target detection on the content through the image recognition module; this is mainly aimed at the situation where the editing software detects the presence of a specific brand of goods in a certain scene. The detection results include the target location, category and related features.

[0032] 2. For ad matching and selection: According to the detected target features, they are matched with the preset information in the advertisement information database. If multiple related advertisements are detected, they are sorted according to the priority field and only the advertisement with the highest priority is selected for embedding.

[0033] 3. For advertising data embedding: Embed advertisement data in the APPn segment of an image (such as a JPEG file): The advertisement data is written into the APP1 or APP2 segment in binary format. The data structure is as follows:

[0034] Embed advertisement data in the extension bit of each frame of a video file (such as an MP4 file): Write complete advertisement data in keyframes; Record the ad index in the prediction frame, inheriting the key frame information.

[0035] 4. For advertisement analysis and display: When the edited video file is played on the terminal device, the decoder parses the extended bit data in real time: If the user pauses or clicks a frame, the ad identifier is parsed and the ad content is redirected.

[0036] 3. Advertisement Priority Management When multiple targets are detected in the same frame, the ad to be displayed needs to be selected based on the ad priority management logic.

[0037] 1. Priority calculation The priority of each ad is calculated using the following formula:

[0038] in: is the priority of the advertisement, is the original weight of the advertisement, is the sum of the weights of all ads.

[0039] 2. Display Restrictions After sorting by priority, up to 3 ads with the highest priority will be selected for display.

[0040] 4. Multi-scenario support 1. Real-time camera scene Users record videos in real time through their mobile phone cameras: Start dynamically detecting the target object (such as goods, brand logo) in the picture; Match advertising data from the advertising information database in real time and embed it into the extended position of each frame of video; After the video is recorded, the advertisement data is embedded into the video file.

[0041] 2. Post-editing scenes The user loads the content in the video editing software and detects and matches the target in combination with the advertising information database; The matched advertisement data is embedded into the video stream to complete the advertisement insertion.

[0042] V. Data Security and Privacy Protection 1. Advertisement data encryption Ad data is encrypted using an encryption algorithm (such as AES-256) before being embedded to prevent the data from being parsed by unauthorized devices.

[0043] 2. User privacy control Users can turn on or off the ad embedding function through settings to ensure the security of personal privacy data.

[0044] Embodiment 2:

[0045] like Figure 2 As shown, this embodiment provides a method for extracting and inserting picture or video advertisement information, including: S1: Perform image recognition on the target object in the picture or video; In real-time scenarios, the video frames or pictures of the target area are collected by the camera device, and the target area is detected and multimodal features are extracted, including image features and semantic features; S2: Match the recognition result with the preset advertisement information database. If the match is successful, extract the corresponding advertisement data, including advertisement identifier, advertisement prompt, advertisement URL and priority information; Match the extracted target features with the advertisement information database, retrieve relevant advertisement data (such as advertisement identifier, prompt, URL and priority), and after a successful match, retain the advertisement data with the highest priority; S3: In the encoding protocol of the picture or video, the advertising data is embedded through the extension bit; In the video encoding protocol (such as the udta box of an MP4 file), the advertising data is structured and written into the extension bit. The key frame is embedded with the complete advertising data, and the predicted frame inherits the key frame data to maintain inter-frame consistency. S4: The terminal device plays the image or video and parses the advertising data in the extended position, and displays the advertising prompt or jumps to the advertising content when triggered by user interaction.

[0046] When the terminal is playing, the advertising data in the extension bit is parsed. Users can trigger the advertising prompt and jump to the advertising page by pausing or clicking on the video. At the same time, the advertising data is protected by an encryption algorithm (such as AES-256) before embedding to ensure that only authorized devices can parse it. Users can choose to turn the advertising embedding function on or off.

[0047] In the post-editing scenario, the user loads the video or picture through the editing software, uses the image recognition module to detect the target of the content, and after matching the advertising information database, embeds the advertising data into the APPn segment of the JPEG file or the extension bit of the MP4 file to complete the advertising insertion.

[0048] Regardless of the scenario, priority management logic is used to ensure that no more than three single-frame ads are displayed, maximizing the user experience and ensuring the security and privacy of advertising data.

[0049] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the drawings and described above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A method for extracting and inserting picture or video advertising information, characterized in that: The following steps are involved: S1: Perform image recognition on the target object in the picture or video; the specific steps are: S1.1: Target area acquisition: using a camera to acquire images or video frames of the target area; S1.2: Target object recognition: Detect the collected images or video frames and identify the target objects contained therein, including places, people and commodities. S1.3: Target feature extraction, extracting multimodal features of the target object, including: Image features, including the shape, color, and texture of the target object; Semantic features, including the category and associated description of the target object; S2: Match the recognition result with the preset advertisement information database. If the match is successful, extract the corresponding advertisement data, including advertisement identifier, advertisement prompt, advertisement URL and priority information; S3: In the encoding protocol of the picture or video, embedding the advertisement data through the extension bit; embedding the advertisement data through the extension bit is achieved through the following steps: S3.1: Determine the extension bit in the encoding protocol of the picture or video; S3.2: Write the advertisement data into the extension bit in a structured format; S3.3: Embed corresponding advertisement data into each frame of the video to ensure inter-frame consistency; S3.4: Generate a checksum for the advertisement data in the extension bit and append it to the file, completing the advertisement embedding operation; S4: The terminal device plays the image or video and parses the advertising data in the extended position, and displays the advertising prompt or jumps to the advertising content when triggered by user interaction.

2. A method for extracting and inserting picture or video advertising information according to claim 1, characterized in that: The extension bit is a data space reserved in a file format or protocol for storing additional information. The specific steps of writing the advertisement data into the extension bit in a structured format are: structuring the advertisement data into a fixed binary format according to the extension bit determined in S3.1, and directly writing it into the extension bit; The advertisement data structure E is represented as: ； Among them, H is the advertisement identifier, D is the advertisement ID, U is the advertisement URL, the advertisement URL is the link address that jumps after the advertisement is clicked, and P is the advertisement priority. The advertisement data is encoded in the format specified by the protocol.

3. A method for extracting and inserting picture or video advertising information according to claim 1, characterized in that: The method also includes priority management of advertisement data, specifically including: In the case where a single frame contains multiple advertisement data, they are sorted according to the preset priority field; The number of advertisement data displayed in a single frame is limited to no more than 3.

4. A method for extracting and inserting picture or video advertising information according to claim 3, characterized in that: When the expansion slot is limited to storing 3 advertisement data, the advertisement data to be displayed is selected based on the priority calculation. The priority calculation formula is as follows: ； in, is the priority of the tth advertisement, is the original weight of the t-th advertisement, is the sum of all ad weights, is the original weight of the ith advertisement, and n is the total number of advertisements.

5. A method for extracting and inserting picture or video advertising information according to claim 1, characterized in that: In S3.3, corresponding advertisement data is embedded in each frame of the video. The specific steps are as follows: a. Arrange and serialize the advertising data: Encode the advertising data into binary form according to the established format and write it into the extension bit of the video frame; b. Embedding advertisement data in key frames: The key frame is the first frame of each group of pictures. The serialized advertisement data is directly written into the extended bit area of ​​the key frame. The key frame stores complete advertisement data, and the prediction frame inherits the advertisement data. c. Predicted frame inherits advertising data: The predicted frame directly references the nearest key frame. Its reference inheritance logic is to record the advertising data index or pointer pointing to the key frame in the extension bit of each predicted frame; d. Advertisement data analysis and display: The decoder extracts advertisement data from the extended bit of the current frame. When the user pauses or clicks on any frame, the advertisement data in the extended bit is called and a menu is generated for the user to select and view advertisement details. e. After the advertisement data is embedded in each group of pictures, the above process is repeated to embed the advertisement data into the key frames of the next group of pictures, and reference inheritance settings are performed on the prediction frames.

6. A method for extracting and inserting picture or video advertising information according to claim 5, characterized in that: The method supports embedding other information besides advertisement data, including: Geographic location information, used for navigation; Background information of the characters, for social media links; Product information, used for e-commerce redirection.

7. A method for extracting and inserting picture or video advertising information according to claim 6, characterized in that: When the user pauses or clicks on the video, the advertising data in the extended position is parsed and a menu containing advertising prompts is generated. The user can choose whether to view the advertising details according to their needs.

8. A method for extracting and inserting picture or video advertising information according to claim 7, characterized in that: The extraction and embedding of advertising data is applicable to the following scenarios: The camera device shoots in real time, extracts the target object through image recognition and embeds the advertising data, or embeds the advertising data in combination with the advertising information database during the post-editing process of the picture or video.

Images