Method, system and device for dynamically inserting advertisements into videos based on image recognition

Abstract

The present invention discloses a method, system, and device for dynamically inserting advertisements into videos based on image recognition, which relates to the field of video processing technology. A trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information. The video frame is segmented into a number of segmented areas, and a target area for advertisement insertion is selected within the segmented areas, and an insertion instruction is issued externally. An advertisement recommendation algorithm is used to match a corresponding target advertisement, and the target advertisement is inserted into the target area. A recommendation coefficient after the advertisement is inserted is generated from the advertisement recommendation data. If the recommendation coefficient is lower than expected, the target advertisement is re-matched and inserted. Advertisement feedback data is obtained and an improvement degree is generated within each observation cycle, and the frequency of advertisement insertion in the current stage is constrained. By making the advertisement more closely match the current live broadcast scene and the audience's interests and preferences, better feedback can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a method, system and device for dynamically inserting advertisements into videos based on image recognition. Background Art

[0002] Dynamic video ad placement is an innovative advertising format that cleverly embeds advertising content in the form of video within video content, such as movies, TV series, online programs, or user-generated videos. This placement method not only maintains the dynamic and vivid nature of the advertising message, but also, through deep integration with the video content, allows the ad to more naturally blend into the audience's viewing experience. Dynamic video ad placement can capture the audience's attention, increase ad exposure and memorability, while reducing audience resistance to the ad and maximizing advertising effectiveness.

[0003] The Chinese invention patent application, publication number CN105141987A, specifically relates to an advertising insertion method and system. These methods involve obtaining an image element marker from a dynamic video, the image element marker comprising at least one image element; determining whether the image element marker matches the advertising primitive to be inserted; and inserting the advertisement if the image element marker matches the advertising primitive to be inserted. This method allows for the insertion of advertisement entities without disrupting the viewer's viewing experience; allows viewers to select their preferred items and conduct product searches; and proactively pushes relevant advertisements based on the viewer's preferences, resulting in a more realistic insertion effect.

[0004] Combined with the above application and the contents of the prior art:

[0005] Existing methods for inserting ads into live video usually involve pausing the current live broadcast and then continuing the broadcast after the ad is finished. However, live video broadcasts, especially sports live broadcasts, require video continuity, so this method of video insertion often fails to generate effective feedback. Therefore, when feedback data such as ad exposure, completion rate, and click-through rate are poor, ads are often inserted into selected portions of the live video feed. This method of ad insertion does not affect the live broadcast and generates relatively good feedback.

[0006] However, this method of dynamically inserting advertisements into videos usually requires first extracting scene information of the current live video broadcast as well as information on audience interests and preferences, and then matching the corresponding target advertisements in a pre-prepared advertising plan library before recommending them to the audience. Although this advertising recommendation method has relatively good feedback effects, after the completion of the advertising recommendation and implantation process, due to the lack of a corresponding feedback evaluation process, the advertising recommendation and implantation cannot be adjusted in a timely and effective manner, resulting in the fact that the effect of advertising implantation may still be difficult to achieve as expected.

[0007] To this end, the present invention provides a method, system and device for dynamically inserting advertisements into videos based on image recognition. Summary of the Invention

[0008] (1) Technical problems solved

[0009] In response to the shortcomings of the existing technology, the present invention provides a method, system, and device for dynamically inserting advertisements in videos based on image recognition. The method divides the video frame into several segments, selects the target area for advertisement insertion within the segments, and issues an insertion instruction to the outside. An advertisement recommendation algorithm is used to match the corresponding target advertisement, and the target advertisement is inserted into the target area. The advertisement recommendation data generates a recommendation coefficient after the advertisement is inserted. If the recommendation coefficient is lower than expected, the target advertisement is re-matched and inserted. Advertisement feedback data is obtained and an improvement degree is generated within each observation cycle, and the frequency of advertisement insertion within the current stage is constrained. By making the advertisement more closely match the current live broadcast scene and the audience's interests and preferences, better feedback can be obtained, thereby solving the technical problems raised in the background technology.

[0010] (2) Technical solution

[0011] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0012] A method for dynamically inserting advertisements into a video based on image recognition includes collecting advertisement feedback data of a live video broadcast to generate an advertisement feedback data set, generating a feedback coefficient Fxs from the advertisement feedback data, and issuing a prompt instruction to an external device if the feedback coefficient Fxs does not exceed a feedback threshold;

[0013] The current live video is captured and key frames are extracted after preprocessing. The extracted key frames are used as input. The trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information.

[0014] Perform edge detection on video frames in live video broadcasts, and segment the video frames into several segments based on the edge detection results. Each segment is scored for importance, and target areas for advertisement insertion are selected within the segment based on the importance scores, and insertion instructions are issued externally.

[0015] Based on the video scene and the audience's characteristic information, the advertising recommendation algorithm is used to match the corresponding target advertisement, and the target advertisement is implanted in the target area. The recommendation coefficient Tjs after the implanted advertisement is generated from the advertising recommendation data. If the recommendation coefficient Tjs is lower than expected, the target advertisement is re-matched and implanted;

[0016] Advertisement feedback data is obtained in each observation period and an improvement degree Imp is generated. If the improvement degree Imp is lower than the improvement threshold, the frequency of ad placement in the current stage is restricted.

[0017] Furthermore, when advertisements are inserted into the live video, advertisement feedback data is collected in each sub-period and aggregated to form an advertisement feedback data set. The exposure rate Bv and click rate Jv are linearly normalized, and the corresponding data values ​​are mapped to the interval [0, 1] as follows:

[0018]

[0019] Weight coefficient: 0≤k1≤1, 0≤k2≤1 and k2+k1=1; Bv i is the exposure rate in the ith sub-period, Bv b is the qualified standard value of exposure rate; Jv i is the click rate in the ith sub-period, Jv b The qualified standard value for click-through rate.

[0020] Furthermore, after receiving the prompt instruction, the current live video is collected and preprocessed to obtain the preprocessed live video; the preprocessed video is used as the target video, the target video is divided into several continuous video frames, and a similarity analysis is performed on the current video frame and other video frames to obtain similarity data.

[0021] Furthermore, the similarity center Xct of each video frame is calculated from a number of similarity data, and each video frame is marked with the similarity center Xct. If the video center Xct exceeds the preset center threshold, it is used as a key frame, and the similarity center Xct is generated as follows:

[0022]

[0023] Among them, n is the number of video frames, Xs(i,j) is the similarity between the i-th video frame and the j-th video frame, Xs a is the average similarity; weight coefficient: 0≤S1≤1, 0≤S2≤1, and S1+S2=1.

[0024] Furthermore, the video frames in the live video are preprocessed, and the edge detection algorithm is applied to identify the edge information in the video frames, the detected edges are refined, unnecessary edge fragments are removed and broken edges are connected to form a complete edge contour; the region growing algorithm is applied to segment the image into multiple regions, each region contains similar pixel values ​​and texture features, and the image is segmented into foreground and background regions to obtain several segmented regions.

[0025] Furthermore, feature extraction is performed on the acquired segmented regions, and the acquired regional features are summarized; a trained segmented region evaluation model is obtained; after classifying several segmented regions, the regional features within the segmented regions are used as input, and the trained segmented region evaluation model is used to score the importance of each segmented region.

[0026] Furthermore, after receiving the implantation instruction, the current advertisement data to be implanted is obtained, including at least the keywords, scene tags, target audience information and advertisement duration that should be associated with each advertisement page, and the advertisement program database is obtained after aggregation;

[0027] Based on the current video scene and the audience's characteristic information, the trained advertising recommendation algorithm is used to match the corresponding target advertisement from the advertising plan database; after the target advertisement is implanted in the target area, the image fusion algorithm is used to integrate the advertising content with the video screen.

[0028] Furthermore, several consecutive observation periods are set, and advertising recommendation data is collected in each observation period, including: the correlation between the recommended advertisements and the video scenes, and the non-overlapping time between the recommended advertisements and the corresponding video scenes, and the advertising recommendation information is aggregated to generate an advertising recommendation data set;

[0029] The recommendation coefficient Tjs after the implanted advertisement is generated from the advertisement recommendation data set. If the recommendation coefficient Tjs is lower than the recommendation threshold, the current live advertisement recommendation algorithm is optimized. According to the optimized advertisement recommendation algorithm, the corresponding target advertisement is matched from the advertisement program database, and the target advertisement is implanted in the current live video.

[0030] Furthermore, the correlation coefficient Rt and the non-overlapping duration Ct are linearly normalized, and the corresponding data values ​​are mapped to the interval [0,1]. The recommendation coefficient Tjs is generated according to the following formula:

[0031]

[0032] Among them, α is the weight coefficient, Rt i is the correlation coefficient between the i-th recommended advertisement and the video scene, Rt p is the acceptable value of the correlation coefficient, Ct iis the non-overlapping duration between the i-th recommended advertisement and the corresponding video scene, Ct p Acceptable values ​​for non-overlapping durations.

[0033] Furthermore, after a number of consecutive observation cycles, advertisement feedback data is obtained in each observation cycle, and a feedback coefficient Fxs in the corresponding observation cycle is generated from the advertisement feedback data, and each observation cycle is marked with the feedback coefficient Fxs;

[0034] The improvement degree Imp is generated by the feedback coefficient Fxs. If the improvement degree mp is lower than the improvement threshold, an implantation frequency constraint instruction is issued to the outside.

[0035] Furthermore, the observation periods before and after the optimization of the advertising recommendation algorithm are sorted according to the time axis and corresponded one to one. The improvement degree Imp is generated by the feedback coefficient Fxs in the following way:

[0036]

[0037] Among them, No i is the median value of improvement in the i-th observation period, No a is its mean, Mo p is its qualified standard value; i=1, 2,…, k, k is the number of observation cycles, and are the feedback coefficients of the i-th observation period before and after maintenance, and is the corresponding mean.

[0038] Furthermore, after receiving the frequency restriction instruction, the frequency of advertisement insertion in the current phase is restricted. The restriction conditions are as follows:

[0039]

[0040] Among them, n is the number of nodes where advertisements are implanted, Z ij is the time interval from the i-th advertisement insertion to the j-th advertisement insertion, Z a is the average value of the time interval, Imp is the improvement;

[0041] The advertisement insertion time node is reselected based on the constrained insertion frequency and used as the insertion node, and advertisements are inserted into the live video at the selected insertion node.

[0042] A system for dynamically inserting advertisements into a video based on image recognition includes a feedback data collection unit for collecting advertisement feedback data of a live video broadcast to generate an advertisement feedback data set, generating a feedback coefficient Fxs from the advertisement feedback data, and issuing a prompt instruction to an external device if the feedback coefficient Fxs does not exceed a feedback threshold;

[0043] The video recognition unit collects the current live video and extracts key frames after pre-processing the video. The extracted key frames are used as input. The trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information.

[0044] The region segmentation unit performs edge detection on the video frames in the live video broadcast and divides the video frames into several segments based on the edge detection results. It then assigns importance scores to each segmented region and selects target regions for advertisement insertion within the segmented regions based on the importance scores and issues insertion instructions to the outside world.

[0045] The recommendation effect evaluation unit uses the advertising recommendation algorithm to match the corresponding target advertisement based on the video scene and the audience's characteristic information, and then implants the target advertisement into the target area. The recommendation coefficient Tjs after the advertisement is implanted is generated from the advertising recommendation data. If the recommendation coefficient Tjs is lower than expected, the target advertisement is re-matched and implanted.

[0046] The improvement analysis unit obtains advertising feedback data in each observation period and generates an improvement degree Imp. If the improvement degree Imp is lower than the improvement threshold, the frequency of advertising insertion in the current stage is restricted.

[0047] A device for dynamically inserting advertisements into a video based on image recognition, comprising at least one processor;

[0048] The memory is used to store executable instructions, where the instructions are executed by at least one of the processors to enable the at least one processor to perform the steps of the method for dynamically inserting advertisements in a video based on image recognition.

[0049] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the computer program performs the steps of a method for dynamically inserting advertisements in a video based on image recognition.

[0050] (3) Beneficial effects

[0051] The present invention provides a method, system, and device for dynamically inserting advertisements into videos based on image recognition, which has the following beneficial effects:

[0052] 1. Evaluate the current advertising effect based on the feedback coefficient Fxs to verify whether the current advertising playback method can achieve the expected effect. If it is difficult to achieve the expected effect, the current advertising playback method can be adjusted in time to improve the advertising playback effect.

[0053] 2. Evaluate the differences between each video frame based on the similarity center Xct, screen out the parts with greater differences from several video frames, and extract key frames. By extracting key frames, the recognition effect can be better when performing scene recognition; perform scene recognition on the extracted key frames. After identifying and obtaining the corresponding scene, more targeted advertising information can be matched based on the recognition data, and better feedback data can also be obtained.

[0054] 3. Use the trained segmentation region evaluation model to score the importance of each segmentation region, and select the area within the video frame where advertising can be inserted. Inserting advertising will not affect the continuation of the live video broadcast, nor does it need to pause the current live video broadcast, and better feedback can be obtained when inserting advertising.

[0055] 4. By collecting data in advance and building an advertising plan database, we can match corresponding embedded advertisements to the audience's interests and preferences based on the recommendation algorithm. By making the advertisements more closely matched with the current live broadcast scene and the audience's interests and preferences, we can obtain better feedback.

[0056] 5. Generate a recommendation coefficient Tjs from the acquired data, and make a comprehensive evaluation of the targeting of the currently matched embedded advertisements based on the recommendation coefficient Tjs to determine whether the currently recommended advertisements are appropriate. If the degree of suitability is low, re-determine a new advertisement recommendation strategy to improve the effect of advertisement embedding.

[0057] 6. Generate an improvement degree MP from the feedback data obtained again, and use the improvement degree MP to evaluate the current advertising implantation effect; by quantitatively evaluating the advertising implantation effect, it is possible to confirm whether the current advertising implantation strategy is feasible. If not, timely adjustments and corrections can be made to ensure that the feedback effect of the advertising implantation in the dynamic video live broadcast can meet the expectations.

[0058] 7. Restrict and adjust the current frequency of advertising insertion, reselect the time node for advertising insertion, reduce the audience's aversion to advertising insertion, and provide a certain guarantee for the effect of advertising insertion in dynamic video live broadcasts. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of a method for dynamically inserting advertisements into videos based on image recognition according to the present invention;

[0060] Figure 2 This is a schematic diagram of the system structure for dynamically inserting advertisements into videos based on image recognition according to the present invention;

[0061] Figure 3 This is a schematic diagram of the structure of a device for dynamically inserting advertisements into videos based on image recognition according to the present invention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0063] See also Figure 1 The present invention provides a method for dynamically inserting advertisements into a video based on image recognition, comprising:

[0064] Step 1: After collecting advertising feedback data of the live video, an advertising feedback data set is generated, and a feedback coefficient Fxs is generated from the advertising feedback data. If the feedback coefficient Fxs does not exceed the feedback threshold, a prompt instruction is issued to the outside;

[0065] The step 1 includes the following:

[0066] Step 101: During a live video broadcast, an observation period consisting of several sub-periods is set. When advertisements are inserted into the live video broadcast, advertisement feedback data of the live video broadcast is collected in each sub-period, including at least exposure rate, click-through rate, etc. The advertisement feedback data is aggregated to form an advertisement feedback data set.

[0067] Step 102: Construct a feedback coefficient Fxs after the advertisement is inserted based on the feedback data in the advertisement feedback data set. The exposure rate Bv and click rate Jv are linearly normalized, and the corresponding data values ​​are mapped to the interval [0, 1] according to the following method:

[0068]

[0069] Weight coefficient: 0≤k1≤1, 0≤k2≤1 and k2+k1=1; the weight coefficient can be obtained by referring to the hierarchical analysis method, Bv i is the exposure rate in the ith sub-period, Bv b is the qualified standard value of exposure rate; Jv i is the click rate in the ith sub-period, Jv b is the qualified standard value of click rate;

[0070] Based on historical data and management expectations of the advertising placement effect, a feedback threshold is pre-set; if the feedback coefficient Fxs does not exceed the feedback threshold, it means that the current advertising placement effect is poor and needs to be readjusted. At this time, a prompt instruction is issued to the outside world;

[0071] When using, combine the contents in steps 101 and 102:

[0072] When advertisements are inserted or played during the current live video broadcast, the current advertisement feedback data is collected, and a corresponding feedback coefficient Fxs is constructed based on the feedback data. The current advertisement playback effect is evaluated based on the feedback coefficient Fxs to verify whether the current advertisement playback method can achieve the expected effect. If it is difficult to achieve the expected effect, the current advertisement playback method can be adjusted in a timely manner to improve the advertisement playback effect.

[0073] The method of dynamically inserting advertisements into videos usually requires first extracting the scene information of the current live video and the audience's interests and preferences, then matching the corresponding target advertisements in a pre-prepared advertising plan library, and then recommending them to the audience. Although this advertising recommendation method has relatively good feedback effects, after the completion of the advertising recommendation and insertion process, due to the lack of a corresponding feedback evaluation process, the advertising recommendation and insertion cannot be adjusted in a timely and effective manner, resulting in the advertising insertion effect may still be difficult to achieve the expected results.

[0074] Step 2: Capture the current live video and extract key frames after preprocessing the video. Using the extracted key frames as input, the trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information.

[0075] The second step includes the following:

[0076] Step 201: After receiving the prompt instruction, the current live video is captured and preprocessed, including noise removal and color correction, to improve the accuracy of image recognition, and the preprocessed live video is obtained; the preprocessed video is used as the target video, the target video is segmented into several consecutive video frames, and a similarity analysis is performed between the current video frame and other video frames to obtain similarity data;

[0077] Step 202: Calculate the similarity center Xct of each video frame from a number of similarity data in the following manner:

[0078]

[0079] Among them, n is the number of video frames, Xs(i,j) is the similarity between the i-th video frame and the j-th video frame, Xs a is the average similarity; weight coefficient: 0≤S1≤1, 0≤S2≤1, and S1+S2=1;

[0080] Each video frame is marked with similarity center Xct. If the video center Xct exceeds the preset center threshold, it is used as a key frame;

[0081] When in use, after analyzing and obtaining several video frames, the similarity center Xct of each video frame is obtained based on the similarity analysis between different video frames. The difference of each video frame is evaluated based on the similarity center Xct. The parts with greater differences can be screened out from several video frames to realize the extraction of key frames. By extracting key frames, the recognition effect can be better when performing scene recognition;

[0082] Step 203: After training a recurrent neural network with the sample data, a scene recognition model is obtained. The extracted key frames are used as input, and the trained scene recognition model recognizes the scene type, objects, and people in the current live video, and outputs the recognized video scene and related information.

[0083] When using, combine the contents in steps 201 to 203:

[0084] After completing the extraction of key frames, the trained scene recognition model is used to perform scene recognition on the extracted key frames. After identifying and obtaining the corresponding scenes, more targeted advertising information can be matched based on the recognition data, and better feedback data can also be obtained.

[0085] Step 3: Perform edge detection on the video frame in the live video broadcast, and divide the video frame into several segments based on the edge detection results. Each segment is scored for importance, and target areas for advertisement insertion are selected within the segment based on the importance score, and an insertion instruction is issued externally.

[0086] The step three includes the following:

[0087] Step 301: Pre-process the video frames in the live video broadcast and apply an edge detection algorithm to identify edge information in the video frames. Then, refine the detected edges, remove unnecessary edge segments, and connect broken edges to form a complete edge contour.

[0088] Applying the region growing algorithm to segment the image into multiple regions, each region contains similar pixel values ​​and texture features, and segmenting the image into foreground and background regions to obtain several segmented regions;

[0089] Step 302: extract features from the acquired segmented regions, including features such as area, shape, and texture, and summarize the acquired regional features; train a convolutional neural network using the sample data to obtain a trained segmented region evaluation model;

[0090] After classifying several segmented regions, the trained segmented region evaluation model uses the regional features within the segmented regions as input to assign importance scores to each segmented region. The importance of each region is evaluated and the target region for ad placement is selected within the segmented region based on the importance scores. This can avoid selecting regions containing important content, such as people and key objects, and issuing external placement instructions.

[0091] When using, combine the contents in steps 301 and 302:

[0092] After edge detection and classification of the video frame, the video frame is segmented into several regions, and the importance of each segmented region is scored using the trained segmented region evaluation model. This allows the selection of regions within the video frame where advertisements can be inserted. After the advertisements are inserted, the inserted advertisements will not affect the continuation of the live video broadcast, nor will the current live video broadcast need to be paused, allowing for better feedback when inserting advertisements.

[0093] Step 4: Based on the video scene and the audience's characteristic information, the advertising recommendation algorithm is used to match the corresponding target advertisement, and the target advertisement is implanted in the target area. The recommendation coefficient Tjs after the implanted advertisement is generated from the advertising recommendation data. If the recommendation coefficient Tjs is lower than expected, the target advertisement is re-matched and implanted;

[0094] The steps include the following:

[0095] Step 401: After receiving the implantation instruction, the current advertisement data to be implanted is obtained, including at least the keywords, scene tags, target audience information, and advertisement duration associated with each advertisement page, and the advertisement program database is obtained after aggregation;

[0096] During live video broadcasts, the trained ad recommendation algorithm is used to match target ads from the ad solution database based on the current video scene and viewer characteristics, such as audience interests and preferences.

[0097] After the target advertisement is implanted in the target area, the image fusion algorithm allows the advertisement content to be seamlessly integrated with the video screen.

[0098] When using it, after selecting the target area for ad placement, data collection and building an ad solution database are performed in advance. During the live video broadcast, the recommendation algorithm can be used to match the corresponding ad placement to the audience's interests and preferences. By making the ads more closely match the current live broadcast scene and the audience's interests and preferences, better feedback can be obtained.

[0099] Step 402: Set a number of consecutive observation periods and collect advertisement recommendation data in each observation period, including the correlation between the recommended advertisements and the video scenes, and the non-overlapping duration between the recommended advertisements and the corresponding video scenes. Aggregate the above advertisement recommendation information to generate an advertisement recommendation data set.

[0100] Step 403: Generate a recommendation coefficient Tjs after the advertisement is embedded from the advertisement recommendation data set. The correlation coefficient Rt and the non-overlap duration Ct are linearly normalized, and the corresponding data values ​​are mapped to the interval [0, 1] according to the following formula:

[0101]

[0102] Among them, α is the weight coefficient, and its value falls within [0.05, 0.95]. It can be obtained through simulation software analysis or reference to the hierarchical analysis method; Rt i is the correlation coefficient between the i-th recommended advertisement and the video scene, Rt p is the acceptable value of the correlation coefficient, Ct i is the non-overlapping duration between the i-th recommended advertisement and the corresponding video scene, Ct p is the acceptable value for non-overlapping duration;

[0103] Based on historical data and the expected effect of embedded advertising recommendations, a recommendation threshold is pre-set; if the recommendation coefficient Tjs is lower than the recommendation threshold, it means that the current recommendation effect has not met expectations, and the current live broadcast advertising recommendation algorithm should be optimized;

[0104] According to the optimized advertising recommendation algorithm, the corresponding target advertisement is matched from the advertising plan database and the target advertisement is embedded in the current live video;

[0105] When using, combine the contents in steps 401 to 403:

[0106] After completing the advertisement recommendation, the advertisement recommendation and its related data are collected, and the recommendation coefficient Tjs is generated from the acquired data. Based on the recommendation coefficient Tjs, a comprehensive evaluation is made on the pertinence of the currently matched implanted advertisement to determine whether the currently recommended advertisement is appropriate. If the degree of suitability is low, a new advertisement recommendation strategy is re-determined to improve the effect of advertisement implantation.

[0107] Step 5: Obtain advertising feedback data in each observation period and generate an improvement degree Imp. If the improvement degree Imp is lower than the improvement threshold, restrict the frequency of advertising insertion in the current stage.

[0108] The step five includes the following:

[0109] Step 501: After several consecutive observation cycles, obtain advertising feedback data in each observation cycle, generate a feedback coefficient Fxs in the corresponding observation cycle from the advertising feedback data, and mark each observation cycle with the feedback coefficient Fxs;

[0110] The observation periods before and after the optimization of the ad recommendation algorithm are sorted according to the time axis and mapped one to another. The improvement degree Imp is generated by the feedback coefficient Fxs, and the current ad placement effect is evaluated by the improvement degree Imp as follows:

[0111]

[0112] Among them, No i is the median value of improvement in the i-th observation period, No a is its mean, No p is its qualified standard value; i=1, 2,…, k, k is the number of observation cycles, and are the feedback coefficients of the i-th observation period before and after maintenance, and is the corresponding mean.

[0113] Based on historical data and the expected effect of advertising recommendations, an improvement threshold is pre-set. If the improvement degree mp is lower than the improvement threshold, it means that after re-implanting the advertisement, the insertion effect of the advertisement still fails to achieve the expected effect. In this case, the frequency of advertisement insertion is adjusted and an insertion frequency constraint instruction is issued externally.

[0114] During use, when advertising is implanted in live video, the feedback data obtained again is used to generate an improvement degree mp compared to the original advertising implantation method, and the current advertising implantation effect is evaluated with the improvement degree mp; by quantitatively evaluating the advertising implantation effect, it is possible to confirm whether the current advertising implantation strategy is feasible. If not, adjustments and corrections can be made in a timely manner to ensure that the feedback effect of implanting advertising in dynamic live video can meet expectations.

[0115] Step 502: After receiving the frequency restriction instruction, restrict the frequency of advertisement insertion in the current phase. The restriction conditions are as follows:

[0116]

[0117] Among them, n is the number of nodes where advertisements are implanted, Z ij is the time interval from the i-th advertisement insertion to the j-th advertisement insertion, Z a is the average value of the time interval, Imp is the improvement;

[0118] Reselecting an advertisement insertion time node based on the constrained insertion frequency and using it as the insertion node, and inserting advertisements into the live video at the selected insertion node;

[0119] When using, combine the contents in steps 501 and 502:

[0120] When the current advertising insertion effect is poor, the current advertising insertion frequency is restricted and adjusted, and the time node for advertising insertion is reselected to reduce the audience's aversion to advertising insertion, thereby forming a certain guarantee for the effect of advertising insertion in dynamic video live broadcast.

[0121] The Analytic Hierarchy Process (AHP) is a decision-making method that breaks down decision-related elements into a hierarchy of objectives, criteria, and options, and then conducts qualitative and quantitative analysis based on this hierarchy. It is particularly well-suited for target systems with hierarchical and interleaved evaluation indicators, and is an effective decision-making tool when target values ​​are difficult to quantify.

[0122] The core of the AHP lies in breaking down the decision problem into multiple levels, forming a hierarchical structure. This structure typically includes the objective level, the criteria level, the sub-criteria level, and the solution level. By solving the eigenvectors of the judgment matrix, the priority weight of each element at each level relative to the element at the previous level is determined. Finally, a weighted sum method is used to recursively combine the final weights of each alternative solution relative to the overall goal, thereby identifying the optimal solution.

[0123] See also Figure 2 The present invention provides a system for dynamically inserting advertisements into videos based on image recognition, comprising:

[0124] A feedback data collection unit collects advertising feedback data from live video broadcasts and generates an advertising feedback data set. The feedback coefficient Fxs is generated from the advertising feedback data. If the feedback coefficient Fxs does not exceed a feedback threshold, a prompt instruction is issued to the outside.

[0125] The video recognition unit collects the current live video and extracts key frames after pre-processing the video. The extracted key frames are used as input. The trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information.

[0126] The region segmentation unit performs edge detection on the video frames in the live video broadcast and divides the video frames into several segments based on the edge detection results. It then assigns importance scores to each segmented region and selects target regions for advertisement insertion within the segmented regions based on the importance scores and issues insertion instructions to the outside world.

[0127] The recommendation effect evaluation unit uses the advertising recommendation algorithm to match the corresponding target advertisement based on the video scene and the audience's characteristic information, and then implants the target advertisement into the target area. The recommendation coefficient Tjs after the advertisement is implanted is generated from the advertising recommendation data. If the recommendation coefficient Tjs is lower than expected, the target advertisement is re-matched and implanted.

[0128] The improvement analysis unit obtains advertising feedback data in each observation period and generates an improvement degree Imp. If the improvement degree Imp is lower than the improvement threshold, the frequency of advertising insertion in the current stage is restricted.

[0129] See also Figure 3 Based on the same inventive concept, according to another aspect of the present invention, the present invention also provides a device for dynamically inserting advertisements in videos based on image recognition, comprising at least one processor; a memory for storing executable instructions, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method for dynamically inserting advertisements in videos based on image recognition.

[0130] See also Figure 3 Based on the same inventive concept, according to another aspect of the present invention, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it performs the steps of the method for dynamically inserting advertisements in videos based on image recognition.

[0131] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0132] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only for some logical functions. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0138] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for dynamically inserting advertisements into a video based on image recognition, characterized by: include, After collecting the advertising feedback data of the live video, an advertising feedback data set is generated, and the feedback coefficient is generated from the advertising feedback data. , if the feedback coefficient Does not exceed the feedback threshold and issues prompt instructions to the outside world; The current live video is captured and key frames are extracted after preprocessing. The extracted key frames are used as input. The trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information. Perform edge detection on video frames in live video broadcasts, and segment the video frames into several segments based on the edge detection results. Each segment is scored for importance, and target areas for advertisement insertion are selected within the segment based on the importance scores, and insertion instructions are issued externally. According to the video scene and the audience's characteristic information, the advertising recommendation algorithm is used to match the corresponding target advertisement, and the target advertisement is implanted into the target area. The recommendation coefficient after the implantation of the advertisement is generated by the advertising recommendation data. , if the recommended coefficient Lower than expected, re-match the target ads and insert them; among them, the correlation coefficient and non-overlapping duration Perform linear normalization and map the corresponding data values ​​to the interval The recommendation coefficient is generated according to the following formula : ； in, is the weight coefficient, For the i The correlation coefficient between the recommended ads and the video scenes, is the acceptable value of the correlation coefficient, For the i The non-overlapping duration of the recommended ads and the corresponding video scenes, is the acceptable value for non-overlapping duration; Obtain advertising feedback data and generate improvement data during each observation period If the improvement If the improvement threshold is lower than the threshold, the frequency of advertisement insertion in the current stage is constrained, and the observation periods before and after the optimization of the advertisement recommendation algorithm are sorted according to the time axis and corresponded one to one. Generate improvement , as follows: ； in, For the i The median improvement over the observation period, is its mean, is its qualified standard value; , is the number of observation periods, and Before and after maintenance i The feedback coefficient of the observation period, and is the corresponding mean; After receiving the frequency restriction instruction, the frequency of advertisement insertion in the current phase is restricted. The restriction conditions are as follows: ； in, n is the number of nodes where advertisements are implanted. It is i Ads are placed in j The time interval between advertisement insertions, is the average value of the time interval, To improve the degree; The advertisement insertion time node is reselected based on the constrained insertion frequency and used as the insertion node, and advertisements are inserted into the live video at the selected insertion node.

2. The method for dynamically inserting advertisements into videos based on image recognition according to claim 1, wherein: When advertisements are inserted into live video, advertisement feedback data is collected in each sub-period and aggregated to form an advertisement feedback data set. and click-through rate Perform linear normalization and map the corresponding data values ​​to the interval In the following way: ； Weight coefficient: , and ; For the i Exposure rate within a sub-period, is the qualified standard value of exposure rate; For the i Click-through rate within a sub-period, The qualified standard value for click-through rate.

3. The method for dynamically inserting advertisements into videos based on image recognition according to claim 2, wherein: After receiving the prompt instruction, the current live video is collected and pre-processed to obtain the pre-processed live video; The preprocessed video is used as the target video, and the target video is divided into several consecutive video frames. The similarity analysis is performed on the current video frame and other video frames to obtain similarity data.

4. The method for dynamically inserting advertisements into videos based on image recognition according to claim 3, wherein: The similarity center of each video frame is obtained by calculating several similarity data , with similarity centrality Mark each video frame. If the video center If the center value exceeds the preset threshold, it will be used as a key frame; similarity center values ​​will be generated as follows: : ； in, n is the number of video frames, It is i Video frame and j The similarity of video frames, is the average similarity; weight coefficient: , ,and .

5. The method for dynamically inserting advertisements into videos based on image recognition according to claim 4, characterized in that: The video frames in the live video are preprocessed, and the edge detection algorithm is applied to identify the edge information in the video frames. The detected edges are refined, unnecessary edge fragments are removed and broken edges are connected to form a complete edge contour; the region growing algorithm is applied to segment the image into multiple regions, each region contains similar pixel values ​​and texture features, and the image is segmented into foreground and background regions to obtain several segmented regions.

6. The method for dynamically inserting advertisements into videos based on image recognition according to claim 5, characterized in that: The obtained segmented regions are subjected to feature extraction, and the obtained regional features are summarized. After several segmented regions are classified, the regional features within the segmented regions are used as input, and the trained segmented region evaluation model is used to score the importance of each segmented region.

7. The method for dynamically inserting advertisements into a video based on image recognition according to claim 6, wherein: After receiving the implantation instruction, the current advertisement data to be implanted is obtained, including at least the relevant keywords, scene tags, target audience information and advertisement duration of each advertisement page, and the advertisement program database is obtained after aggregation; Based on the current video scene and the audience's characteristic information, the trained ad recommendation algorithm is used to match the corresponding target ads from the ad solution database; After the target advertisement is implanted in the target area, the image fusion algorithm is used to merge the advertisement content with the video image.

8. The method for dynamically inserting advertisements into videos based on image recognition according to claim 7, wherein: Set up several consecutive observation cycles and collect advertising recommendation data in each observation cycle, including: the correlation between the recommended ads and the video scenes, and the non-overlapping time between the recommended ads and the corresponding video scenes. Aggregate the advertising recommendation information to generate an advertising recommendation data set; Generate the recommendation coefficient after inserting advertisements from the advertisement recommendation data set , if the recommended coefficient If the value is lower than the recommendation threshold, the current live broadcast advertisement recommendation algorithm is optimized; the corresponding target advertisement is matched from the advertisement program database based on the optimized advertisement recommendation algorithm, and the target advertisement is embedded in the current video live broadcast.

9. The method for dynamically inserting advertisements into videos based on image recognition according to claim 8, characterized in that: After several consecutive observation cycles, advertising feedback data is obtained in each observation cycle, and the feedback coefficient in the corresponding observation cycle is generated from the advertising feedback data. , with the feedback coefficient Mark each observation period; By the feedback coefficient Generate improvement If the improvement If the frequency is lower than the improvement threshold, an implant frequency constraint instruction is issued to the outside.

10. A system for dynamically inserting advertisements into videos based on image recognition, applying the method of claim 9, characterized in that: include, Feedback data collection unit, collects advertising feedback data of live video and generates advertising feedback data set, and generates feedback coefficient from advertising feedback data , if the feedback coefficient Does not exceed the feedback threshold and issues prompt instructions to the outside world; The video recognition unit collects the current live video and extracts key frames after pre-processing the video. The extracted key frames are used as input. The trained scene recognition model identifies the scene type in the current live video and outputs the identified video scene and related information. The region segmentation unit performs edge detection on the video frames in the live video broadcast and divides the video frames into several segments based on the edge detection results. It then assigns importance scores to each segmented region and selects target regions for advertisement insertion within the segmented regions based on the importance scores and issues insertion instructions to the outside world. The recommendation effect evaluation unit uses the advertising recommendation algorithm to match the corresponding target advertisement based on the video scene and the audience's characteristic information, and then implants the target advertisement into the target area. The recommendation coefficient after the advertisement is implanted is generated from the advertising recommendation data. , if the recommended coefficient If the result is lower than expected, re-target the advertisement and re-implement it; Improve the analysis unit to obtain advertising feedback data and generate improvement scores during each observation period If the improvement If the value is lower than the improvement threshold, the frequency of advertisement insertion in the current stage will be restricted.

11. A device for dynamically inserting advertisements into videos based on image recognition, characterized in that: include, at least one processor; A memory for storing executable instructions, wherein the instructions are executed by at least one of the processors to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing a computer program, characterized in that: The computer program is used by a processor to execute the steps of the method according to any one of claims 1 to 9.

Images