Advertisement replacement method and device, electronic equipment and readable storage medium

Abstract

This invention provides an advertisement replacement method, apparatus, electronic device, and readable storage medium, comprising: generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes from a trademark advertisement image to be replaced based on a target segmentation network and a target recognition network; dividing the first-class trademark advertisement segmentation image into a third-class trademark advertisement segmentation image with rectangular bounding boxes and a fourth-class trademark advertisement segmentation image with non-rectangular bounding boxes based on a first target algorithm; performing an affine transformation on the foreground of the target trademark advertisement image to generate the first trademark advertisement image; performing an affine transformation on the portion within the rectangular bounding boxes of the target trademark advertisement image and the third-class trademark advertisement segmentation image to generate the second trademark advertisement image; and performing an affine transformation on the portion within the minimum bounding rectangle of the non-rectangular bounding boxes of the target trademark advertisement image and the fourth-class trademark advertisement segmentation image to generate the third trademark advertisement image, thereby improving the advertisement replacement effect.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and in particular relates to an advertising replacement method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] In the information explosion era, advertising bears the responsibility of brand communication and serves as a bridge between brands and consumers. Our lives are filled with ubiquitous advertising: LED screens on buildings, bus stops, elevators, stairwells, billboards, flyers, posters, and embedded ads on major television stations and video platforms. This proliferation of advertising methods has led to a year-on-year increase in ineffective traffic. In this context, precise analysis of user needs based on regional information and browsing history, along with the technical replacement of ads appearing in high-viewer videos with relevant content, and then targeting the desired audience, can significantly improve marketing effectiveness and save on advertising costs. Furthermore, using technology to replace ads can reduce production costs; for live videos, replacement can be done in real time, and for pre-recorded videos, only video processing is required.

[0003] Existing ad replacement technologies generally replace the ad frames of interstitial ads, or replace the billboards with rectangular bounding boxes in video frames using deep learning technologies such as object detection and instance segmentation. However, these technologies do not cover complex scenarios where non-rectangular trademark ads are embedded. For example, in many live variety shows, text trademarks or non-rectangular trademark ads appear in the background, resulting in relatively simple ad replacement results and low replacement efficiency. Summary of the Invention

[0004] The present invention provides an ad replacement method, apparatus, electronic device, and readable storage medium to solve the problems of relatively simple ad replacement results and low replacement efficiency in the prior art.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows: In a first aspect, the present invention provides an advertisement replacement method, the method comprising: The target segmentation network and the target recognition network are used to label the advertisement image of the trademark to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes; According to the first objective algorithm, the first type of trademark advertisement segmentation map is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box. The foreground of the target trademark advertisement image is transformed into the second-class trademark advertisement segmentation image through an affine transformation to generate the first trademark advertisement image; A second trademark advertisement image is generated by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement. Affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth class trademark advertisement to generate a third trademark advertisement image.

[0006] Optionally, before labeling the advertisement image of the trademark to be replaced according to the target segmentation network and the target recognition network respectively, and generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes, the method further includes: Obtain the target video frame according to preset rules; Video keyframes are obtained from the target video frame, and each frame of the video keyframe includes the trademark advertisement image to be replaced; The trademark advertisement image to be replaced is obtained from the keyframes of the video using an object detection network.

[0007] Optionally, obtaining the trademark advertisement image to be replaced from the video keyframes using the target detection network includes: The minimum bounding rectangle of the trademark advertisement image to be replaced is generated according to the first objective algorithm; The target detection network generates positioning information about the minimum bounding rectangle. The trademark advertisement image to be replaced is obtained from the video keyframes based on the location information.

[0008] Optionally, the step of labeling the advertisement image of the trademark to be replaced according to the target segmentation network and the target recognition network respectively, and generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes includes: Obtain the set of pixels of the advertisement image of the trademark to be replaced; The foreground marker, background marker, and non-trademark marker of the trademark advertisement image to be replaced are generated based on the target segmentation network and the pixel set. A first marker and a second marker are generated based on the target recognition network and the pixel set. The first marker is used to mark the trademark advertisement image to be replaced with a bounding box, and the second marker is used to mark the trademark advertisement image to be replaced without a bounding box. Based on the foreground marker, the background marker, the non-replaceable trademark advertisement image marker, the first marker, and the second marker, a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box are determined.

[0009] Optionally, dividing the first type of trademark advertisement segmentation map into a third type of trademark advertisement segmentation map with rectangular bounding boxes and a fourth type of trademark advertisement segmentation map with non-rectangular bounding boxes according to the first target algorithm includes: A first target algorithm is obtained, which includes an edge extraction algorithm, a binarization algorithm, and a Hough detection algorithm. An edge intensity map of the first type of trademark advertisement segmentation map is generated according to the edge extraction algorithm; The pixels of the edge intensity map are binarized according to the binarization algorithm. According to the Hough detection algorithm, the first type of trademark advertisement segmentation map after binarization is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box.

[0010] Optionally, the step of generating the first trademark advertisement image by performing an affine transformation on the foreground of the target trademark advertisement image in the second class trademark advertisement segmentation image includes: Obtain the pixels of the foreground portion of the segmented image of the Class 2 trademark advertisement; After removing the pixels of the foreground portion, the image is filled using an image filling network to generate a second type of trademark advertising background filling image; Obtain the first coordinate information of the four vertices of the target trademark advertisement image and the trademark advertisement image to be replaced; Generate a first affine transformation matrix based on the first coordinate information; Based on the first affine transformation matrix, the foreground of the target trademark advertisement image is affinely transformed in the background-filled image of the second type of trademark advertisement to generate the first trademark advertisement image.

[0011] Optionally, the step of performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement to generate a second trademark advertisement image includes: Obtain the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the segmentation image of the third-class trademark advertisement; Generate a second affine transformation matrix based on the second coordinate information; The second trademark advertisement image is generated by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the third-class trademark advertisement segmentation image based on the second affine transformation matrix.

[0012] Optionally, before obtaining the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the third-class trademark advertisement segmentation image, the method further includes: Obtain the pixel marker values ​​of the segmentation image of the Class 3 trademark advertisement; The third type of trademark advertisement is segmented according to the pixel marker value. Figure 2 Value-based processing generates a first-order binary image; The first binary image is processed according to the second objective algorithm to obtain the portion within the rectangular bounding box in the third type of trademark advertisement segmentation image.

[0013] Optionally, the step of generating a third trademark advertisement image by performing an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth class trademark advertisement includes: Obtain the pixel marker values ​​of the advertisement segmentation image for Class 4 trademarks; The fourth type of trademark advertisement is segmented according to the pixel marker value. Figure 2 Value-based processing generates a second binary image; The minimum bounding rectangle of the non-rectangular bounding box in the fourth type of trademark advertisement segmentation diagram is generated according to the third objective algorithm; Obtain the third coordinate information of the four vertices of the target trademark advertisement image and the minimum bounding rectangle; Generate a third affine transformation matrix based on the third coordinate information; The third trademark advertisement image is generated by performing an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement based on the third affine transformation matrix.

[0014] In a second aspect, the present invention provides an advertisement replacement device, the device comprising: The first generation module is used to mark the trademark advertisement image to be replaced according to the target segmentation network and the target recognition network, and generate a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes. The segmentation module is used to divide the first type of trademark advertisement segmentation image into a third type of trademark advertisement segmentation image with rectangular bounding boxes and a fourth type of trademark advertisement segmentation image with non-rectangular bounding boxes according to the first target algorithm. The second generation module is used to perform an affine transformation on the foreground of the target trademark advertisement image in the second class of trademark advertisement segmentation image to generate the first trademark advertisement image; The third generation module is used to perform an affine transformation on the target trademark advertisement image and the portion within the rectangular bounding box of the third class trademark advertisement segmentation image to generate a second trademark advertisement image. The fourth generation module is used to perform an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement to generate a third trademark advertisement image.

[0015] Thirdly, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described advertising replacement method.

[0016] Fourthly, the present invention provides a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described advertisement replacement method.

[0017] In this embodiment of the invention, the target segmentation network and the target recognition network respectively label the trademark advertisement image to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes. A deep learning network is used to label the trademark advertisement image to be replaced, facilitating subsequent differentiation and processing of the trademark advertisement image to be replaced. According to the first target algorithm, the first-class trademark advertisement segmentation image is divided into a third-class trademark advertisement segmentation image with rectangular bounding boxes and a fourth-class trademark advertisement segmentation image with non-rectangular bounding boxes. The foreground of the target trademark advertisement image is subjected to an affine transformation in the second-class trademark advertisement segmentation image to generate the first trademark advertisement image. This can achieve the processing of borderless trademark advertisement images. The replacement method involves performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement to generate a second trademark advertisement image. Similarly, an affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmented image of the fourth-class trademark advertisement to generate a third trademark advertisement image. This method can replace trademark advertisements with borders and allows for further refinement based on different borders, resulting in a better trademark replacement effect. In summary, this invention can replace not only trademark advertisements with rectangular bounding boxes but also those without borders or with non-rectangular bounding boxes, avoiding the problem of relatively simple and inefficient trademark advertisement replacement results and improving the overall replacement effect. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of an advertisement replacement method provided in an embodiment of the present invention; Figure 2 This is the second step in the flowchart of an advertisement replacement method provided in an embodiment of the present invention; Figure 3This is the third step in the flowchart of an advertisement replacement method provided in an embodiment of the present invention; Figure 4 This is a structural diagram of an advertising replacement device provided in an embodiment of the present invention; Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Figure 1 This is one of the flowcharts of an advertisement replacement method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method may include: Step 101: Mark the trademark advertisement image to be replaced according to the target segmentation network and the target recognition network respectively, and generate a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes.

[0022] In this embodiment of the invention, the cut-out trademark advertisement image to be replaced is fed into a target segmentation network and a target recognition network, respectively. In the target segmentation network, the pixels in the trademark advertisement image to be replaced are marked, and the foreground, background and non-trademark advertisement parts in the image are divided. In the target recognition network, the trademark advertisement image to be replaced is marked and divided according to whether it has a bounding box or not. Both of these division methods are generated by marking on the trademark advertisement image to be replaced. Therefore, through the target segmentation network and the target recognition network, a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box can be obtained.

[0023] It should be noted that the target segmentation network in this embodiment of the invention is a deep learning network. Multiple sample images are collected, and the pixel datasets of these images are input into the learning network. Corresponding hierarchical labels are output, thereby training the deep learning network. Typically, text and meaningful patterns in the advertisement are labeled as foreground, solid colors as background, and areas significantly different from the background as non-trademark advertisement parts. For example, the deep learning target segmentation network labels the pixels on the advertisement image containing the trademark to be replaced as three categories: the foreground is labeled as 2, the background as 1, and the non-trademark advertisement parts as 0. Furthermore, various deep learning networks can be used, such as the Unet network; this invention does not specifically limit the types used.

[0024] It should be noted that the target recognition network in this embodiment of the invention is also a deep learning network. Multiple sample images are collected, and the pixel dataset of these images is input into the learning network. Different classification labels are output based on whether or not the image has a bounding box, thereby training the deep learning network. For example, the target recognition network performs binary classification on the trademark advertisement image to be replaced. Images with bounding boxes are labeled with a first identifier, while those without are labeled with a second identifier. Based on these identifiers, the trademark advertisement images to be replaced can be distinguished.

[0025] Therefore, in this embodiment of the invention, by inputting the pixel set of the trademark advertisement image to be replaced into a target segmentation network and a target recognition network, the trademark advertisement image to be replaced is distinguished according to whether it has a bounding box or not. Simultaneously, the interior of the trademark advertisement image to be replaced is layered to identify the foreground, background, and non-trademark advertisement image. The two recognition results are combined to obtain a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box. Specific implementation steps include: Obtain the set of pixels of the advertisement image of the trademark to be replaced; The foreground marker, background marker, and non-trademark marker of the trademark advertisement image to be replaced are generated based on the target segmentation network and the pixel set. A first marker and a second marker are generated based on the target recognition network and the pixel set. The first marker is used to mark the trademark advertisement image to be replaced with a bounding box, and the second marker is used to mark the trademark advertisement image to be replaced without a bounding box. Based on the foreground marker, the background marker, the non-replaceable trademark advertisement image marker, the first marker, and the second marker, a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box are determined.

[0026] Step 102: According to the first target algorithm, the first type of trademark advertisement segmentation map is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box.

[0027] In this embodiment of the invention, trademark advertisements are divided into those with rectangular borders, those with non-rectangular borders, and those without borders, and then replaced separately. Therefore, after dividing the trademark advertisement images to be replaced into those with borders and those without, the trademark advertisement images with borders are further divided into those with rectangular borders and those with non-rectangular borders.

[0028] To determine the shape of the bounding box in the segmentation image of the first type of trademark advertisement, an edge intensity map is first obtained by using an edge extraction algorithm. Then, the edge intensity map is binarized to obtain a binary image. For example, the pixel values ​​of non-edge regions in the edge intensity map are set to 0, and the pixel values ​​of edge regions are set to 1. The Hough detection algorithm is then used to perform rectangle detection on the binary image to identify whether the bounding box in the segmentation image of the first type of trademark advertisement is rectangular, and then segmentation is performed to generate a third type of trademark advertisement segmentation image with rectangular bounding boxes and a fourth type of trademark advertisement segmentation image with non-rectangular bounding boxes. Therefore, the first objective algorithm includes an edge extraction algorithm, a binarization algorithm, and a Hough detection algorithm. The specific implementation steps include: The first target algorithm is obtained, which includes an edge extraction algorithm, a binarization algorithm, and a Hough detection algorithm. An edge intensity map of the first type of trademark advertisement segmentation map is generated based on the edge extraction algorithm; The pixels of the edge intensity map are binarized according to the binarization algorithm; Based on the Hough detection algorithm, the binarized first-class trademark advertisement segmentation map is divided into a third-class trademark advertisement segmentation map with rectangular bounding boxes and a fourth-class trademark advertisement segmentation map with non-rectangular bounding boxes.

[0029] In addition, to ensure a natural transition when the target trademark advertising image is replaced with the trademark advertising image to be replaced, it is necessary to perform anti-aliasing on the binary image and then perform image smoothing on the anti-aliased image. The smoothing operation can be performed by calling the 5×5 kernel function 5×5blur to change the 0 and 1 in the binary image to values ​​of 0 to 1. It should be noted that in addition to the 5×5 kernel function, other kernel functions can also be set, but this invention does not make specific limitations here.

[0030] Step 103: Perform an affine transformation on the foreground of the target trademark advertisement image in the second-class trademark advertisement segmentation image to generate the first trademark advertisement image.

[0031] After classifying the trademark advertisement images to be replaced, different replacement methods are used for different types of trademark advertisements. This embodiment of the invention replaces trademark advertisements in a segmented image of a second type of trademark advertisement without bounding boxes. The replacement method includes: removing pixels belonging to the foreground markers in the segmented image of the second type of trademark advertisement, and then filling the removed parts using an image filling network, where the image filling network can be a PDGAN network. After processing the trademark advertisement images to be replaced, the replacement operation begins. Because the length and width of the target trademark advertisement image and the replacement trademark advertisement image are not completely identical, it is necessary to obtain the first coordinate information of the four vertices of both. Based on the first coordinate information, an affine transformation matrix can be calculated. Then, based on the generated affine transformation matrix, the scaling ratio required for the target trademark advertisement to completely correspond to the replacement trademark advertisement image is determined. Since this is a trademark advertisement replacement of a segmented image of a second type of trademark advertisement without bounding boxes, only the foreground part can be replaced. That is, the foreground of the target trademark advertisement image is affinely transformed in the segmented image of the second type of trademark advertisement to generate the first trademark advertisement image.

[0032] The specific implementation steps include: Obtain the pixels of the foreground portion of the segmented image of the Class 2 trademark advertisement; After removing the pixels of the foreground portion, the image is filled using an image filling network to generate a second type of trademark advertising background filling image; Obtain the first coordinate information of the four vertices of the target trademark advertisement image and the trademark advertisement image to be replaced; Generate a first affine transformation matrix based on the first coordinate information; Based on the first affine transformation matrix, the foreground of the target trademark advertisement image is affinely transformed in the background-filled image of the second type of trademark advertisement to generate the first trademark advertisement image.

[0033] Step 104: Perform an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmentation image of the third-class trademark advertisement to generate a second trademark advertisement image.

[0034] The above-described replacement of trademark advertisements for Class II trademark advertisement segments without bounding boxes is further elaborated in this embodiment of the invention for Class III trademark advertisement segments with rectangular bounding boxes. When replacing this type of trademark advertisement image, since the replacement involves replacing the portion within the rectangular bounding box of the target trademark advertisement image with the portion within the rectangular bounding box of the Class III trademark advertisement segment image, the target trademark advertisement image is first obtained. Figure 4The coordinates of the first vertex and the coordinates of the four vertices of the rectangular bounding box in the third-class trademark advertisement segmentation image are used to calculate the affine transformation matrix. Then, based on the generated affine transformation matrix, the scaling ratio required to replace the third-class trademark advertisement segmentation image with the target trademark advertisement is determined. In addition, the final position of the target trademark advertisement replacement can be determined based on the coordinates of the four vertices of the rectangular bounding box in the third-class trademark advertisement segmentation image. Finally, the target trademark advertisement image and the part within the rectangular bounding box in the third-class trademark advertisement segmentation image are subjected to affine transformation to generate the second trademark advertisement image.

[0035] Step 105: Perform an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the fourth class trademark advertisement segmentation image to generate the third trademark advertisement image.

[0036] In this embodiment of the invention, advertisement replacement is also performed on the fourth type of trademark advertisement segmentation image with non-rectangular bounding boxes. When replacing this type of trademark advertisement image, because the shapes of non-rectangular bounding boxes are varied, including circles, triangles, polygons, and even irregular shapes, setting a replacement method for each would be computationally intensive and costly. Therefore, this embodiment of the invention sets up a unified operation to replace these fourth type of trademark advertisement segmentation images with non-rectangular bounding boxes. First, the smallest bounding rectangle of these non-rectangular bounding boxes is taken, and then the operation is performed according to the replacement method in the third type of trademark advertisement segmentation image, that is, the target trademark advertisement is obtained. Figure 4 The coordinates of the four vertices of the rectangular bounding box in the fourth-class trademark advertisement segmentation image are used to calculate the affine transformation matrix. Then, based on the generated affine transformation matrix, the scaling ratio required to replace the fourth-class trademark advertisement segmentation image with the target trademark advertisement is determined. Finally, the portion within the smallest bounding rectangle of the non-rectangular bounding box in both the target trademark advertisement image and the fourth-class trademark advertisement segmentation image is subjected to an affine transformation to generate a third trademark advertisement image. The specific implementation steps include: Obtain the pixel marker values ​​of the advertisement segmentation image for Class 4 trademarks; The fourth type of trademark advertisement is segmented according to the pixel marker value. Figure 2 Value-based processing generates a second binary image; The minimum bounding rectangle of the non-rectangular bounding box in the fourth type of trademark advertisement segmentation diagram is generated according to the third objective algorithm; Obtain the third coordinate information of the four vertices of the target trademark advertisement image and the minimum bounding rectangle; Generate a third affine transformation matrix based on the third coordinate information; The third trademark advertisement image is generated by performing an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement based on the third affine transformation matrix.

[0037] In addition, since the fourth type of trademark advertisement segmentation image may also contain non-trademark advertisement parts, binarization processing will be used to obtain the minimum bounding rectangle of the trademark advertisement image part to be replaced. The third objective algorithm can be the minAreaRect method, or it can be the cv2.minAreaRect(cnt) function of Python OpenCV. This invention does not make specific limitations here.

[0038] In this embodiment of the invention, the target segmentation network and the target recognition network respectively label the trademark advertisement image to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes. A deep learning network is used to label the trademark advertisement image to be replaced, facilitating subsequent differentiation and processing of the trademark advertisement image to be replaced. According to the first target algorithm, the first-class trademark advertisement segmentation image is divided into a third-class trademark advertisement segmentation image with rectangular bounding boxes and a fourth-class trademark advertisement segmentation image with non-rectangular bounding boxes. The foreground of the target trademark advertisement image is subjected to an affine transformation in the second-class trademark advertisement segmentation image to generate the first trademark advertisement image. This can achieve the processing of borderless trademark advertisement images. The replacement method involves performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement to generate a second trademark advertisement image. Similarly, an affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmented image of the fourth-class trademark advertisement to generate a third trademark advertisement image. This method can replace trademark advertisements with borders and allows for further refinement based on different borders, resulting in a better trademark replacement effect. In summary, this invention can replace not only trademark advertisements with rectangular bounding boxes but also those without borders or with non-rectangular bounding boxes, avoiding the problem of relatively simple and inefficient trademark advertisement replacement results and improving the overall replacement effect.

[0039] Figure 2 This is the second step in the flowchart of an advertisement replacement method provided in an embodiment of the present invention. This method is similar to... Figure 1 The methods for replacing advertisements are basically the same, except that, before step 101, the following may also be included: Step 201: Obtain the target video frame according to the preset rules.

[0040] Before obtaining the trademark advertisement image to be replaced, this embodiment of the invention first determines which video frames contain the trademark advertisements to be replaced. The preset rules here can be based on precise analysis of user needs using regional information, user browsing history, and video content, and then correspondingly obtain the target video frames for advertisement replacement and the target trademark advertisement images to be replaced. For example, advertisement replacement can be performed on video frames with high view counts, and advertisements for children's snacks, clothing, etc., can be replaced in parent-child videos.

[0041] Step 202: Obtain video keyframes from the target video frame. Each keyframe contains the trademark advertisement image to be replaced.

[0042] In this embodiment of the invention, after obtaining the target video frame, since the trademark advertisement in the video frame is to be replaced, it is also necessary to filter out the video key frames that include the trademark advertisement image to be replaced in the target video frame.

[0043] Step 203: Obtain the trademark advertisement image to be replaced from the video keyframes using the object detection network.

[0044] In this embodiment of the invention, after selecting video keyframes containing the trademark advertisement image to be replaced, these video keyframes are input into a target detection network to locate the trademark advertisement images and obtain the target's location information. It should be noted that because the identified trademark advertisement images include those with non-rectangular bounding boxes and those without bounding boxes, the smallest bounding rectangle of these trademark advertisement images is used for uniformity. Then, the vertex coordinates of these smallest bounding rectangles are taken as the location information. These can be the coordinates of the top left and bottom right corners, or the coordinates of the four corners; this invention does not impose a specific limitation. After obtaining the location information of the trademark advertisement image to be replaced, it is then cut out from the video keyframes based on the location information. The specific implementation steps include: Generate the minimum bounding rectangle of the trademark advertisement image to be replaced based on the first objective algorithm; The object detection network generates localization information about the minimum bounding rectangle. The trademark advertisement image to be replaced is obtained from the keyframes of the video based on the location information.

[0045] Figure 3 This is the third step in the flowchart of an advertisement replacement method provided in an embodiment of the present invention. This method is related to... Figure 1 The methods for replacing advertisements are basically the same, the difference being that step 104 may also include: Step 301: Obtain the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the segmentation image of the third-class trademark advertisement.

[0046] In this embodiment of the invention, after dividing the first type of trademark advertisement segmentation image into a third type of trademark advertisement segmentation image with rectangular bounding boxes and a fourth type of trademark advertisement segmentation image with non-rectangular bounding boxes using a first algorithm, the third type of trademark advertisement segmentation image with rectangular bounding boxes is processed. First, the third type of trademark advertisement segmentation image is binarized, assigning different values ​​to the parts belonging to the trademark advertisement and the parts not belonging to the trademark advertisement to generate a first binary image. Then, a second target algorithm is used to process the first binary image to obtain the part within the rectangular bounding box in the third type of trademark advertisement segmentation image. This second target algorithm includes anti-aliasing algorithms, smoothing algorithms, and rectangular border recognition algorithms, etc., which are not specifically limited here. Specific implementation steps include: Obtain the pixel marker values ​​of the segmentation image of the Class 3 trademark advertisement; The third type of trademark advertisement is segmented according to the pixel marker value. Figure 2 Value-based processing generates a first-order binary image; The first binary image is processed according to the second objective algorithm to obtain the portion within the rectangular bounding box in the segmentation image of the third type of trademark advertisement.

[0047] In addition, after identifying the portion of the advertisement that belongs to the trademark within the rectangular bounding box in the segmentation diagram of the third-class trademark advertisement, it is also necessary to obtain the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement diagram and the segmentation diagram of the third-class trademark advertisement, so as to calculate the affine transformation matrix.

[0048] Step 302: Generate the second affine transformation matrix based on the second coordinate information.

[0049] In this embodiment of the invention, after obtaining the second coordinate information, the affine transformation matrix is ​​calculated based on the coordinate information of both coordinates to generate the second affine transformation matrix.

[0050] Step 303: Perform an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the third-class trademark advertisement segmentation image according to the second affine transformation matrix to generate the second trademark advertisement image.

[0051] In this embodiment of the invention, the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third type of trademark advertisement is transformed using a second affine transformation matrix to generate a second trademark advertisement image. Furthermore, to ensure a natural transition at the boundary of the interpolated image, anti-aliasing is applied to the first binary image, and image smoothing is performed on the anti-aliased image. The smoothing operation can be achieved by calling a 5×5 kernel function, 5×5Blur, to convert the 0s and 1s in the first binary image to values ​​between 0 and 1. Interpolation coefficients are then obtained based on 5×5Blur, and these coefficients are used to ensure a natural transition at the boundary of the interpolated image. It should be noted that other kernel functions besides 5×5 can be used; this invention does not impose specific limitations on these kernel functions.

[0052] Figure 4 This is a structural diagram of an advertisement replacement device provided in an embodiment of the present invention. The device may include: The first generation module 401 is used to mark the trademark advertisement image to be replaced according to the target segmentation network and the target recognition network respectively, and generate a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes.

[0053] The segmentation module 402 is used to divide the first class trademark advertisement segmentation map into a third class trademark advertisement segmentation map with a rectangular bounding box and a fourth class trademark advertisement segmentation map with a non-rectangular bounding box according to the first target algorithm.

[0054] The second generation module 403 is used to perform an affine transformation on the foreground of the target trademark advertisement image in the second class trademark advertisement segmentation image to generate the first trademark advertisement image.

[0055] The third generation module 404 is used to perform affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the third-class trademark advertisement segmentation image to generate a second trademark advertisement image.

[0056] The fourth generation module 405 is used to perform an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the fourth class trademark advertisement segmentation image to generate a third trademark advertisement image.

[0057] Optionally, the advertisement replacement device also includes: The first acquisition module is used to acquire target video frames according to preset rules.

[0058] The second acquisition module is used to acquire video keyframes from the target video frame, and each keyframe includes the trademark advertisement image to be replaced.

[0059] The third acquisition module is used to acquire the trademark advertisement image to be replaced from the video keyframes based on the object detection network.

[0060] Optionally, the third acquisition module also includes: The first generation submodule is used to generate the minimum bounding rectangle of the trademark advertisement image to be replaced according to the first objective algorithm.

[0061] The second generation submodule is used to generate localization information about the minimum bounding rectangle based on the object detection network.

[0062] The first acquisition submodule is used to obtain the trademark advertisement image to be replaced from the video keyframes based on the positioning information.

[0063] Optionally, the first generation module 401 further includes: The second acquisition submodule is used to acquire the set of pixels of the trademark advertisement image to be replaced.

[0064] The third generation submodule is used to generate foreground markers, background markers, and non-trademark markers for the trademark advertisement image to be replaced, based on the target segmentation network and pixel set.

[0065] The fourth generation submodule is used to generate a first marker and a second marker based on the target recognition network and the pixel set. The first marker is used to mark the trademark advertisement image to be replaced with a bounding box, and the second marker is used to mark the trademark advertisement image to be replaced without a bounding box.

[0066] The first determination submodule is used to determine a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box based on the foreground mark, background mark, non-to-be-replaced trademark advertisement image mark, first mark, and second mark.

[0067] Optionally, the partitioning module 402 further includes: The third acquisition submodule is used to acquire the first target algorithm, which includes an edge extraction algorithm, a binarization algorithm, and a Hough detection algorithm.

[0068] The fifth generation submodule is used to generate the edge intensity map of the first type of trademark advertisement segmentation map based on the edge extraction algorithm.

[0069] The binarization processing submodule is used to binarize the pixels of the edge intensity map according to the binarization algorithm.

[0070] The first segmentation submodule is used to divide the binarized first-class trademark advertisement segmentation map into a third-class trademark advertisement segmentation map with rectangular bounding boxes and a fourth-class trademark advertisement segmentation map with non-rectangular bounding boxes according to the Hough detection algorithm.

[0071] Optionally, the second generation module 403 further includes: The fourth acquisition submodule is used to acquire the pixels of the foreground part of the second type of trademark advertisement segmentation image.

[0072] The sixth generation submodule is used to remove the pixels of the foreground part and then fill them through an image filling network to generate the background filling image for the second type of trademark advertisement.

[0073] The fifth acquisition submodule is used to acquire the first coordinate information of the four vertices of the target trademark advertisement image and the trademark advertisement image to be replaced.

[0074] The seventh generation submodule is used to generate the first affine transformation matrix based on the first coordinate information.

[0075] The eighth generation submodule is used to perform an affine transformation on the foreground of the target trademark advertisement image in the background-filled image of the second type of trademark advertisement based on the first affine transformation matrix to generate the first trademark advertisement image.

[0076] Optionally, the third generation module 404 further includes: The sixth acquisition submodule is used to acquire the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the segmentation image of the third-class trademark advertisement.

[0077] The ninth generation submodule is used to generate the second affine transformation matrix based on the second coordinate information.

[0078] The tenth generation submodule is used to perform an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the third-class trademark advertisement segmentation image based on the second affine transformation matrix to generate the second trademark advertisement image.

[0079] The seventh submodule is used to obtain the pixel marker values ​​of the third-class trademark advertisement segmentation image.

[0080] The eleventh generation submodule is used to segment the Class 3 trademark advertisement based on pixel marker values. Figure 2 Value-enhanced processing generates a first-valued binary image.

[0081] The ninth acquisition submodule is used to process the first binary image according to the second target algorithm to obtain the part within the rectangular bounding box in the segmentation image of the third type of trademark advertisement.

[0082] Optionally, the fourth generation module 405 further includes: The tenth submodule is used to obtain the pixel marker values ​​of the fourth class trademark advertisement segmentation image.

[0083] The twelfth generation submodule is used to segment the Class 4 trademark advertisement based on the pixel marker values. Figure 2 Value-enhanced processing generates a second binary image.

[0084] The thirteenth generation submodule is used to generate the minimum bounding rectangle of the non-rectangular bounding box in the fourth category trademark advertisement segmentation graph according to the third objective algorithm.

[0085] The eleventh submodule is used to obtain the third coordinate information of the four vertices of the target trademark advertisement image and the minimum bounding rectangle.

[0086] The fourteenth generation submodule is used to generate the third affine transformation matrix based on the third coordinate information.

[0087] The fifteenth generation submodule is used to perform an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the fourth class trademark advertisement segmentation image based on the third affine transformation matrix to generate the third trademark advertisement image.

[0088] In this embodiment of the invention, the target segmentation network and the target recognition network respectively label the trademark advertisement image to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes. A deep learning network is used to label the trademark advertisement image to be replaced, facilitating subsequent differentiation and processing of the trademark advertisement image. According to the first target algorithm, the first-class trademark advertisement segmentation image is divided into a third-class trademark advertisement segmentation image with rectangular bounding boxes and a fourth-class trademark advertisement segmentation image with non-rectangular bounding boxes. The foreground of the target trademark advertisement image is subjected to an affine transformation in the second-class trademark advertisement segmentation image to generate the first trademark advertisement image. This method can achieve the processing of borderless trademark advertisement images. The replacement method involves performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement to generate a second trademark advertisement image. Similarly, an affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmented image of the fourth-class trademark advertisement to generate a third trademark advertisement image. This method can replace trademark advertisements with borders and allows for further refinement based on different borders, resulting in a better trademark replacement effect. In summary, this invention can replace not only trademark advertisements with rectangular bounding boxes but also those without borders or with non-rectangular bounding boxes, avoiding the problem of relatively simple and inefficient trademark advertisement replacement results and improving the overall replacement effect.

[0089] The present invention also provides an electronic device, Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. See also... Figure 5 The system includes a processor 501, a memory 502, and a computer program 5021 stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: an advertisement replacement method. The target segmentation network and the target recognition network are used to label the advertisement image of the trademark to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes; According to the first objective algorithm, the first type of trademark advertisement segmentation map is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box. The foreground of the target trademark advertisement image is transformed into the second-class trademark advertisement segmentation image through an affine transformation to generate the first trademark advertisement image; A second trademark advertisement image is generated by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement. Affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement to generate the third trademark advertisement image.

[0090] The present invention also provides a readable storage medium, wherein when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is able to perform the advertising replacement method of the foregoing embodiments.

[0091] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0092] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. The structure required to construct such a system is readily apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0093] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0094] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0095] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.

[0096] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0097] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0098] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0100] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0101] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. An advertisement replacement method, characterized in that, The method includes: The target segmentation network and the target recognition network are used to label the advertisement image of the trademark to be replaced, generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes; According to the first objective algorithm, the first type of trademark advertisement segmentation map is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box. The foreground of the target trademark advertisement image is transformed into the second-class trademark advertisement segmentation image through an affine transformation to generate the first trademark advertisement image; A second trademark advertisement image is generated by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement. Affine transformation is performed on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth class trademark advertisement to generate a third trademark advertisement image.

2. The method according to claim 1, characterized in that, Before the step of labeling the advertisement image of the trademark to be replaced according to the target segmentation network and the target recognition network respectively, and generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes, the method further includes: Obtain the target video frame according to preset rules; Video keyframes are obtained from the target video frame, and each frame of the video keyframe includes the trademark advertisement image to be replaced; The trademark advertisement image to be replaced is obtained from the keyframes of the video using an object detection network.

3. The method according to claim 2, characterized in that, The step of obtaining the trademark advertisement image to be replaced from the video keyframes based on the target detection network includes: The minimum bounding rectangle of the trademark advertisement image to be replaced is generated according to the first objective algorithm; The target detection network generates positioning information about the minimum bounding rectangle. The trademark advertisement image to be replaced is obtained from the video keyframes based on the location information.

4. The method according to claim 1, characterized in that, The step of marking the trademark advertisement image to be replaced according to the target segmentation network and the target recognition network respectively, and generating a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes includes: Obtain the set of pixels of the advertisement image of the trademark to be replaced; The foreground marker, background marker, and non-trademark marker of the trademark advertisement image to be replaced are generated based on the target segmentation network and the pixel set. A first marker and a second marker are generated based on the target recognition network and the pixel set. The first marker is used to mark the trademark advertisement image to be replaced with a bounding box, and the second marker is used to mark the trademark advertisement image to be replaced without a bounding box. Based on the foreground marker, the background marker, the non-replaceable trademark advertisement image marker, the first marker, and the second marker, a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box are determined.

5. The method according to claim 1, characterized in that, The step of dividing the first type of trademark advertisement segmentation map into a third type of trademark advertisement segmentation map with rectangular bounding boxes and a fourth type of trademark advertisement segmentation map with non-rectangular bounding boxes according to the first target algorithm includes: A first target algorithm is obtained, which includes an edge extraction algorithm, a binarization algorithm, and a Hough detection algorithm. An edge intensity map of the first type of trademark advertisement segmentation map is generated according to the edge extraction algorithm; The pixels of the edge intensity map are binarized according to the binarization algorithm. According to the Hough detection algorithm, the first type of trademark advertisement segmentation map after binarization is divided into a third type of trademark advertisement segmentation map with a rectangular bounding box and a fourth type of trademark advertisement segmentation map with a non-rectangular bounding box.

6. The method according to claim 1, characterized in that, The step of generating the first trademark advertisement image by performing an affine transformation on the foreground of the target trademark advertisement image in the second class trademark advertisement segmentation image includes: Obtain the pixels of the foreground portion of the segmented image of the Class 2 trademark advertisement; After removing the pixels of the foreground portion, the image is filled using an image filling network to generate a second type of trademark advertising background filling image; Obtain the first coordinate information of the four vertices of the target trademark advertisement image and the trademark advertisement image to be replaced; Generate a first affine transformation matrix based on the first coordinate information; Based on the first affine transformation matrix, the foreground of the target trademark advertisement image is affinely transformed in the background-filled image of the second type of trademark advertisement to generate the first trademark advertisement image.

7. The method according to claim 1, characterized in that, The step of generating a second trademark advertisement image by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the segmented image of the third-class trademark advertisement includes: Obtain the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the segmentation image of the third-class trademark advertisement; Generate a second affine transformation matrix based on the second coordinate information; The second trademark advertisement image is generated by performing an affine transformation on the portion within the rectangular bounding box of the target trademark advertisement image and the third-class trademark advertisement segmentation image based on the second affine transformation matrix.

8. The method according to claim 7, characterized in that, Before obtaining the second coordinate information of the four vertices of the rectangular bounding box in the target trademark advertisement image and the third-class trademark advertisement segmentation image, the method further includes: Obtain the pixel marker values ​​of the segmented image of the Class 3 trademark advertisement; The third-class trademark advertisement segmentation image is binarized based on the pixel marker values ​​to generate a first binary image; The first binary image is processed according to the second objective algorithm to obtain the portion within the rectangular bounding box in the third type of trademark advertisement segmentation image.

9. The method according to claim 1, characterized in that, The step of generating a third trademark advertisement image by performing an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmented image of the fourth class trademark advertisement includes: Obtain the pixel marker values ​​of the advertisement segmentation image for Class 4 trademarks; The fourth-class trademark advertisement segmentation image is binarized based on the pixel marker values ​​to generate a second binary image. The minimum bounding rectangle of the non-rectangular bounding box in the fourth type of trademark advertisement segmentation diagram is generated according to the third objective algorithm; Obtain the third coordinate information of the four vertices of the target trademark advertisement image and the minimum bounding rectangle; Generate a third affine transformation matrix based on the third coordinate information; The third trademark advertisement image is generated by performing an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement based on the third affine transformation matrix.

10. An advertising replacement device, characterized in that, The device includes: The first generation module is used to mark the trademark advertisement image to be replaced according to the target segmentation network and the target recognition network, and generate a first-class trademark advertisement segmentation image with bounding boxes and a second-class trademark advertisement segmentation image without bounding boxes. The segmentation module is used to divide the first type of trademark advertisement segmentation image into a third type of trademark advertisement segmentation image with rectangular bounding boxes and a fourth type of trademark advertisement segmentation image with non-rectangular bounding boxes according to the first target algorithm. The second generation module is used to perform an affine transformation on the foreground of the target trademark advertisement image in the second class of trademark advertisement segmentation image to generate the first trademark advertisement image; The third generation module is used to perform an affine transformation on the target trademark advertisement image and the portion within the rectangular bounding box of the third class trademark advertisement segmentation image to generate a second trademark advertisement image. The fourth generation module is used to perform an affine transformation on the portion within the smallest bounding rectangle of the non-rectangular bounding box of the target trademark advertisement image and the segmentation image of the fourth type of trademark advertisement to generate a third trademark advertisement image.

11. The apparatus according to claim 10, characterized in that, The advertisement replacement device also includes: The first acquisition module is used to acquire target video frames according to preset rules; The second acquisition module is used to acquire video key frames from the target video frame, and each frame of the video key frame includes the trademark advertisement image to be replaced. The third acquisition module is used to acquire the trademark advertisement image to be replaced from the video keyframes based on the target detection network.

12. The apparatus according to claim 11, characterized in that, The third acquisition module further includes: The first generation submodule is used to generate the minimum bounding rectangle of the trademark advertisement image to be replaced according to the first target algorithm; The second generation submodule is used to generate positioning information about the minimum bounding rectangle based on the target detection network; The first acquisition submodule is used to acquire the trademark advertisement image to be replaced from the video keyframes based on the positioning information.

13. The apparatus according to claim 10, characterized in that, The first generation module further includes: The second acquisition submodule is used to acquire the set of pixels of the trademark advertisement image to be replaced; The third generation submodule is used to generate foreground markers, background markers, and non-trademark advertising image markers of the trademark to be replaced based on the target segmentation network and the pixel set. The fourth generation submodule is used to generate a first marker and a second marker based on the target recognition network and the pixel set. The first marker is used to mark the trademark advertisement image to be replaced with a bounding box, and the second marker is used to mark the trademark advertisement image to be replaced without a bounding box. The first determining submodule is used to determine a first-class trademark advertisement segmentation image with a bounding box and a second-class trademark advertisement segmentation image without a bounding box based on the foreground marker, the background marker, the non-to-be-replaced trademark advertisement image marker, the first marker, and the second marker.

14. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the advertising replacement method as described in any one of claims 1-9.

15. A readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform one or more of the advertising replacement methods described in claims 1-9.

Images