Video material background replacement method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence and image processing, and provides a video material background replacement method and device, equipment and a medium, and the method comprises the steps: obtaining an original video material, and carrying out the batch extraction of video material frame images; performing foreground and background segmentation on each video material frame image by using a pre-trained image segmentation model to obtain a corresponding target foreground image; obtaining a target background image; generating a target video material frame image according to each of the target foreground image and the target background image; and carrying out integration processing on the target video material frame image to obtain a target video material. Through the technical scheme of the invention, the finally obtained target video material looks more natural and harmonious, the use experience is greatly improved, the image segmentation is carried out through the image segmentation model, the labor cost is reduced, the video material does not need to be manually and complexly segmented and processed, and the user experience is improved. And a large amount of human resources and time cost are saved.

Description

[Technical field]

[0001] The present application relates to the field of artificial intelligence and image processing technology, and in particular to a method, device, equipment and medium for replacing the background of a video material. [Background technology]

[0002] In order to operate the AI ​​tracking business, the relevant technology uses a fixed background template produced by the product, and the template is used as the background and synthesized with the shot video material during shooting. Since the background of the video material when the photographer shoots and the background template are used as the background of the video material together, the background and foreground of the shot content appear inconsistent, the user experience is poor, and there is a lack of immersion. In addition, since the position of the background template is fixed, the photographer is required to shoot at a fixed perspective, and the video material content lacks flexibility.

[0003] In addition, because the video material is a synthesized template during shooting, the video material that the shooter has shot before must be shot again before it can be used. It is not possible to merge multiple sets of templates through one piece of video material to generate multiple sets of video materials. [Summary of the invention]

[0004] The embodiments of the present application provide a method, device, equipment and medium for replacing the background of a video material, aiming to solve the technical problems existing in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for replacing a background of a video material, comprising:

[0006] Obtain original video material and perform batch extraction of video material frame images;

[0007] Use the pre-trained image segmentation model to segment the foreground and background of each video frame image to obtain the corresponding target foreground image;

[0008] Get the target background image;

[0009] Generate a target video material frame image according to each of the target foreground images and the target background images;

[0010] The target video material frame images are integrated and processed to obtain the target video material.

[0011] In a second aspect, an embodiment of the present application provides a video material background replacement device, comprising:

[0012] The extraction module is used to obtain the original video material and perform batch extraction of video material frame images;

[0013] A segmentation module is used to segment the foreground and background of each video material frame image using a pre-trained image segmentation model to obtain a corresponding target foreground image;

[0014] An acquisition module, used for acquiring a target background image;

[0015] A generating module, used for generating a target video material frame image according to each of the target foreground image and the target background image;

[0016] The integration module is used to integrate the target video material frame images to obtain the target video material.

[0017] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned video material background replacement method when executing the computer program.

[0018] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned video material background replacement method are implemented.

[0019] In the above scheme implemented by the method, device, equipment and medium for replacing the background of video material, the original video material can be obtained, and the video material frame images can be extracted in batches; the foreground and background of each video material frame image can be segmented using a pre-trained image segmentation model to obtain the corresponding target foreground image; the target background image can be obtained; according to each of the target foreground image and the target background image, a target video material frame image can be generated; the target video material frame image can be integrated and processed to obtain the target video material. Through the above technical scheme of the present invention, the foreground and background of the original video material are segmented frame by frame by a pre-trained image segmentation model, so that a target foreground image with a better segmentation effect can be obtained, and then the target foreground image and the target background image are generated frame by frame to generate the target video material frame image, and then integrated into the target video material, so that the target video material finally obtained can look more natural and harmonious, and the user experience is greatly improved. At the same time, the image segmentation model is used to perform image segmentation, which reduces the labor cost, and there is no need to manually perform complex segmentation and processing of the video material, saving a lot of human resources and time costs.

Brief Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic flow chart of a method for replacing a background of a video material according to an embodiment of the present application is shown.

[0022] Figure 2 A schematic flow chart of step S102 in a method for replacing a background of a video material according to an embodiment of the present application is shown.

[0023] Figure 3 A schematic flow chart of step S104 in a method for replacing a background of a video material according to an embodiment of the present application is shown.

[0024] Figure 4 A schematic flow chart of step S302 in a method for replacing a background of a video material according to an embodiment of the present application is shown.

[0025] Figure 5 A block diagram of a device for replacing background of video material according to an embodiment of the present application is shown.

[0026] Figure 6 A block diagram of a computer device according to an embodiment of the present application is shown. [Specific implementation method]

[0027] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0028] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0029] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0030] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] It should be noted that the embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0032] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0033] See also Figure 1 , Figure 1 A schematic flow chart of a method for replacing a background of a video material according to an embodiment of the present application is shown.

[0034] like Figure 1 As shown, the process of the video material background replacement method according to an embodiment of the present application includes:

[0035] Step S101, obtaining original video material, and performing batch extraction of video material frame images;

[0036] In one embodiment, optionally, obtaining the original video material includes:

[0037] Get the original video material stored locally; or

[0038] Get the original video footage shot in real time.

[0039] In this step, the original video material can be either the video material that has been stored locally or the video material that has been shot in real time. In this way, the photographer only needs to shoot a piece of original video material, and can replace the background of the video material at will, without the need for real-time shooting, thereby reducing the requirements for the photographer's video material. The photographer only needs to ensure that the foreground is simple when shooting video materials. The background template is more flexible, and the photographer can replace various styles of backgrounds according to business scenarios. This makes it easy to merge multiple sets of background templates according to a piece of video material and generate multiple sets of video materials.

[0040] The video material may be in the format of MP4, AVI, etc.

[0041] When performing batch extraction of video material frame images, FFMPEG can be used to perform batch extraction of video material frames. Specifically, the following command can be used to extract frames from the video:

[0042] ffmpeg-iinput_video.mp4 frames / frame_%04d.jpg

[0043] Among them, input_video.mp4 is the input video file name, frames / frame_%04d.jpg means saving the extracted frames as pictures, and the picture file name starts with "frame_" followed by a four-digit number, such as frame_0001.jpg, frame_0002.jpg, etc.

[0044] If you want to control the frame rate of the extraction, you can use the -r parameter. For example, to extract at 30 frames per second, you can use the following command:

[0045] ffmpeg-iinput_video.mp4-r 30frames / frame_%04d.jpg

[0046] Furthermore, if you only want to extract frames from a specific period of time in the video, you can use the -ss (start time) and -t (duration) parameters. For example, to start from the 10th second of the video and extract the next 20 seconds of frames, you can use the following command:

[0047] ffmpeg-iinput_video.mp4-ss 00:00:10-t 00:00:20frames / frame_%04d.jpg

[0048] Afterwards, the extracted video material frame image files are cached and wait for segmentation processing.

[0049] Step S102, using a pre-trained image segmentation model to perform foreground and background segmentation on each video material frame image to obtain a corresponding target foreground image;

[0050] like Figure 2 As shown, in one embodiment, optionally, step S102 includes:

[0051] Step S201, loading a pre-trained image segmentation model and outputting prompt information for selecting a target foreground image, wherein the image segmentation model includes a SAM image segmentation model;

[0052] The SAM (Segment Anything Model) image segmentation model is a powerful basic model for image segmentation developed by Meta AI. Unlike previous image segmentation models that can only process specific types of images, SAM can process all types of images, whether they are natural scenes, medical images, industrial images, and other fields. It can segment unfamiliar objects and images without the need for additional training, and can achieve good recognition and segmentation effects even for objects or image types that have never been seen before. For example, SAM can also better identify and segment the outlines of novel objects that have not appeared in the training data set.

[0053] Step S202, receiving target foreground prompt information input by the user according to the prompt information;

[0054] In this step, the image segmentation model can recognize various input prompts and determine the content in the image that needs to be segmented. The prompt can be a foreground / background point, a rough box or mask, free text, etc., or a specified person, animal, plant, etc. By designing appropriate prompts, users can flexibly implement various segmentation tasks. For example, if the user wants to extract a specified person image, they can enter the person prompt word, so that a mask image will be generated for the person so that the mask image can be extracted.

[0055] Step S203, performing foreground and background segmentation on each video material frame image according to the target foreground prompt information, so as to obtain a target foreground image corresponding to the target foreground prompt information;

[0056] In this step, foreground and background segmentation is performed on each video material frame image to obtain a target foreground image corresponding to the target foreground prompt information, wherein the target foreground prompt information can be for any target object in the image, such as a person, an animal, a plant or a specified object, etc., so as to achieve accurate segmentation and obtain a target foreground image.

[0057] Step S204, arranging and saving the target foreground images according to the video material frame numbers.

[0058] In this step, foreground and background are segmented for each video material frame image, and the foreground image is output and saved according to the video material frame sequence number. In this way, it is convenient to integrate according to the video material frame sequence number later.

[0059] Step S103, obtaining a target background image;

[0060] In one embodiment, optionally, step S103 includes:

[0061] Receive a target background image input by the user; or

[0062] A preset video image is acquired, and the target background image is extracted from the preset video image according to a target background image extraction command.

[0063] In this step, the target background image can be the target background image that the photographer wants to replace by uploading, of course, it can also be the target background image extracted from the preset video image, so that the different background requirements of different users can be met, thereby generating the video material that the user wants.

[0064] Step S104, generating a target video material frame image according to each of the target foreground images and the target background images;

[0065] like Figure 3 As shown, in one embodiment, optionally, step S104 includes:

[0066] Step S301, preprocessing the target background image so that the preprocessed target background image has the same size as the video material frame image;

[0067] In this step, the target background image is preprocessed, and the cut and segmented background image and the video material frame image have the same size, that is, the same width and height. In this way, it is ensured that the target video material frame image after reintegration is the same as the original video material frame image except for the background.

[0068] Step S302: fuse each of the target foreground images and the target background images to generate the target video material frame image.

[0069] In this step, each target foreground image and target background image are fused to generate a target video material frame image, which can be cached.

[0070] like Figure 4 As shown, in one embodiment, optionally, step S302 further includes:

[0071] Step S401, performing edge smoothing processing on each of the target foreground images to obtain a processed target foreground image;

[0072] In this step, before the target foreground image and the target background image are fused, edge smoothing may be performed on each target foreground image to avoid an obvious edge line appearing at the junction of the foreground and the background.

[0073] Step S402, generating an initial video material frame image according to each of the processed target foreground images and target background images;

[0074] Step S403: adjusting parameters of the initial video material frame image to obtain the target video material frame image, wherein the parameter adjustment includes at least one of the following: color adjustment and illumination adjustment.

[0075] In this step, you can adjust the color and lighting parameters of the video material frame image. Specifically, you can adjust the color matching of the foreground and background to make it look more natural and harmonious. Of course, you can also adjust the brightness and contrast of the background according to the foreground lighting conditions. In this way, the final video material can look more real and natural.

[0076] Step S105, integrating the target video material frame images to obtain the target video material.

[0077] In one embodiment, optionally, step S105 includes:

[0078] According to the video material frame sequence number, all target video material frame images are integrated and processed to obtain the target video material.

[0079] In this step, the processed target video material frame image can be resynthesized into video material through ffmpeg tool, the command is as follows: ffmpeg-framerate 30-i frame_%04d.png-c:vlibx264-pix_fmtyuv420p output.mp4

[0080] The above technical solution of the present invention uses a pre-trained image segmentation model to segment the foreground and background of the original video material frame by frame, which reduces labor costs, eliminates the need to manually perform complex segmentation and processing on the video material, saves a lot of human resources and time costs, and can also obtain a target foreground image with a better segmentation effect. In this way, the foreground and background in the video material can be automatically identified and segmented, and the photographer does not need to make preliminary scene preparations when shooting the video material. Video materials of daily scenes can be used for follow-up synthesis. At the same time, the photographer only needs to shoot a character video material, and can generate a variety of video materials for use by replacing different backgrounds, thereby improving efficiency and shooting costs. In addition, users can also easily expand and customize according to their needs, suitable for different scenes and needs. Users can choose different scene backgrounds according to their business needs to quickly replace and generate the desired video material, thereby improving the user experience.

[0081] Of course, this case can also be applied to other application scenarios that require video background segmentation and synthesis, which are not listed here one by one.

[0082] Figure 5 A block diagram of a device for replacing background of video material according to an embodiment of the present application is shown.

[0083] like Figure 5 As shown, in a second aspect, an embodiment of the present application provides a video material background replacement device 50, comprising:

[0084] The extraction module 51 is used to obtain the original video material and perform batch extraction of video material frame images;

[0085] A segmentation module 52 is used to segment each video material frame image into foreground and background using a pre-trained image segmentation model to obtain a corresponding target foreground image;

[0086] An acquisition module 53 is used to acquire a target background image;

[0087] A generating module 54, configured to generate a target video material frame image according to each of the target foreground image and the target background image;

[0088] The integration module 55 is used to integrate the target video material frame images to obtain the target video material.

[0089] In one embodiment, optionally, the extraction module 51 is used to:

[0090] Get the original video material stored locally; or

[0091] Get the original video footage shot in real time.

[0092] In one embodiment, optionally, the segmentation module 52 includes:

[0093] A prompt unit, used for loading a pre-trained image segmentation model and outputting prompt information for selecting a target foreground image, wherein the image segmentation model includes a SAM image segmentation model;

[0094] A receiving unit, used for receiving target foreground prompt information input by the user according to the prompt information;

[0095] An image segmentation unit, configured to segment each video material frame image into foreground and background according to the target foreground prompt information, so as to obtain a target foreground image corresponding to the target foreground prompt information;

[0096] The sorting unit is used to arrange and save the target foreground image according to the frame sequence number of the video material.

[0097] In one embodiment, optionally, the acquisition module is used to:

[0098] Receive a target background image input by the user; or

[0099] A preset video image is acquired, and the target background image is extracted from the preset video image according to a target background image extraction command.

[0100] In one embodiment, optionally, the generating module includes:

[0101] A first processing unit, configured to preprocess the target background image so that the preprocessed target background image has the same size as the video material frame image;

[0102] The fusion unit is used to fuse each of the target foreground images and the target background images to generate the target video material frame image.

[0103] In one embodiment, optionally, the integration module is used to:

[0104] According to the video material frame sequence number, all target video material frame images are integrated and processed to obtain the target video material.

[0105] In one embodiment, optionally, the generating module further includes:

[0106] A second processing unit, configured to perform edge smoothing processing on each of the target foreground images to obtain a processed target foreground image;

[0107] An image generating unit, used for generating an initial video material frame image according to each of the processed target foreground images and target background images;

[0108] The parameter adjustment unit is used to perform parameter adjustment on the initial video material frame image to obtain the target video material frame image, wherein the parameter adjustment includes at least one of the following: color adjustment and illumination adjustment.

[0109] In the above scheme implemented by the method, device, equipment and medium for replacing the background of video material, the original video material can be obtained, and the video material frame images can be extracted in batches; the foreground and background of each video material frame image can be segmented using a pre-trained image segmentation model to obtain the corresponding target foreground image; the target background image can be obtained; according to each of the target foreground image and the target background image, a target video material frame image can be generated; the target video material frame image can be integrated and processed to obtain the target video material. Through the above technical scheme of the present invention, the foreground and background of the original video material are segmented frame by frame by a pre-trained image segmentation model, so that a target foreground image with a better segmentation effect can be obtained, and then the target foreground image and the target background image are generated frame by frame to generate the target video material frame image, and then integrated into the target video material, so that the target video material finally obtained can look more natural and harmonious, and the user experience is greatly improved. At the same time, the image segmentation model is used to perform image segmentation, which reduces the labor cost, and there is no need to manually perform complex segmentation and processing of the video material, saving a lot of human resources and time costs.

[0110] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned video material background replacement method when executing the computer program.

[0111] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned video material background replacement method are implemented.

[0112] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the video material background replacement device and each module described above can refer to the corresponding process in the aforementioned video material background replacement method embodiment, and will not be repeated here.

[0113] It should be noted that those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the model training device and each module described above can refer to the corresponding processes in the aforementioned video material background replacement method embodiment, and will not be repeated here.

[0114] The above-mentioned video material background replacement device can be implemented in the form of a computer program. The computer program can be used in Figure 6 Runs on the computer device shown.

[0115] Figure 6 A block diagram of a computer device according to an embodiment of the present application is shown.

[0116] See also Figure 6 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a storage medium and an internal memory.

[0117] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any method for replacing the background of a video material of multi-source data provided in the embodiments of the present application.

[0118] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0119] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any infectious disease transmission path analysis method or prediction neural network training method. The storage medium can be non-volatile or volatile.

[0120] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0122] The computer device of the embodiment of the present application exists in various forms, including but not limited to:

[0123] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0124] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0125] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0126] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0127] (5) Other electronic devices with data interaction functions.

[0128] In addition, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to perform the following steps:

[0129] Obtain original video material and perform batch extraction of video material frame images;

[0130] Use the pre-trained image segmentation model to segment the foreground and background of each video frame image to obtain the corresponding target foreground image;

[0131] Get the target background image;

[0132] Generate a target video material frame image according to each of the target foreground images and the target background images;

[0133] The target video material frame images are integrated and processed to obtain the target video material.

[0134] In one embodiment, optionally, obtaining the original video material includes:

[0135] Get the original video material stored locally; or

[0136] Get the original video footage shot in real time.

[0137] In one embodiment, optionally, a pre-trained image segmentation model is used to segment the foreground and background of each video material frame image to obtain a corresponding target foreground image, including:

[0138] Loading a pre-trained image segmentation model and outputting prompt information for selecting a target foreground image, wherein the image segmentation model includes a SAM image segmentation model;

[0139] receiving target foreground prompt information input by the user according to the prompt information;

[0140] According to the target foreground prompt information, foreground and background segmentation is performed on each video material frame image to obtain a target foreground image corresponding to the target foreground prompt information;

[0141] The target foreground images are arranged and saved according to the video material frame sequence numbers.

[0142] In one embodiment, optionally, obtaining a target background image includes:

[0143] Receive a target background image input by the user; or

[0144] A preset video image is acquired, and the target background image is extracted from the preset video image according to a target background image extraction command.

[0145] In one embodiment, optionally, generating a target video material frame image according to each of the target foreground images and the target background images includes:

[0146] Preprocessing the target background image so that the size of the preprocessed target background image is the same as that of the video material frame image;

[0147] Each of the target foreground images and the target background image is fused to generate the target video material frame image.

[0148] In one embodiment, optionally, the target video material frame images are integrated to obtain the target video material, including:

[0149] According to the video material frame sequence number, all target video material frame images are integrated and processed to obtain the target video material.

[0150] In one embodiment, optionally, when generating a target video material frame image according to each of the target foreground image and the target background image, the method further includes:

[0151] Performing edge smoothing processing on each of the target foreground images to obtain a processed target foreground image;

[0152] Generating an initial video material frame image according to each of the processed target foreground images and target background images;

[0153] Parameters of the initial video material frame image are adjusted to obtain the target video material frame image, wherein the parameter adjustment includes at least one of the following: color adjustment and illumination adjustment.

[0154] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or electronic device can refer to the relevant description in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0155] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0156] It should be understood that, although the terms first, second, etc. may be used to describe the setting unit in the embodiments of the present application, these setting units should not be limited to these terms. These terms are only used to distinguish the setting units from each other. For example, without departing from the scope of the embodiments of the present application, the first setting unit may also be referred to as the second setting unit, and similarly, the second setting unit may also be referred to as the first setting unit.

[0157] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0158] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0160] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0161] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for replacing the background of a video material, characterized in that: The method comprises: Obtain original video material and perform batch extraction of video material frame images; Use the pre-trained image segmentation model to segment the foreground and background of each video frame image to obtain the corresponding target foreground image; Get the target background image; Generate a target video material frame image according to each of the target foreground images and the target background images; The target video material frame images are integrated and processed to obtain the target video material.

2. The video material background replacement method according to claim 1, characterized in that: The obtaining of the original video material comprises: Get the original video material stored locally; or Get the original video footage shot in real time.

3. The video material background replacement method according to claim 1, characterized in that: Use the pre-trained image segmentation model to segment the foreground and background of each video frame image to obtain the corresponding target foreground image, including: Loading a pre-trained image segmentation model and outputting prompt information for selecting a target foreground image, wherein the image segmentation model includes a SAM image segmentation model; receiving target foreground prompt information input by the user according to the prompt information; According to the target foreground prompt information, foreground and background segmentation is performed on each video material frame image to obtain a target foreground image corresponding to the target foreground prompt information; The target foreground images are arranged and saved according to the video material frame sequence numbers.

4. The video material background replacement method according to claim 1, characterized in that: Get the target background image, including: Receive a target background image input by the user; or A preset video image is acquired, and the target background image is extracted from the preset video image according to a target background image extraction command.

5. The video material background replacement method according to claim 1, characterized in that: Generating a target video material frame image according to each of the target foreground images and the target background images, including: Preprocessing the target background image so that the size of the preprocessed target background image is the same as that of the video material frame image; Each of the target foreground images and the target background image is fused to generate the target video material frame image.

6. The video material background replacement method according to claim 3, characterized in that: The target video material frame images are integrated and processed to obtain the target video material, including: According to the video material frame sequence number, all target video material frame images are integrated and processed to obtain the target video material.

7. The video material background replacement method according to claim 1, characterized in that: When generating a target video material frame image according to each of the target foreground images and the target background images, it also includes: Performing edge smoothing processing on each of the target foreground images to obtain a processed target foreground image; Generating an initial video material frame image according to each of the processed target foreground images and target background images; Parameters of the initial video material frame image are adjusted to obtain the target video material frame image, wherein the parameter adjustment includes at least one of the following: color adjustment and illumination adjustment.

8. A video material background replacement device, characterized in that: include: The extraction module is used to obtain the original video material and perform batch extraction of video material frame images; A segmentation module is used to segment the foreground and background of each video material frame image using a pre-trained image segmentation model to obtain a corresponding target foreground image; An acquisition module, used for acquiring a target background image; A generating module, used for generating a target video material frame image according to each of the target foreground image and the target background image; The integration module is used to integrate the target video material frame images to obtain the target video material.

9. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the method process according to any one of claims 1 to 7.

Images