Background replacement method, apparatus, computer equipment, and storage medium

Abstract

This application relates to a background replacement method, apparatus, computer equipment, storage medium, and computer program product, which can be used in the fintech field or other related fields to improve the efficiency of background replacement in face-to-face interview videos. The method includes: performing target detection processing on a first face-to-face interview video to obtain a first background region in the first face-to-face interview video; displaying background setting prompt information on the screen of the first face-to-face interview video when the first background region does not contain a physical background; performing target detection processing on a second face-to-face interview video to obtain a second background region in the second face-to-face interview video; the second face-to-face interview video is the face-to-face interview video obtained after displaying the background prompt information and at a preset time interval; and, when the second background region does not contain either a physical background or a virtual background, acquiring a target virtual background image and replacing the second background image in the second face-to-face interview video with the target virtual background image to obtain the target face-to-face interview video of the second face-to-face interview video.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a background replacement method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] Financial institutions such as bank branches can provide video interview services for customers who are unable to come to the branch in person, such as loan interviews, through their branch agents (i.e., customer service personnel). To provide more professional and rigorous remote interview services, a physical background wall needs to be set up in the remote agent center to prevent irrelevant backgrounds from entering the video.

[0003] In existing technologies, the backdrop placed behind the seats is a physical backdrop, such as a standard whiteboard, promotional video, or sales area notice board. These physical backdrops require designers to first design a background image, then create a physical backdrop based on the image, and place it behind the seats. This process is relatively cumbersome, resulting in low efficiency in replacing the background in face-to-face interview videos. Summary of the Invention

[0004] Therefore, it is necessary to provide a background replacement method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of background replacement in face-to-face interview videos, in order to address the aforementioned technical problems.

[0005] Firstly, this application provides a background replacement method. The method includes:

[0006] Target detection processing is performed on the first interview video to obtain the first background region in the first interview video;

[0007] If the first background area does not contain a physical background, a background setting prompt message will be displayed on the screen of the first interview video.

[0008] The second interview video is subjected to target detection processing to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval;

[0009] In the case where the second background area does not contain the physical background and the virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video.

[0010] In one embodiment, when the first background area does not contain a physical background, before displaying the background setting prompt information on the screen of the first interview video, the method further includes:

[0011] Obtain the entity background features of the entity background image of the entity background, and the first background features of the first background region;

[0012] Determine the feature similarity between the first background feature and the entity background feature;

[0013] Based on the feature similarity, it is determined whether the first background region contains the entity background.

[0014] In one embodiment, the background setting prompt information includes a first prompt information and a second prompt information; the first prompt information is used to prompt the agent account to set the physical background; the second prompt information is used to prompt the agent account to set the virtual background;

[0015] When the first background area does not contain a physical background, displaying background setting prompts on the screen of the first interview video includes:

[0016] If the feature similarity is less than a preset similarity threshold, it is determined that the first background region does not contain an entity background.

[0017] The first and second prompt messages are triggered and displayed on the screen of the first interview video.

[0018] In one embodiment, after the first and second prompt messages are displayed on the screen of the first interview video, the method further includes:

[0019] In response to the agent account's triggering operation on the second prompt message, multiple candidate virtual backgrounds are displayed on the screen of the first face-to-face video for the agent account to select from;

[0020] The first interview video is subjected to target segmentation processing to obtain a first background image in the first interview video; the background segmentation accuracy of the first background image is higher than that of the first background region.

[0021] Replace the first background image with the candidate virtual background selected by the agent account.

[0022] In one embodiment, after the first and second prompt messages are displayed on the screen of the first interview video, the method further includes:

[0023] In response to the agent account's triggering operation on the second prompt information, obtain the target account information of the target account associated with the first face-to-face video;

[0024] The target account information is subjected to feature mapping processing to obtain the account features corresponding to the target account information;

[0025] By using a virtual background generation model, the account features are fused to obtain a matching virtual background image that matches the target account information;

[0026] The first background image in the first interview video is replaced with the matching virtual background image; the first background image is obtained by performing target segmentation processing on the first interview video.

[0027] In one embodiment, after performing target detection processing on the second interview video to obtain the second background region in the second interview video, the method further includes:

[0028] The second background region is subjected to feature extraction processing to obtain the second background features of the second background region;

[0029] Determine the entity feature similarity between the second background feature and the entity background feature, and the virtual feature similarity between the second background feature and the virtual background feature of the virtual background image;

[0030] If the entity similarity does not reach a preset entity similarity threshold and the virtual feature similarity does not reach a preset virtual similarity threshold, it is confirmed that the second background region does not contain the entity background or the virtual background.

[0031] In one embodiment, acquiring the target virtual background image includes:

[0032] Determine the recommendation level for each virtual background image in the virtual background library;

[0033] Based on the recommendation level of each virtual background image, target virtual background images that meet the preset recommendation level conditions are selected from the virtual background library.

[0034] Secondly, this application also provides a background replacement device. The device includes:

[0035] The first target detection module is used to perform target detection processing on the first interview video to obtain the first background region in the first interview video.

[0036] The prompt message setting module is used to display background setting prompt messages on the screen of the first interview video when the first background area does not contain a physical background.

[0037] The second target detection module is used to perform target detection processing on the second interview video to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval;

[0038] The background image replacement module is used to obtain a target virtual background image when the second background area does not contain the physical background and the virtual background, and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video.

[0039] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0040] Target detection processing is performed on the first interview video to obtain the first background region in the first interview video;

[0041] If the first background area does not contain a physical background, a background setting prompt message will be displayed on the screen of the first interview video.

[0042] The second interview video is subjected to target detection processing to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval;

[0043] In the case where the second background area does not contain the physical background and the virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video.

[0044] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0045] Target detection processing is performed on the first interview video to obtain the first background region in the first interview video;

[0046] If the first background area does not contain a physical background, a background setting prompt message will be displayed on the screen of the first interview video.

[0047] The second interview video is subjected to target detection processing to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval;

[0048] In the case where the second background area does not contain the physical background and the virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video.

[0049] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0050] Target detection processing is performed on the first interview video to obtain the first background region in the first interview video;

[0051] If the first background area does not contain a physical background, a background setting prompt message will be displayed on the screen of the first interview video.

[0052] The second interview video is subjected to target detection processing to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval;

[0053] In the case where the second background area does not contain the physical background and the virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video.

[0054] The aforementioned background replacement method, apparatus, computer equipment, storage medium, and computer program product perform target detection processing on a first interview video to obtain a first background area in the first interview video; when the first background area does not contain a physical background, display background setting prompt information on the screen of the first interview video; perform target detection processing on a second interview video to obtain a second background area in the second interview video; wherein the second interview video is an interview video obtained after displaying the background prompt information and at a preset time interval; when the second background area does not contain a physical background or a virtual background, acquire a target virtual background image, and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video. This method can display background prompts in the interview video when the background area of ​​the interview video does not contain a preset physical background, reminding the agent to add a preset physical background or replace it with a virtual background. If the background area of ​​the interview video does not contain a preset physical background again and no virtual background replacement is used, the replacement of the target virtual background image is forcibly started. Compared with the existing technology that manually creates a physical background wall, this method can greatly simplify the background generation process and thus improve the background replacement efficiency of the interview video. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a background replacement method in one embodiment;

[0056] Figure 2 This is a schematic diagram of a face-to-face interview video in one embodiment;

[0057] Figure 3 This is a flowchart illustrating the step of replacing the first background image in a first interview video with a matching virtual background image in one embodiment.

[0058] Figure 4 This is a flowchart illustrating the background replacement method in another embodiment;

[0059] Figure 5 This is a structural block diagram of a background replacement device in one embodiment;

[0060] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the background replacement method and apparatus provided in this application can be used in the field of fintech to improve the efficiency of background replacement in face-to-face interview videos. They can also be used in any field other than fintech for background replacement applications, such as the field of computer technology. This application does not limit the application field of the background replacement method and apparatus.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, agent accounts, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0064] In one embodiment, such as Figure 1 As shown, a background replacement method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0065] Step S101: Perform target detection processing on the first interview video to obtain the first background region in the first interview video.

[0066] The first interview video refers to the initial (original) interview video obtained at the beginning. The first background area refers to the background area detected from the first interview video.

[0067] Specifically, at the start of the face-to-face interview service, the terminal acquires the first face-to-face interview video from the current interface; this first face-to-face interview video may be displayed as a pop-up window floating on the terminal's interface. The terminal extracts the agent preview screen from the first face-to-face interview video and performs object detection processing on the agent preview screen. This can be done by using an object detection model to identify pixels in the agent preview screen that are categorized as background (or human image), and using detection boxes to mark the boundaries of pixels categorized as background. Thus, the terminal obtains the first background area in the agent preview screen.

[0068] Figure 2 This is an illustration of a face-to-face interview video, such as... Figure 2 As shown, the face-to-face interview video mainly consists of two parts: the agent preview screen and the client preview screen. To provide a more professional and rigorous remote face-to-face interview service, a physical background can be set behind the agent to prevent irrelevant backgrounds from entering the video; if no physical background wall is set, a virtual background can be generated in the face-to-face interview video via the terminal. The agent preview screen refers to the video screen when the agent account is providing the face-to-face interview service; it displays a physical background (such as a background wall) and the agent's seat. The client preview screen refers to the video screen when the client account is providing the face-to-face interview service. It is understood that either preview screen can be selected as the main screen displayed on the terminal, while the other preview screen defaults to a smaller secondary screen, for example... Figure 2 The image shows the agent preview screen as the main screen and the customer preview screen as the secondary screen.

[0069] In step S102, if the first background area does not contain a physical background, display background setting prompts on the screen of the first interview video.

[0070] A physical backdrop refers to a background wall placed behind agents according to the regulations of banks and other financial institutions. This backdrop can be a whiteboard, promotional video, or a sales area sign. Creating a physical backdrop requires designers to first design a background image, then create a physical backdrop based on that image, and finally place it behind the agents; this process is relatively complex.

[0071] Specifically, the terminal determines whether the first background area contains a physical background. If the first background area contains a physical background, the terminal prompts the agent account to begin the face-to-face interview service. If the first background area does not contain a physical background, the terminal displays background setting prompts on the current screen of the first interview video for the agent account to view. These background setting prompts are information used to guide the agent account to set a virtual or physical background.

[0072] Step S103: Perform target detection processing on the second interview video to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying background prompt information and at a preset time interval.

[0073] The preset time refers to the time interval set for the collection of the second interview video.

[0074] Specifically, both the second and first interview videos are for the interview stage, but they correspond to different time points. After displaying background setting prompts on the first interview video, the terminal will wait a preset time interval before acquiring the second interview video. The terminal also performs object detection processing on the second interview video. This can be done by using an object detection model to identify pixels in the second interview video that are categorized as background (or human image), and using detection boxes to mark the boundaries of pixels categorized as background. In this way, the terminal obtains the second background region in the second interview video.

[0075] Step S104: In the case that the second background area does not contain a physical background or a virtual background, obtain the target virtual background image and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video.

[0076] The target virtual background image refers to the virtual background image actively generated by the terminal for the second interview video, used to replace the background. The target interview video refers to the interview video obtained after the terminal actively performs background replacement processing on the second interview video; the background in the target interview video is a virtual background.

[0077] Specifically, the terminal detects whether the second background area contains both a physical background and a virtual background. If the second background area does not contain either a physical background or a virtual background, the terminal actively performs background replacement processing on the second interview video. This can be done by first obtaining the target virtual background image for replacement, then segmenting the second background image to be replaced from the second interview video, and then replacing the second background image in the second interview video with the target virtual background image. In this way, the terminal obtains the target interview video of the second interview video.

[0078] In the aforementioned background replacement method, target detection processing is performed on the first interview video to obtain a first background area in the first interview video; if the first background area does not contain a physical background, a background setting prompt message is displayed on the screen of the first interview video; target detection processing is performed on the second interview video to obtain a second background area in the second interview video; wherein the second interview video is the interview video obtained after displaying the background prompt message and at a preset time interval; if the second background area does not contain either a physical background or a virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video. Using this method, when it is detected that the background area of ​​the interview video does not contain a preset physical background, a background prompt message is displayed in the interview video to remind the agent to add a preset physical background or use a virtual background replacement. If it is detected again that the background area of ​​the interview video does not contain a preset physical background and no virtual background replacement is used, the replacement of the target virtual background image is forcibly initiated. Compared with the existing technology that manually creates a physical background wall, this method greatly simplifies the background generation process, thereby improving the background replacement efficiency of the interview video.

[0079] In one embodiment, before displaying the background setting prompt information on the screen of the first interview video in step S102, when the first background area does not contain a physical background, the method further includes: obtaining the physical background features of the physical background image of the physical background and the first background features of the first background area; determining the feature similarity between the first background features and the physical background features; and determining whether the first background area contains a physical background based on the feature similarity.

[0080] The number of entity backgrounds can be one or more, and the number of entity background images can also be one or more.

[0081] Specifically, the terminal acquires an entity background image and performs feature extraction processing on the entity background image to obtain entity background features. The terminal can also perform feature extraction processing on a first background region to obtain first background features for the first background region. The terminal calculates the similarity between the first background features and the entity background features to obtain the feature similarity between the two features. The terminal uses the feature similarity to determine whether the first background region contains an entity background, for example, whether the feature similarity reaches a preset similarity threshold.

[0082] In this embodiment, the feature similarity between the first background feature of the first background region and the entity background feature of the entity background image is first determined. Then, based on the feature similarity, it can be determined whether the first background region contains an entity background, thus achieving accurate judgment of the entity background in the first interview video.

[0083] In one embodiment, the background setting prompt information includes a first prompt information and a second prompt information; the first prompt information is used to prompt the agent account to set a physical background; the second prompt information is used to prompt the agent account to set a virtual background. Step S102, where the first background area does not contain a physical background, displays the background setting prompt information on the screen of the first interview video, specifically includes the following: if the feature similarity is less than a preset similarity threshold, it is determined that the first background area does not contain a physical background; the first and second prompt information are then triggered to be displayed on the screen of the first interview video.

[0084] Among them, the seat account refers to the account used by the agent participating in the face-to-face interview service during the face-to-face interview video.

[0085] Specifically, when the feature similarity is greater than or equal to a preset similarity threshold, the terminal confirms that the first background area of ​​the agent preview screen contains a physical background. When the feature similarity is less than the preset similarity threshold, the terminal confirms that the first background area does not contain a physical background, and then triggers the terminal to display a prompt box on the screen of the first interview video. The prompt box includes a first prompt message and a second prompt message. The first prompt message is used to remind the agent account to set a preset physical background, such as "Please add a sales area notice board"; the second prompt message is used to remind the agent account whether to set a virtual background.

[0086] In this embodiment, if the feature similarity is less than a preset similarity threshold, it can be determined that the first background area does not contain an entity background, thereby triggering the terminal to display the first and second prompt information on the screen of the first interview video, so that the agent account can view the prompt information and perform corresponding operations according to the prompt information.

[0087] In one embodiment, after the first and second prompt messages are displayed on the screen of the first interview video, the method further includes: in response to the agent account's triggering operation on the second prompt message, displaying multiple candidate virtual backgrounds on the screen of the first interview video for the agent account to select; performing target segmentation processing on the first interview video to obtain a first background image in the first interview video; the background segmentation accuracy of the first background image is higher than that of the first background region; and replacing the first background image with the candidate virtual background selected by the agent account.

[0088] Among them, candidate virtual backgrounds refer to virtual backgrounds that are pre-made for agent accounts to choose from.

[0089] Specifically, after the terminal displays the first and second prompt messages on the first interview video screen, the agent account can choose to close the prompt boxes for the first and second prompt messages, or trigger an operation on the second prompt message within the prompt box. The terminal receives and responds to the agent account's trigger operation on the second prompt message, displaying multiple candidate virtual backgrounds on the first interview video screen, such as candidate virtual background 1, candidate virtual background 2, ..., candidate virtual background n, for the agent account to select. After the agent account selects a candidate virtual background, they can click on the selected candidate virtual background, and the terminal obtains the candidate virtual background selected by the agent account. The terminal performs target segmentation processing on the first interview video to obtain the first background image in the first interview video; then, it constructs a replacement relationship between the first background image and the candidate virtual background selected by the agent account; finally, using this replacement relationship, it replaces the first background image with the candidate virtual background selected by the agent account, that is, it replaces the part of the original background in the first interview video with the selected candidate virtual background.

[0090] It is understood that object detection in this disclosure identifies the approximate area of ​​the background in the seat preview screen, while object segmentation requires accurately segmenting the background part in the seat preview screen. Therefore, the background segmentation accuracy of the first background image is higher than that of the first background area.

[0091] Furthermore, the agent account can also ignore the first and second prompts displayed on the first interview video, and continue to maintain the original background of the first interview video without making any changes.

[0092] In this embodiment, the first interview video is first segmented to obtain the first background image in the first interview video; then the first background image is replaced with the candidate virtual background selected by the agent account, thus realizing the background image replacement of the interview video without the need for manual replacement of the physical background, thereby improving the background replacement efficiency of the interview video.

[0093] In one embodiment, such as Figure 3 As shown, after the first and second prompt messages are displayed on the screen of the first interview video, the following also applies:

[0094] Step S301: In response to the agent account's triggering operation on the second prompt information, obtain the target account information of the target account associated with the first face-to-face video.

[0095] The target account can be a customer service representative account or a client account participating in the face-to-face interview; the target account information can be either the account information of a customer service representative account or the account information of a client account.

[0096] Specifically, after the terminal displays the first and second prompt messages on the screen of the first interview video, the agent account can choose to close the prompt boxes for the first and second prompt messages, or trigger an operation on the second prompt message within the prompt box. The terminal receives and responds to the agent account's trigger operation on the second prompt message, and retrieves the target account information associated with the target account of the first interview video.

[0097] Step S302: Perform feature mapping processing on the target account information to obtain the account features corresponding to the target account information.

[0098] Specifically, the terminal performs feature mapping processing on the target account information, and then the terminal obtains the account features corresponding to the target account information.

[0099] Feature mapping can employ various mapping algorithms, such as word vector algorithms and One-Hot encoding, and account features can be represented by vectors.

[0100] Step S303: The account features are fused using a virtual background generation model to obtain a matching virtual background image that matches the target account information.

[0101] Among them, the matching virtual background image refers to the virtual background image generated based on the account characteristics of the target account information.

[0102] Specifically, the terminal inputs the obtained account features into a pre-trained virtual background generation model to fuse multiple account features through the virtual background generation model. The fused features can also be mapped to corresponding virtual background images, so that the terminal obtains a matching virtual background image that matches the target account information.

[0103] Step S304: Replace the first background image in the first interview video with a matching virtual background image; the first background image is obtained by performing target segmentation processing on the first interview video.

[0104] Specifically, the terminal performs target segmentation processing on the first interview video to obtain the first background image in the first interview video; then it constructs a replacement relationship between the first background image and the matching virtual background image; finally, it uses this replacement relationship to replace the first background image with the matching virtual background image, that is, it replaces the part of the original background in the first interview video with the matching virtual background image.

[0105] In this embodiment, by performing feature mapping processing on the target account information, the account features corresponding to the target account information are obtained; by using a virtual background generation model, the account features are fused to obtain a matching virtual background image that matches the target account information; thus, the first background image in the first interview video can be replaced with the matching virtual background image. By using account features, another way of replacing the background image in the interview video is realized, which not only improves the efficiency of background replacement in the interview video, but also makes the obtained matching virtual background image more in line with the preferences of the target account information, realizing personalized customization of the virtual background.

[0106] In one embodiment, after performing target detection processing on the second interview video to obtain the second background region in the second interview video in step S103, the method further includes: performing feature extraction processing on the second background region to obtain the second background features of the second background region; determining the entity feature similarity between the second background features and the entity background features, and the virtual feature similarity between the second background features and the virtual background features of the virtual background image; and confirming that the second background region does not contain entity background or virtual background if the entity similarity does not reach a preset entity similarity threshold and the virtual feature similarity does not reach a preset virtual similarity threshold.

[0107] Virtual background images refer to background images synthesized using computer technology.

[0108] Specifically, the terminal performs feature extraction processing on the second background region to obtain the second background features of the second background region; the terminal can also obtain a virtual background image from a virtual background library, perform feature extraction processing on the virtual background image to obtain the virtual background features of the virtual background image. The terminal calculates the similarity between the second background features and the entity background features to obtain the entity feature similarity between the second background features and the entity background features. In addition, the terminal calculates the similarity between the second background features and the virtual background features to obtain the virtual feature similarity between the second background features and the virtual background features. Then, the terminal determines whether the entity similarity reaches a preset entity similarity threshold and whether the virtual feature similarity reaches a preset virtual similarity threshold. If the entity similarity does not reach the preset entity similarity threshold and the virtual feature similarity does not reach the preset virtual similarity threshold, it can be confirmed that the second background region does not contain entity background or virtual background.

[0109] In this embodiment, by using the similarity of the entity features between the second background feature and the entity background feature, and the similarity of the virtual features between the second background feature and the virtual background feature of the virtual background image, it is possible to determine whether the second background area contains an entity background or a virtual background. This allows the terminal to determine whether to actively enable the virtual background replacement function, ensuring the compliance of the face-to-face video and saving the agent account time for subsequent virtual background replacement, thereby improving the background replacement efficiency of the face-to-face video.

[0110] In one embodiment, step S104, obtaining the target virtual background image, specifically includes the following: determining the recommendation level of each virtual background image in the virtual background library; and selecting target virtual background images that meet the preset recommendation level conditions from the virtual background library based on the recommendation level of each virtual background image.

[0111] Among them, the recommendation score is used to measure the degree to which virtual background images in the virtual background library are recommended.

[0112] Specifically, the virtual background library stores multiple pre-generated virtual background images. The terminal can determine the recommendation level of each virtual background image in the virtual background library based on the account characteristics of the target account information. Then, based on the recommendation level of each virtual background image, the terminal selects the target virtual background image from the virtual background library that meets the preset recommendation level condition. The preset recommendation level condition can be that the virtual background image with the highest recommendation level is selected as the target virtual background image.

[0113] Furthermore, the terminal can perform feature mapping processing on the target account information to obtain the account features corresponding to the target account information; through a virtual background generation model, the account features are fused to obtain a matching virtual background image that matches the target account information; and then the matching virtual background image is used as the target virtual background image.

[0114] In this embodiment, by selecting target virtual background images that meet the preset recommendation criteria from the virtual background library based on the recommendation level of each virtual background image, the target virtual background image for replacement can be obtained conveniently and quickly, greatly simplifying the background generation process and improving the efficiency of background replacement in face-to-face interview videos.

[0115] In one embodiment, such as Figure 4 As shown, another background replacement method is provided. Taking the application of this method to a terminal as an example, the steps include:

[0116] Step S401: Perform target detection processing on the first interview video to obtain the first background region in the first interview video.

[0117] Step S402: Obtain the entity background features of the entity background image and the first background features of the first background region; determine the feature similarity between the first background features and the entity background features.

[0118] Step S403: Determine whether the first background region contains an entity background based on feature similarity.

[0119] Step S404: If the feature similarity is less than a preset similarity threshold, determine that the first background area does not contain an entity background; trigger the first prompt message and the second prompt message to be displayed on the screen of the first interview video.

[0120] In step S405, in response to the agent account's triggering operation on the second prompt message, multiple candidate virtual backgrounds are displayed on the screen of the first interview video for the agent account to select.

[0121] Step S406: Perform target segmentation processing on the first interview video to obtain the first background image in the first interview video; the background segmentation accuracy of the first background image is higher than that of the first background region; replace the first background image with the candidate virtual background selected by the agent account.

[0122] Step S407: In response to the agent account's triggering operation on the second prompt information, obtain the target account information of the target account associated with the first face-to-face video; perform feature mapping processing on the target account information to obtain the account features corresponding to the target account information.

[0123] Step S408: The account features are fused using a virtual background generation model to obtain a matching virtual background image that matches the target account information; the first background image in the first interview video is replaced with the matching virtual background image.

[0124] The first background image is obtained by performing target segmentation processing on the first interview video.

[0125] It is understandable that after displaying multiple candidate virtual backgrounds on the screen of the first interview video, the replacement of the first background image in the first interview video can be completed through the above steps S405 to S406, or the replacement of the first background image in the first interview video can be completed through the above steps S407 to S408.

[0126] Step S409: Perform target detection processing on the second interview video to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying background prompt information and at a preset time interval.

[0127] Step S410: Perform feature extraction processing on the second background region to obtain the second background features of the second background region; determine the entity feature similarity between the second background features and the entity background features, and the virtual feature similarity between the second background features and the virtual background features of the virtual background image.

[0128] Step S411: If the entity similarity does not reach the preset entity similarity threshold and the virtual feature similarity does not reach the preset virtual similarity threshold, confirm that the second background region does not contain entity background and virtual background.

[0129] Step S412: In the case that the second background area does not contain a physical background or a virtual background, obtain the target virtual background image and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video.

[0130] The aforementioned background replacement method can achieve the following beneficial effects: when the background area of ​​the interview video does not contain a preset physical background, a background prompt message is displayed in the interview video to remind the agent to supplement the preset physical background or use a virtual background replacement. If the background area of ​​the interview video does not contain a preset physical background again and no virtual background replacement is used, the replacement of the target virtual background image is forcibly started. Compared with the existing technology of manually creating a physical background wall, this method can greatly simplify the background generation process and thus improve the background replacement efficiency of the interview video.

[0131] To more clearly illustrate the background replacement method provided in this disclosure, a specific embodiment is given below to describe the background replacement method in detail. Another background replacement method is also provided, which can be applied to a terminal, and specifically includes the following:

[0132] The terminal detects whether the background area of ​​the interview video contains a preset physical background. If the background area does not contain a preset physical background, a background setting prompt message is displayed in the interview video to remind the agent account to add a preset physical background or replace it with a virtual background. After a preset interval, the terminal checks again whether the interview video preview screen contains a preset physical background. If it does not contain a preset physical background, the terminal checks whether a virtual background is used. If a virtual background is not used, the terminal forcibly starts virtual background replacement to obtain the target interview video.

[0133] In this embodiment, the present invention achieves the following three effects: First, it uses an interactive method to replace the virtual background, eliminating the need for physical background replacement and improving background replacement efficiency. Second, it simplifies the virtual background generation process, thereby increasing the efficiency of virtual background generation. Third, it overcomes the drawback of high costs associated with producing physical backgrounds, saving significant production costs.

[0134] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0135] Based on the same inventive concept, this application also provides a background replacement apparatus for implementing the background replacement method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more background replacement apparatus embodiments provided below can be found in the limitations of the background replacement method described above, and will not be repeated here.

[0136] In one embodiment, such as Figure 5 As shown, a background replacement device 500 is provided, including: a first target detection module 501, a prompt information setting module 502, a second target detection module 503, and a background image replacement module 504, wherein:

[0137] The first target detection module 501 is used to perform target detection processing on the first interview video to obtain the first background area in the first interview video.

[0138] The prompt message setting module 502 is used to display background setting prompt messages on the screen of the first interview video when the first background area does not contain a physical background.

[0139] The second target detection module 503 is used to perform target detection processing on the second interview video to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying background prompt information and at a preset time interval.

[0140] Background image replacement module 504 is used to obtain a target virtual background image when the second background area does not contain a physical background or a virtual background, and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video.

[0141] In one embodiment, the background replacement device 500 further includes a first background determination module, configured to acquire entity background features of an entity background image of an entity background, and first background features of a first background region; determine the feature similarity between the first background features and the entity background features; and determine whether the first background region contains an entity background based on the feature similarity.

[0142] In one embodiment, the background setting prompt information includes a first prompt information and a second prompt information; the first prompt information is used to prompt the agent account to set a physical background; the second prompt information is used to prompt the agent account to set a virtual background; the prompt information setting module 502 is further used to determine that the first background area does not contain a physical background when the feature similarity is less than a preset similarity threshold; and to trigger the first prompt information and the second prompt information to be displayed on the screen of the first interview video.

[0143] In one embodiment, the background replacement device 500 further includes a first background replacement module, configured to, in response to a triggering operation of a second prompt message by an agent account, display multiple candidate virtual backgrounds on the screen of the first interview video for the agent account to select; perform target segmentation processing on the first interview video to obtain a first background image in the first interview video; the background segmentation accuracy of the first background image is higher than that of the first background region; and replace the first background image with the candidate virtual background selected by the agent account.

[0144] In one embodiment, the background replacement device 500 further includes a second background replacement module, configured to, in response to a triggering operation of a seat account on a second prompt message, obtain target account information of a target account associated with the first interview video; perform feature mapping processing on the target account information to obtain account features corresponding to the target account information; perform fusion processing on the account features through a virtual background generation model to obtain a matching virtual background image that matches the target account information; and replace the first background image in the first interview video with the matching virtual background image; the first background image is obtained by performing target segmentation processing on the first interview video.

[0145] In one embodiment, the background replacement device 500 further includes a second background judgment module, which is used to perform feature extraction processing on the second background region to obtain the second background features of the second background region; determine the entity feature similarity between the second background features and the entity background features, and the virtual feature similarity between the second background features and the virtual background features of the virtual background image; and confirm that the second background region does not contain entity background and virtual background if the entity similarity does not reach a preset entity similarity threshold and the virtual feature similarity does not reach a preset virtual similarity threshold.

[0146] In one embodiment, the background image replacement module 504 is further configured to determine the recommendation level of each virtual background image in the virtual background library; and to select target virtual background images that meet the preset recommendation level conditions from the virtual background library based on the recommendation level of each virtual background image.

[0147] Each module in the aforementioned background replacement device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0148] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a background replacement method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0149] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0154] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A background replacement method, characterized in that, The method includes: Target detection processing is performed on the first interview video to obtain the first background region in the first interview video; If the first background area does not contain a physical background, a background setting prompt message will be displayed on the screen of the first interview video. The second interview video is subjected to target detection processing to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval; In the case where the second background area does not contain the physical background and the virtual background, a target virtual background image is obtained, and the second background image in the second interview video is replaced with the target virtual background image to obtain the target interview video of the second interview video.

2. The method according to claim 1, characterized in that, If the first background area does not contain a physical background, before displaying the background setting prompt information on the screen of the first interview video, the method further includes: Obtain the entity background features of the entity background image of the entity background, and the first background features of the first background region; Determine the feature similarity between the first background feature and the entity background feature; Based on the feature similarity, it is determined whether the first background region contains the entity background.

3. The method according to claim 2, characterized in that, The background setting prompt information includes a first prompt information and a second prompt information; the first prompt information is used to prompt the agent account to set the physical background; the second prompt information is used to prompt the agent account to set the virtual background; When the first background area does not contain a physical background, displaying background setting prompts on the screen of the first interview video includes: If the feature similarity is less than a preset similarity threshold, it is determined that the first background region does not contain an entity background. The first and second prompt messages are triggered and displayed on the screen of the first interview video.

4. The method according to claim 3, characterized in that, After the first and second prompt messages are displayed on the screen of the first interview video, the process further includes: In response to the agent account's triggering operation on the second prompt message, multiple candidate virtual backgrounds are displayed on the screen of the first face-to-face video for the agent account to select from; The first interview video is subjected to target segmentation processing to obtain a first background image in the first interview video; the background segmentation accuracy of the first background image is higher than that of the first background region. Replace the first background image with the candidate virtual background selected by the agent account.

5. The method according to claim 3, characterized in that, After the first and second prompt messages are displayed on the screen of the first interview video, the process further includes: In response to the agent account's triggering operation on the second prompt information, obtain the target account information of the target account associated with the first face-to-face video; The target account information is subjected to feature mapping processing to obtain the account features corresponding to the target account information; By using a virtual background generation model, the account features are fused to obtain a matching virtual background image that matches the target account information; The first background image in the first interview video is replaced with the matching virtual background image; the first background image is obtained by performing target segmentation processing on the first interview video.

6. The method according to claim 2, characterized in that, After performing target detection processing on the second interview video to obtain the second background region in the second interview video, the process further includes: The second background region is subjected to feature extraction processing to obtain the second background features of the second background region; Determine the entity feature similarity between the second background feature and the entity background feature, and the virtual feature similarity between the second background feature and the virtual background feature of the virtual background image; If the entity feature similarity does not reach a preset entity similarity threshold and the virtual feature similarity does not reach a preset virtual similarity threshold, it is confirmed that the second background region does not contain the entity background and the virtual background.

7. The method according to claim 1, characterized in that, The acquisition of the target virtual background image includes: Determine the recommendation level for each virtual background image in the virtual background library; Based on the recommendation level of each virtual background image, target virtual background images that meet the preset recommendation level conditions are selected from the virtual background library.

8. A background replacement device, characterized in that, The device includes: The first target detection module is used to perform target detection processing on the first interview video to obtain the first background region in the first interview video. The prompt message setting module is used to display background setting prompt messages on the screen of the first interview video when the first background area does not contain a physical background. The second target detection module is used to perform target detection processing on the second interview video to obtain the second background area in the second interview video; the second interview video is the interview video obtained after displaying the background prompt information and at a preset time interval; The background image replacement module is used to obtain a target virtual background image when the second background area does not contain the physical background and the virtual background, and replace the second background image in the second interview video with the target virtual background image to obtain the target interview video of the second interview video.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images