An image processing method, apparatus and electronic device

Abstract

The present application provides an image processing method, device and electronic equipment, comprising: determining that a background virtual replacement function is started, the background virtual replacement function separates a human body image from a current original image; identifying a handheld object image from the current original image; splicing the handheld object image into the human body image to obtain a spliced image, and displaying the spliced image. In the present scheme, the background virtual replacement function can separate the human body image from the current original image, and when it is determined that the background virtual replacement function is started, the current original image is separated to obtain the human body image. Moreover, the handheld object image identified from the current original image is spliced into the human body image to obtain a spliced image, and the spliced image is displayed. When the background virtual replacement function is started, the handheld object image in the current original image can be regarded as part of the human body image and will not be replaced as a background, thereby improving the demonstration effect.

Description

Technical Field

[0001] This application relates to the field of information technology, and more specifically, to an image processing method, apparatus, and electronic device. Background Technology

[0002] Current conferencing software has background blurring and replacement functions. When people participate in online meetings, they usually use background blurring or replacement functions to hide cluttered backgrounds. Specifically, the speaker in the video is identified as the foreground, and other objects, including objects held in the hands, are segmented as the background. Therefore, if you want to show the objects in your hands, the objects in your hands will be used as the background and replaced along with the background, making it impossible to demonstrate or explain. Summary of the Invention

[0003] In view of this, this application provides an image processing method, as follows:

[0004] An image processing method, comprising:

[0005] The background virtual replacement function is enabled, which segments the human body image from the current original image;

[0006] Identify the handheld object image from the current original image;

[0007] The image of the handheld object is stitched onto the image of the human body to obtain a stitched image;

[0008] The stitched image is displayed.

[0009] Optionally, in the above method, before identifying the handheld object image from the current original image, the method further includes:

[0010] Upper limb movements are identified based on the upper limb feature information contained in the current original image and the upper limb feature information contained in the historical original images;

[0011] The upper limb movement was determined to be a target movement.

[0012] Optionally, in the above method, displaying the stitched image includes:

[0013] Get the preset background image;

[0014] The stitched image and the preset background image are displayed synchronously, with the stitched image serving as the foreground image corresponding to the preset background image.

[0015] Optionally, the above method, which identifies upper limb movements based on upper limb feature information contained in the current original image and upper limb feature information contained in historical original images, includes:

[0016] Analysis of the current original image determines that it contains upper limb feature information;

[0017] The upper limb movement mode is determined based on a first region containing upper limb feature information in a historical original image and a second region containing upper limb feature information in the current original image.

[0018] Based on the agreed upper limb movement rules, the upper limb movements corresponding to the upper limb movement methods are determined.

[0019] Optionally, after analyzing the current original image to determine that it contains upper limb feature information, the above method further includes:

[0020] Based on the upper limb feature information contained in the current original image, the second region is determined, and the second region is a second region containing upper limb feature information;

[0021] Add at least two upper limb skeleton markers to the second region.

[0022] Optionally, in the above method, determining the upper limb movement mode based on a first region containing upper limb feature information in a historical original image and a second region containing upper limb feature information in the current original image includes:

[0023] The movement mode of the upper limb is determined based on at least two upper limb skeleton markers in a first region of the historical original image and at least two upper limb skeleton markers in a second region of the current original image.

[0024] Optionally, in the above method, identifying the handheld object image from the current original image includes:

[0025] By combining the upper limb movements with the analysis of the current original image and the historical original image, it is determined that the current original image contains an image of a handheld object, and the image of the handheld object changes synchronously with the upper limb movements in the historical original image and the current original image.

[0026] Optionally, the above method, combining the analysis of the upper limb movements with the current original image and historical original images, determines that the current original image contains an image of a handheld object, including:

[0027] Based on the analysis of the current original image and the historical original images, the change parameters of the target object image are obtained;

[0028] Determine that the changing parameters of the target object image match the upper limb movement;

[0029] The target object image is determined to be a handheld object image.

[0030] An image processing apparatus, comprising:

[0031] The determination module is used to determine whether the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0032] An object recognition module is used to recognize a handheld object image from the current original image;

[0033] The stitching module is used to stitch the image of the handheld object onto the human body image to obtain a stitched image;

[0034] The display module is used to display the stitched image.

[0035] An electronic device includes: a memory and a processor;

[0036] The memory stores the processing program;

[0037] The processor is used to load and execute the processing program stored in the memory to implement the steps of the image processing method as described in any of the preceding claims.

[0038] As can be seen from the above technical solution, this application provides an image processing method, including: determining that a background virtual replacement function is enabled, wherein the background virtual replacement function segments a human image from a current original image; identifying a handheld object image from the current original image; stitching the handheld object image to the human image to obtain a stitched image, and displaying the stitched image. In this solution, the background virtual replacement function can segment a human image from the current original image. When the background virtual replacement function is enabled, the human image is segmented from the current original image, and the handheld object image identified from the current original image is stitched to the human image to obtain a stitched image, which is then displayed. When the background virtual replacement function is enabled, the handheld object image in the current original image can be used as part of the human image and will not be replaced as background, thus improving the demonstration effect. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 This is a flowchart of an embodiment 1 of an information processing method provided in this application;

[0041] Figure 2 This is a flowchart of an embodiment 2 of an information processing method provided in this application;

[0042] Figure 3This is a flowchart of an embodiment 3 of an information processing method provided in this application;

[0043] Figure 4 This is a flowchart of embodiment 4 of the information processing method provided in this application;

[0044] Figure 5 This is a flowchart of embodiment 5 of the information processing method provided in this application;

[0045] Figure 6 This is a flowchart of an embodiment 6 of an information processing method provided in this application;

[0046] Figure 7 This is a schematic diagram of an embodiment of an information processing device provided in this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] like Figure 1 The diagram shown is a flowchart of an embodiment 1 of an information processing method provided in this application. The method is applied to an electronic device and includes the following steps:

[0049] Step S101: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0050] Image segmentation refers to the process of detecting and locating a specific target in an image as the foreground, and accurately segmenting it from the background of the image.

[0051] In this process, electronic devices need to process several consecutive frames of the original video image, with each frame being processed as the current original image.

[0052] The video can be a live recording or a rebroadcast.

[0053] Among them, the application in electronic devices has a background virtual replacement function. After the background virtual replacement function is enabled, it can segment the current original replacement into human images and non-human images.

[0054] S102: Identify the handheld object image from the current original image;

[0055] The background virtual replacement function can only perform preliminary segmentation. When a person in the current original image is holding an object, the held object will be segmented as a non-human image. Therefore, this solution further analyzes the current original image to determine whether there is an image region belonging to a held object.

[0056] In this process, the original image is identified to obtain an image of the handheld object.

[0057] The handheld object image can be either an image where both the hand and the object are present (with part of the hand being obscured by the object) or an image where only the object is present (with the hand being completely obscured by the object).

[0058] S103: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0059] When a handheld object image is identified in the current original image, the handheld object image is stitched together with the segmented human body image to obtain a stitched image.

[0060] The stitched image includes images of a human body and a handheld object.

[0061] S104: Display the stitched image.

[0062] The image is a composite image of the human body and the object being held, which is displayed to ensure that the human body and the object being held are displayed in their entirety and are not replaced as background during the display process.

[0063] Specifically, the edges of the stitched image are smoothed, such as by using a filter for rendering.

[0064] It should be noted that each frame of a video is treated as the current original image and processed in this scheme, so that the images of the human body and the objects it holds in the final output video are complete.

[0065] In summary, this application provides an image processing method, comprising: determining that a background virtual replacement function is enabled; the background virtual replacement function segmenting a human body image from a current original image; identifying a handheld object image from the current original image; stitching the handheld object image to the human body image to obtain a stitched image; and displaying the stitched image. In this solution, the background virtual replacement function can segment a human body image from the current original image. When the background virtual replacement function is enabled, the human body image is segmented from the current original image. Furthermore, the handheld object image identified from the current original image is stitched to the human body image to obtain a stitched image, which is then displayed. When the background virtual replacement function is enabled, the handheld object image in the current original image can be used as part of the human body image and will not be replaced as background, thus improving the presentation effect.

[0066] like Figure 2 The diagram shown is a flowchart of an embodiment 2 of an information processing method provided in this application. The method includes the following steps:

[0067] Step S201: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0068] Step S201 is the same as step S101 in Embodiment 1, and will not be described again in this embodiment.

[0069] Step S202: Identify upper limb movements based on the upper limb feature information contained in the current original image and the upper limb feature information contained in the historical original images;

[0070] In this process, upper limb feature information from each frame of the original image is combined to identify the upper limb movements in the original image.

[0071] Specifically, the upper limb feature information contained in the current original image of the video is combined with the upper limb feature information contained in the historical original images.

[0072] Among them, upper limb feature information in each frame of the original image is identified based on the upper limb feature recognition rules set in the preset model.

[0073] The upper limb includes multiple components such as the upper arm, forearm, wrist, and hand. Specifically, the hand includes the palm and fingers.

[0074] In this process, the corresponding upper limb movements are identified based on changes in upper limb feature information within a series of multiple original images.

[0075] It should be noted that since the object being held is a demonstration object held by a human hand in the original image, the upper limb movement can be a hand movement obtained based on the motion recognition of the hand and wrist images. When the hand and wrist are completely obscured by the object being held, the hand movement can also be inferred based on the movement of the forearm and upper arm.

[0076] Step S203: Determine that the upper limb movement belongs to the target movement;

[0077] This target action can be used as a trigger condition for subsequent recognition of handheld object images.

[0078] Specifically, it determines whether the upper limb movement belongs to the target movement, and if the identified upper limb movement belongs to the target movement, it executes subsequent steps.

[0079] The target action can be a specific action performed by the upper limb holding the demonstration object, or it can be a specific action performed by the upper limb not holding the demonstration object to introduce the handheld object.

[0080] The target action can be a specific action performed by the upper limb holding the demonstration object. The preset target actions include, but are not limited to, the following actions: drawing circles with the hand, moving the hand back and forth, and bringing the hand close to the screen.

[0081] Specifically, drawing circles with the hand means drawing circles around the demonstration object; moving the hand back and forth means shaking the demonstration object left and right / up and down; and bringing the hand close to the screen means bringing the demonstration object close to the screen.

[0082] When the upper limb movement is not part of the target movement, the person in the current original image has not performed the demonstration of the object. Therefore, there is no need to perform the subsequent step of identifying the handheld object from the current original image.

[0083] Specifically, when describing a specific action of holding an object, the preset target action includes, but is not limited to, the following actions: palm facing outwards, fingertips pointing downwards, index finger extended, and the other fingers bent into a fist, etc.

[0084] Moreover, when describing a specific action of holding an object, such as making the target action with the left hand and holding the object with the right hand.

[0085] Step S204: Identify the handheld object image from the current original image;

[0086] Step S205: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0087] Step S206: Display the stitched image.

[0088] Steps S204-206 are the same as steps S102-104 in Example 1, and will not be described again in this example.

[0089] In summary, this application provides an image processing method that further includes: identifying upper limb movements based on upper limb feature information contained in the current original image and upper limb feature information contained in historical original images; and determining that the upper limb movement belongs to a target movement. In this solution, the upper limb feature information contained in the current original image and the upper limb feature information contained in historical original images are combined to identify the upper limb movement. Only when the upper limb movement is determined to belong to the target movement is the image of the held object in the current original image identified. If the upper limb movement is determined not to belong to the target movement, it indicates that the person in the current original image is not performing the demonstration of the object, thus eliminating the need for held object image recognition and image stitching.

[0090] like Figure 3 The diagram shown is a flowchart of an embodiment 3 of an information processing method provided in this application. The method includes the following steps:

[0091] Step S301: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0092] Step S302: Identify the handheld object image from the current original image;

[0093] Step S303: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0094] Steps S301-303 are the same as steps S101-103 in Example 1, and will not be described again in this example.

[0095] Step S304: Obtain a preset background image;

[0096] In this context, electronic devices may have one or more background images configured for the background virtual replacement function.

[0097] In this process, after determining the stitched image that serves as the foreground in the current original image, the remaining portion is used as the background.

[0098] Specifically, a preset background image is obtained for the background, so as to blur or replace the background in the current original image.

[0099] If the background in the current original image is to be blurred, the preset background image can be the blurred background; if the background in the current original image is to be replaced, the preset background image can be another background image that has been set.

[0100] Step S305: Display the stitched image and the preset background image synchronously, with the stitched image serving as the foreground image corresponding to the preset background image.

[0101] The process involves smoothing the edges of the spliced ​​image, such as by using a filter for rendering; then, the background area is occluded, and the preset background image is used to replace and merge the background area to obtain the image with the replaced background.

[0102] Specifically, the stitched image is used as the foreground image of the preset background image, and the stitched image and the preset background image are displayed synchronously. This allows the human image and the handheld object image in the current original image to be used as the foreground image, and a virtual replacement background is displayed synchronously when the foreground image is displayed, thus demonstrating the handheld object.

[0103] In summary, this application provides an image processing method in which displaying the stitched image includes: acquiring a preset background image; and synchronously displaying the stitched image and the preset background image, wherein the stitched image serves as the foreground image corresponding to the preset background image. In this solution, by using the stitched image as the foreground image and the preset background image as the background, and displaying them synchronously, it ensures that the portion of the handheld object image in the current original image that is intended to be a human figure is not replaced as background, while simultaneously performing background virtual replacement on the current original image, thus improving the presentation effect.

[0104] like Figure 4 The diagram shown is a flowchart of an embodiment 4 of an information processing method provided in this application. The method includes the following steps:

[0105] Step S401: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0106] Step S401 is the same as step S201 in Embodiment 2, and will not be described again in this embodiment.

[0107] Step S402: Analyze the current original image to determine if it contains upper limb feature information;

[0108] Among them, upper limb feature information in the current original image is obtained by recognizing the upper limb feature based on the set upper limb feature recognition rules.

[0109] The upper limb includes multiple components such as the upper arm, forearm, wrist, and hand. Specifically, the hand includes the palm and fingers.

[0110] This refers to the current original image containing some or all of the upper limb components.

[0111] The current original image may include all of the above-mentioned upper limb parts, such as the upper arm, forearm, wrist, and hand; however, the hand only has fingers and the palm is obscured.

[0112] The current original image may contain only a portion of the aforementioned upper limb, such as only the upper arm and forearm, while the wrist and hand are obscured.

[0113] Step S403: Determine the upper limb movement mode based on the first region containing upper limb feature information in the historical original image and the second region containing upper limb feature information in the current original image;

[0114] In the video, the person holding and explaining the object moves their upper limbs while explaining it; and the person also moves their upper limbs when not explaining the object.

[0115] Specifically, in the historical original image, the portion containing upper limb feature information is identified in the first region; in the current original image, the portion containing upper limb feature information is identified in the second region.

[0116] In any consecutive multiple frames of images, the trajectory of upper limb movement is continuous. Therefore, in this scheme, the movement mode of the upper limb is obtained by analyzing the changes between the first region in the historical original image and the second region in the current original image.

[0117] For example, the movement of the upper limb can include moving left and right, moving forward and backward relative to the screen, moving up and down, or other movement methods.

[0118] Step S404: Based on the agreed upper limb movement rules, determine the upper limb movement corresponding to the upper limb movement mode;

[0119] Among them, the preset upper limb movement rules define the upper limb movements corresponding to the upper limb movement methods.

[0120] For example, if the area corresponding to the hand in the upper limb draws a circle around a specific area, the corresponding upper limb movement is a hand circling motion.

[0121] For example, when the area of ​​the upper limb corresponding to the hand moves left and right or up and down, the corresponding upper limb movement is a back-and-forth translational movement of the hand;

[0122] For example, the area of ​​the upper limb corresponding to the hand is close to the screen, and the corresponding upper limb movement is to bring the hand close to the screen.

[0123] Specifically, the movement of the upper limbs in multiple consecutive frames of original images is detected and tracked to determine the upper limb movements in the video.

[0124] Step S405: Determine that the upper limb movement belongs to the target movement;

[0125] Step S406: Identify the handheld object image from the current original image;

[0126] Step S407: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0127] Step S408: Display the stitched image.

[0128] Step S405 is the same as steps S203-206 in Example 2, and will not be described again in this example.

[0129] In summary, this application provides an image processing method for recognizing upper limb movements based on upper limb feature information contained in the current original image and upper limb feature information contained in historical original images. The method includes: analyzing the current original image to determine if it contains upper limb feature information; determining the upper limb movement pattern based on a first region containing upper limb feature information in the historical original image and a second region containing upper limb feature information in the current original image; and determining the upper limb movement corresponding to the upper limb movement pattern based on agreed-upon upper limb movement rules. In this solution, the movement pattern of the upper limb is detected and tracked in multiple consecutive frames of original images to determine the upper limb movement in the video, providing a basis for subsequent steps to determine whether the upper limb movement belongs to the target movement.

[0130] like Figure 5 The flowchart shown is a 5th embodiment of an information processing method provided in this application. The method includes the following steps:

[0131] Step S501: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0132] Step S502: Analyze the current original image to determine if it contains upper limb feature information;

[0133] Steps S501-502 are the same as steps S401-402 in Example 4, and will not be described again in this example.

[0134] Step S503: Based on the upper limb feature information contained in the current original image, determine the second region, wherein the second region is a second region containing upper limb feature information;

[0135] Specifically, the upper limb feature information contained in the current original image is analyzed, and the second region of the upper limb feature information in the current original image is determined.

[0136] The second region contains upper limb feature information.

[0137] Specifically, when the upper limb feature information obtained by the identification includes the hand, the second region can be the area where the upper arm, forearm, and hand are located;

[0138] Specifically, when the upper limb feature information obtained by the identification does not include the hand, the second region can be a region that includes the upper arm and forearm, and the second region also includes the region where the predicted hand is located. The predicted region is obtained based on the relative positional relationship between the upper arm and forearm.

[0139] Step S504: Add at least two upper limb skeleton markers to the second region;

[0140] Specifically, upper limb skeleton markers are added to the structures corresponding to the upper limb feature information in the second region.

[0141] Specifically, upper limb skeletal markings are set for different parts of the upper limb.

[0142] For example, add one or more upper limb skeleton markers to the upper arm, one or more upper limb skeleton markers to the forearm, and one or more upper limb skeleton markers to the hand.

[0143] It should be noted that, in the specific implementation, each frame of the original image being processed in the video is taken as the current original image. After adding the upper limb skeleton marker to the current original image, the next frame is taken as the current original image, and steps S502-504 are executed again to obtain multiple consecutive frames of original images in which the upper limb skeleton marker has been added.

[0144] Step S505: Based on at least two upper limb skeleton markers in the first region of the historical original image and at least two upper limb skeleton markers in the second region of the current original image, determine the movement mode of the upper limb;

[0145] Specifically, the upper limb skeleton markers added to the multiple consecutive original images are analyzed to determine the movement mode of the upper limb skeleton markers at the corresponding positions. By combining the movement mode of the upper limb skeleton markers at each position among the multiple upper limb skeleton markers, the overall movement mode of the upper limb is determined.

[0146] By monitoring the movement of upper limb skeletal markers at each location in multiple upper limb skeletons, the movement of the upper limb can be tracked to further determine the movement mode of the upper limb.

[0147] Step S506: Based on the agreed upper limb movement rules, determine the upper limb movement corresponding to the upper limb movement mode;

[0148] Step S507: Determine that the upper limb movement belongs to the target movement;

[0149] Step S508: Identify the handheld object image from the current original image;

[0150] Step S509: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0151] Step S510: Display the stitched image.

[0152] Steps S506-510 are the same as steps S404-408 in Example 4, and will not be described again in this example.

[0153] In summary, this application provides an image processing method in which, based on the upper limb feature information contained in the current original image, a second region is determined, wherein the second region is a region containing upper limb feature information; at least two upper limb skeleton markers are added to the second region; and the movement mode of the upper limb is determined based on at least two upper limb skeleton markers in the first region of the historical original image and at least two upper limb skeleton markers in the second region of the current original image. In this scheme, upper limb skeleton markers are added to the regions containing upper limb feature information in each frame of the original image in the video. Based on the upper limb skeleton markers in multiple consecutive frames of the original image, the movement mode of the upper limb skeleton markers at the same position is determined, and the movement of the upper limb is tracked to determine the movement mode of the upper limb.

[0154] like Figure 6 The flowchart shown is a sample of embodiment 6 of an information processing method provided in this application. The method includes the following steps:

[0155] Step S601: Determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0156] Step S602: Identify upper limb movements based on the upper limb feature information contained in the current original image and the upper limb feature information contained in the historical original images;

[0157] Step S603: Determine that the upper limb movement belongs to the target movement;

[0158] Steps S601-603 are the same as steps S201-203 in Example 2, and will not be described again in this example.

[0159] Step S604: Analyze the current original image and the historical original image in conjunction with the upper limb movements to determine that the current original image contains an image of a handheld object;

[0160] The image of the handheld object changes synchronously with the upper limb movements in both the historical original image and the current original image.

[0161] In this process, identifying the upper limb movement as a target movement indicates that the person in the image has performed the target movement. Further determining whether there is a handheld object in the original image that moves synchronously with the target movement will help determine whether there is a handheld object.

[0162] The analysis of the current original image and historical original images reveals that both images contain a certain object, but the size and position of this object may differ in different images.

[0163] Furthermore, determine whether the object has changed in different original images. If it has not changed, it can be determined that the object did not move synchronously with the upper limb of the person in the original image. If it has changed, further determine whether the object moves synchronously with the upper limb in the original image. If they move synchronously, it is determined that the object is a handheld object; otherwise, it is not.

[0164] Specifically, step S604 includes:

[0165] Step S6041: Analyze the change parameters of the target object image based on the current original image and the historical original images;

[0166] The target object is contained in both the current original image and the historical original images. The position and size of the target object in the original image are obtained by analyzing the current original image and the historical original images.

[0167] Then, by comparing the position and size of the target object in any two consecutive frames of the original image, the variation parameters of the target object image can be obtained.

[0168] Specifically, this analysis only needs to examine the change parameters of the target object image in the current original image and the previous historical original image. The change parameters of the target object image in multiple historical original images can be obtained based on the results of previous analysis.

[0169] These changing parameters include positional movement and size changes.

[0170] Step S6042: Determine whether the change parameters of the target object image match the upper limb movement;

[0171] The changing parameters of the target object image characterize the movement of the target object.

[0172] For example, moving the position left or right means that the target object moves left or right, and increasing the size means that the target object moves closer to the screen.

[0173] Specifically, the movement of the target object is compared with the upper limb movement identified in step S602. If they do not match, it indicates that the target object may only be obscuring the hand / upper limb of the person in the original image, but is not being held and moved together. When they match, it indicates that the target object is being held and moved together by the person in the original image.

[0174] Step S6043: Determine that the target object image is a handheld object image.

[0175] If the changing parameters of the target object image match the upper limb movement corresponding to the upper limb in the original image, the target object is determined to be a handheld object, and the target object image is a handheld object image.

[0176] Step S605: Stitch the image of the handheld object to the image of the human body to obtain a stitched image;

[0177] Step S606: Display the stitched image.

[0178] Steps S605-606 are the same as steps S205-206 in Example 2, and will not be described again in this example.

[0179] In summary, this application provides an image processing method in which the step of identifying a handheld object image from the current original image includes: analyzing the current original image and historical original images in conjunction with the upper limb movements to determine that the current original image contains a handheld object image, wherein the handheld object image changes synchronously with the upper limb movements in both the historical original images and the current original image. In this solution, based on the fact that the object is present in both original images and that the object moves synchronously with the upper limb movements in the original images, it is determined that the object is a handheld object, and the target object image in the current original image is determined to be a handheld object image.

[0180] Corresponding to the above-described embodiment of an image processing method provided in this application, this application also provides an embodiment of an apparatus for applying the image processing method.

[0181] like Figure 7 The diagram shown is a structural schematic of an embodiment of an image processing device provided in this application. The device includes the following components: a determining module 701, an identification module 702, a stitching module 703, and a display module 704.

[0182] The determining module 701 is used to determine that the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image;

[0183] The object recognition module 702 is used to recognize a handheld object image from the current original image;

[0184] The stitching module 703 is used to stitch the image of the handheld object into the image of the human body to obtain a stitched image;

[0185] The display module 704 is used to display the spliced ​​image.

[0186] Optional, also includes:

[0187] The action recognition module is used to recognize upper limb actions based on the upper limb feature information contained in the current original image and the upper limb feature information contained in the historical original images; and to determine that the upper limb action belongs to the target action.

[0188] Optionally, the display module is specifically used for:

[0189] Get the preset background image;

[0190] The stitched image and the preset background image are displayed synchronously, with the stitched image serving as the foreground image corresponding to the preset background image.

[0191] Optionally, the action recognition module includes:

[0192] The analysis unit is used to analyze the current raw image to determine if it contains upper limb feature information;

[0193] The movement mode determination unit is used to determine the upper limb movement mode based on a first region containing upper limb feature information in a historical original image and a second region containing upper limb feature information in the current original image;

[0194] The action determination unit is used to determine the upper limb action corresponding to the upper limb movement mode based on the agreed upper limb movement rules.

[0195] Optionally, the action recognition module further includes:

[0196] The region determination unit is used to determine the second region based on the upper limb feature information contained in the current original image, wherein the second region is a second region containing upper limb feature information;

[0197] A marker-adding unit is used to add at least two upper limb skeleton markers to the second region.

[0198] Optionally, the movement mode determination unit is specifically used for:

[0199] The movement mode of the upper limb is determined based on at least two upper limb skeleton markers in a first region of the historical original image and at least two upper limb skeleton markers in a second region of the current original image.

[0200] Optional, object recognition module, specifically used for:

[0201] By combining the upper limb movements with the analysis of the current original image and the historical original image, it is determined that the current original image contains an image of a handheld object, and the image of the handheld object changes synchronously with the upper limb movements in the historical original image and the current original image.

[0202] Optional, the object recognition module includes:

[0203] The change analysis unit is used to analyze and obtain the change parameters of the target object image based on the current original image and the historical original images;

[0204] A comparison unit is used to compare whether the changing parameters of the target object image match the upper limb movement;

[0205] The determining unit is used to determine that the target object image is a handheld object image based on the matching of the changing parameters of the target object image with the upper limb movement.

[0206] The functions of the image processing apparatus in this application are explained with reference to the method embodiments, and will not be described in detail in this embodiment.

[0207] In summary, this application provides an image processing apparatus in which a background virtual replacement function can segment a human body image from a current original image. When the background virtual replacement function is activated, the human body image is segmented from the current original image. Furthermore, the handheld object image identified from the current original image is stitched into the human body image to obtain a stitched image, which is then displayed. When the background virtual replacement function is enabled, the handheld object image in the current original image can be used as part of the human body image and will not be replaced as background, thus improving the presentation effect.

[0208] Corresponding to the above-described embodiment of an image processing method provided in this application, this application also provides an electronic device and a readable storage medium corresponding to the image processing method.

[0209] The electronic device includes: a memory and a processor;

[0210] The memory stores the processing program;

[0211] The processor is used to load and execute the processing program stored in the memory to implement the steps of the image processing method as described in any of the preceding claims.

[0212] For details on the specific image processing method implemented in this electronic device, please refer to the aforementioned image processing method embodiments.

[0213] The readable storage medium stores a computer program that is invoked and executed by a processor to implement the steps of the image processing method as described in any one of the preceding claims.

[0214] Specifically, the computer program stored in the readable storage medium executes the image processing method, as can be found in the aforementioned image processing method embodiments.

[0215] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The apparatus provided in the embodiments is described simply because it corresponds to the method provided in the embodiments; relevant parts can be found in the method section.

[0216] The above description of the provided embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features provided herein.

Claims

1. An image processing method, comprising: The background virtual replacement function is enabled, which segments the human body image from the current original image; Identify a handheld object image from the current original image, wherein the handheld object image includes: a case where both the hand and the object are present; or a case where only the object is present and the hand is completely obscured by the object; The image of the handheld object is stitched onto the image of the human body to obtain a stitched image; The stitched image is displayed.

2. The method according to claim 1, wherein before identifying the handheld object image from the current original image, the method further comprises: Upper limb movements are identified based on the upper limb feature information contained in the current original image and the upper limb feature information contained in the historical original images; The upper limb movement was determined to be a target movement.

3. The method according to claim 1, wherein displaying the stitched image comprises: Get the preset background image; The stitched image and the preset background image are displayed synchronously, with the stitched image serving as the foreground image corresponding to the preset background image.

4. The method according to claim 2, wherein upper limb movement is identified based on upper limb feature information contained in the current original image and upper limb feature information contained in historical original images, comprising: Analysis of the current original image determines that it contains upper limb feature information; The upper limb movement mode is determined based on a first region containing upper limb feature information in a historical original image and a second region containing upper limb feature information in the current original image. Based on the agreed upper limb movement rules, the upper limb movements corresponding to the upper limb movement methods are determined.

5. The method according to claim 4, after analyzing the current original image to determine that it contains upper limb feature information, further comprising: Based on the upper limb feature information contained in the current original image, the second region is determined, and the second region is a second region containing upper limb feature information; Add at least two upper limb skeleton markers to the second region.

6. The method according to claim 5, wherein determining the upper limb movement mode based on a first region containing upper limb feature information in a historical original image and a second region containing upper limb feature information in the current original image includes: The movement mode of the upper limb is determined based on at least two upper limb skeleton markers in a first region of the historical original image and at least two upper limb skeleton markers in a second region of the current original image.

7. The method according to claim 2, wherein identifying the handheld object image from the current original image comprises: By combining the upper limb movements with the analysis of the current original image and the historical original image, it is determined that the current original image contains an image of a handheld object, and the image of the handheld object changes synchronously with the upper limb movements in the historical original image and the current original image.

8. The method according to claim 7, wherein the current original image and historical original images are analyzed in conjunction with the upper limb movements to determine that the current original image contains an image of a handheld object, comprising: Based on the analysis of the current original image and the historical original images, the change parameters of the target object image are obtained; Determine that the changing parameters of the target object image match the upper limb movement; The target object image is determined to be a handheld object image.

9. An image processing apparatus, comprising: The determination module is used to determine whether the background virtual replacement function is enabled, wherein the background virtual replacement function segments the human body image from the current original image; An object recognition module is used to recognize a handheld object image from the current original image. The handheld object image includes: a case where both the hand and the object are present; or a case where only the object is present and the hand is completely obscured by the object. The stitching module is used to stitch the image of the handheld object onto the human body image to obtain a stitched image; The display module is used to display the stitched image.

10. An electronic device, comprising: Memory, processor; The memory stores the processing program; The processor is used to load and execute the processing program stored in the memory to implement the steps of the image processing method as described in any one of claims 1-8.

Images