Visual stereoscopic enhancement method, device, equipment and storage medium

Abstract

The present disclosure provides a visual stereoscopic enhancement method, device, equipment and storage medium, relates to the technical field of image processing, in particular to the technical field of image mask, image translation, image fusion and the like, and can be applied to the scenes of portrait visual stereoscopic enhancement, virtual digital person, image or video space sense enhancement and the like. The specific implementation scheme comprises the following steps: according to the configured light direction and light angle, a mask is translated by a target distance in a first mask image corresponding to a target object in a target image to obtain a second mask image; after the first mask image and the second mask image are subtracted pixel by pixel, the regions with a translation increment less than 0 and greater than 1 are discarded to obtain a shadow image; after the shadow image and the first mask image are added pixel by pixel, the target image is multiplied pixel by pixel to obtain an image after stereoscopic enhancement of the target object. The present disclosure can simply and quickly realize visual stereoscopic enhancement and improve visual stereoscopic sense.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to the fields of image masking, image translation, and image fusion, and can be applied to scenarios such as human portrait stereoscopic enhancement, virtual digital humans, and image or video spatial enhancement. In particular, it relates to a method, apparatus, device, and storage medium for stereoscopic enhancement. Background Technology

[0002] Images and videos are two-dimensional planes, losing their spatial depth and three-dimensionality when representing three-dimensional physical space. Enhancing the visual three-dimensionality of an image can improve its expressiveness. For example, in portrait images, enhancing the visual three-dimensionality of the person makes the portrait appear more realistic.

[0003] Currently, the visual stereoscopic enhancement of images is usually achieved through methods such as the three-dimensional color space (HSB) of hue (H), saturation (S), and brightness (B), composition, and contrast between real and virtual elements.

[0004] However, current methods for enhancing visual stereoscopic imagery generally suffer from high implementation difficulty, complex implementation methods, and difficulty in being applied to a wide variety of application scenarios. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, and storage medium for visual stereoscopic enhancement, which can easily and quickly enhance the visual stereoscopic effect of target objects in target images and improve the visual stereoscopic effect of target objects.

[0006] According to a first aspect of this disclosure, a method for enhancing visual stereoscopic vision is provided, the method comprising:

[0007] The process involves acquiring a target image and a first mask image corresponding to the target object within the target image. The first mask image includes a mask of the target object. Based on the configured light direction and angle, the mask of the target object is shifted by a target distance in the first mask image to obtain a second mask image. The magnitude of the target distance is negatively correlated with the magnitude of the light angle. The first mask image and the second mask image are subtracted pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image. Regions in the second mask image with translation increments less than 0 and regions with translation increments greater than 1 are discarded to obtain the shadow image corresponding to the target object. The shadow image and the first mask image are added pixel by pixel to obtain a third mask image. The third mask image and the target image are multiplied pixel by pixel to obtain the image after stereo enhancement of the target object.

[0008] According to a second aspect of this disclosure, a visual stereoscopic enhancement device is provided, the device comprising:

[0009] The acquisition unit is used to acquire a target image and a first mask image corresponding to the target object in the target image, the first mask image including the mask of the target object; the translation unit is used to translate the mask of the target object in the first mask image by a target distance according to the configured light direction and light angle, to obtain a second mask image, the magnitude of the target distance is negatively correlated with the magnitude of the light angle; the shadow generation unit is used to subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image, and then discard the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1, to obtain the shadow image corresponding to the target object; the fusion unit is used to add the shadow image and the first mask image pixel by pixel to obtain a third mask image; the fusion unit is also used to multiply the third mask image and the target image pixel by pixel to obtain an image after stereo enhancement of the target object.

[0010] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0011] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the method described in the first aspect.

[0012] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to the first aspect.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0015] Figure 1 A schematic flowchart of the visual stereoscopic enhancement method provided in the embodiments of this disclosure;

[0016] Figure 2 This is a schematic diagram of a human portrait image;

[0017] Figure 3 for Figure 2A schematic diagram of the first mask image corresponding to the human figure in the image;

[0018] Figure 4 for Figure 2 A schematic diagram of the shadow image corresponding to the human figure in the image;

[0019] Figure 5 To Figure 2 A schematic diagram of a human figure after 3D enhancement;

[0020] Figure 6 Another schematic flowchart of the visual stereoscopic enhancement method provided in this embodiment of the disclosure;

[0021] Figure 7 This is another schematic flowchart of the visual stereoscopic enhancement method provided in the embodiments of this disclosure;

[0022] Figure 8 A schematic diagram of the composition of the visual stereoscopic enhancement device provided in the embodiments of this disclosure;

[0023] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] It should be understood that in the embodiments of this disclosure, the character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0026] Images and videos are two-dimensional planes, losing their spatial depth and three-dimensionality when representing three-dimensional physical space. Enhancing the visual three-dimensionality of an image can improve its expressiveness. For example, in portrait images, enhancing the visual three-dimensionality of the person makes the portrait appear more realistic.

[0027] Currently, the visual stereoscopic enhancement of images is usually achieved through methods such as the three-dimensional color space (HSB) of hue (H), saturation (S), and brightness (B), composition, and contrast between real and virtual elements.

[0028] For example, the HSB color space can process colors at different distances using hue, saturation, and brightness to enhance the visual depth of an image. Furthermore, in terms of composition, techniques such as occlusion, perspective, and reference points can be used to express depth and enhance the visual three-dimensionality of an image. Finally, in terms of contrast between sharpness and blur, techniques like overfocus and lighting can be used to improve the sense of depth and perspective.

[0029] However, current methods for enhancing visual stereoscopic vision are difficult to implement, complex in implementation, and unsuitable for a wide variety of application scenarios.

[0030] For example, in images with a portrait as the foreground, it's difficult to achieve visual stereoscopic enhancement through composition alone. Methods based on the HSB color space require complex calculations and adjustments, making them very difficult to implement and unsuitable for enhancing visual stereoscopic effect in scenarios involving speech or movement. Methods based on the contrast between reality and virtuality require technicians to use professional tools (such as Photoshop) for fine-tuning, which is also difficult to implement and hard to apply to frequently changing interactive scenarios, such as live streaming, game character interaction, and virtual customer service.

[0031] Against this background, this disclosure provides a visual stereoscopic enhancement method that can easily and quickly enhance the visual stereoscopic effect of a target object in a target image by constructing a background shadow, thereby improving the visual stereoscopic effect of the target object.

[0032] The execution subject of this method can be a computer or server, or other devices with image processing capabilities. No restrictions are placed on the execution subject of this method.

[0033] In some embodiments, the server can be a single server, or it can be a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This disclosure does not limit the specific implementation of the server.

[0034] The following is an exemplary description of the visual stereoscopic enhancement method. In the following description, the mask may also be referred to as a mask, and this disclosure does not limit the description of the name.

[0035] Figure 1 This is a schematic flowchart of the visual stereoscopic enhancement method provided in an embodiment of this disclosure. Figure 1 As shown, the method may include:

[0036] S101. Obtain the target image and the first mask image corresponding to the target object in the target image, wherein the first mask image includes the mask of the target object.

[0037] For example, the target image can be an image containing a target object. The target object can be a person, animal, building, mountain, or other object, and there is no limitation on the type of target object.

[0038] Taking a person as the target object as an example, the target image can be a portrait image. The first mask image can be the portrait mask image corresponding to the person.

[0039] Optionally, the target image can be a photograph or any frame from a video; there is no limitation on this.

[0040] S102. Based on the configured light direction and light angle, the mask of the target object is shifted by the target distance in the first mask image to obtain the second mask image. The magnitude of the target distance is negatively correlated with the magnitude of the light angle.

[0041] The direction of the light beam can refer to the direction in which the light shines on the target object. For example, the range of the light beam direction can be from 0 degrees to 360 degrees. The light beam can shine on the target object along a certain direction within the range of 0 degrees to 360 degrees.

[0042] The angle of light can refer to the tilt angle of the direction of light when it shines on a target object. The range of the angle of light can be from 0 degrees to 90 degrees.

[0043] In this embodiment of the disclosure, the direction and angle of the light rays can be configured by the user. For example, the user can determine which side or area of ​​the target object's location they want to cast a shadow on, and configure the corresponding direction and angle of the light rays.

[0044] Users can configure the same or different light directions and angles to suit different scenarios.

[0045] In step S102, the mask of the target object can be translated in the first mask image along the direction indicated by the light rays, according to the light ray direction configured by the user. The translation distance can be called the target distance, and the magnitude of the target distance can be determined based on the light ray angle. The target distance and the light ray angle are negatively correlated; that is, the larger the light ray angle, the smaller the target distance; and the smaller the light ray angle, the larger the target distance.

[0046] For example, assuming the light direction is 0 degrees and the light angle is 60 degrees, the target distance D can be determined based on the negative correlation between the light angle and the target distance. Then, the target object's mask is shifted by the target distance D in the first mask image along the 0-degree direction. The image obtained after shifting the target object's mask by the target distance D is the second mask image.

[0047] Optionally, the target distance can vary from 0 to infinity based on the light angle range of 0 to 90 degrees. That is, when the light angle is 0 degrees, the target distance can be infinite; when the light angle is 90 degrees, the target distance can be 0.

[0048] For example, in this embodiment of the disclosure, the target distance can be represented by pixels.

[0049] S103. Subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image. Then discard the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1 to obtain the shadow image corresponding to the target object.

[0050] After obtaining the second mask image, the first and second mask images can be subtracted pixel by pixel. By subtracting the pixel value of each pixel in the first mask image from the pixel value of each pixel in the second mask image, the translation increment of the second mask image relative to the first mask image can be obtained. For example, each pixel will have a corresponding difference, which is the translation increment for that pixel. After obtaining the translation increment of the second mask image relative to the first mask image, regions with translation increments less than 0 and regions with translation increments greater than 1 in the second mask image can be discarded. The image obtained after discarding these regions is the shadow image corresponding to the target object.

[0051] S104. Add the shadow image and the first mask image pixel by pixel to obtain the third mask image.

[0052] After obtaining the shadow image, the shadow image and the first mask image can be added pixel by pixel (the pixel values ​​of each pixel are added together). The image obtained by adding the shadow image and the first mask image pixel by pixel can be called the third mask image.

[0053] S105. Multiply the third mask image and the target image pixel by pixel to obtain the image after stereo enhancement of the target object.

[0054] After obtaining the third mask image, the third mask image and the target image can be multiplied pixel by pixel. For example, the pixel values ​​corresponding to each pixel can be multiplied. The image obtained after multiplying the third mask image and the target image pixel by pixel is the image after stereo enhancement of the target object.

[0055] For example, taking a human portrait as the target image, Figure 2 This is a schematic diagram of a human portrait image. In step S101 above, [the following can be done / constructed]... Figure 2 The portrait image shown is masked to obtain... Figure 2 The first mask image corresponding to the portrait in the image. For example, Figure 3 for Figure 2 A schematic diagram of the first mask image corresponding to the human figure in the image. Figure 3 It can include Figure 2 The mask corresponding to the human image in the image.

[0056] In S102, the light direction and angle can be configured by the user. Figure 3 The mask corresponding to the general's portrait is shifted by the target distance to obtain... Figure 2 The second mask image corresponding to the portrait in the image.

[0057] In S103, it is possible to... Figure 3 Subtract the second mask image pixel by pixel to obtain the relative second mask image. Figure 3 The translation increment, and the second mask image relative to Figure 3 After the translation increment, regions with translation increments less than 0 and regions with translation increments greater than 1 in the second mask image are discarded, resulting in... Figure 2 The shadow image corresponding to the portrait in the image. For example, Figure 4 for Figure 2 A schematic diagram of the shadow image corresponding to the human figure in the image.

[0058] S104 can be Figure 4 and Figure 3 The third mask image is obtained by adding each pixel individually.

[0059] In S105, the third mask image and Figure 2 Pixel-by-pixel multiplication yields an image with stereoscopic enhancement of the target object. For example, Figure 5 To Figure 2 A schematic diagram of a portrait after stereoscopic enhancement, showing how enhancement can be achieved quickly and easily using steps S101-S105. Figure 2 The three-dimensionality of the portrait in the image.

[0060] This embodiment of the disclosure acquires a target image and a first mask image corresponding to a target object in the target image; based on the configured light direction and light angle, the mask of the target object is shifted by a target distance in the first mask image to obtain a second mask image; the first and second mask images are subtracted pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image, and the regions in the second mask image with translation increments less than 0 and greater than 1 are discarded to obtain the shadow image corresponding to the target object; the shadow image and the first mask image are added pixel by pixel to obtain a third mask image; the third mask image and the target image are multiplied pixel by pixel to obtain the image after stereoscopic enhancement of the target object. This achieves visual stereoscopic enhancement of the target object in the target image based on the construction of background shadows, enhancing the spatial stereoscopic sense of the 2D image, reflecting the three-dimensional spatial hierarchy, and improving the visual stereoscopic sense of the target object. This method achieves visual stereoscopic enhancement in a simple and quick manner, and can be better applied to a variety of application scenarios.

[0061] For example, this method can be applied to enhance the visual stereoscopic appearance of virtual digital humans and game characters in scenarios such as virtual digital humans (e.g., virtual anchors, virtual customer service representatives, virtual idols, etc.) and games (2D games, 3D games).

[0062] Compared to most current methods of visual stereoscopic enhancement, the method provided in this disclosure has a simple processing procedure, requiring no manual adjustments to details and only needing a foreground mask to generate shadows. Furthermore, shadow generation is determined by the user-configured light direction and angle, allowing users to flexibly adjust the stereoscopic enhancement effect by configuring these parameters, such as adjusting shadow direction and shadow length.

[0063] In some embodiments, the target distance is non-linearly related to the angle of the light rays.

[0064] For example, the target distance can vary non-linearly from 0 to infinity, depending on the light angle range of 0 to 90 degrees.

[0065] When the distance to the target is non-linearly related to the angle of the light, the shadow in the shadow image corresponding to the target object can be closer to the real light projection. That is, the distance between the shadow and the target object, as well as the shape of the shadow, are closer to the real shadow.

[0066] In this embodiment, the target distance and the angle of light are non-linearly related, which makes the shadow effect more realistic and further enhances the visual three-dimensionality.

[0067] In other embodiments, the target distance may also be linearly related to the angle of light, and this disclosure does not limit this.

[0068] In some embodiments, the target object may include a first target object or a second target object. The first target object and the second target object are different; for example, the first target object is a person, and the second target object is a building.

[0069] When the target object is the first target object, the correlation between the target distance and the angle of the light is the first correlation; when the target object is the second target object, the correlation between the target distance and the angle of the light is the second correlation; the first correlation and the second correlation are different.

[0070] In other words, in this embodiment, the relationship between the target distance and the light angle can differ for different target objects. This difference refers to the varying degrees to which the target distance changes with the light angle.

[0071] In this embodiment, the relationship between the target distance and the light angle varies for different target objects. This allows for a more accurate adaptation of the relationship between the target distance and the light angle for different target objects, resulting in shadows that are closer to the real shadows of the target objects and further enhancing the visual stereoscopic effect.

[0072] In other embodiments, the relationship between the target distance and the light angle may be the same for different target objects, and this disclosure does not impose any limitations.

[0073] In some embodiments, before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the method further includes: blurring the edges of the shadow image.

[0074] For example, Figure 6 Another schematic diagram of the visual stereoscopic enhancement method provided in this embodiment of the disclosure. Figure 6 As shown, the method may include:

[0075] S601. Obtain the target image and the first mask image corresponding to the target object in the target image, wherein the first mask image includes the mask of the target object.

[0076] S602. Based on the configured light direction and light angle, the mask of the target object is shifted by the target distance in the first mask image to obtain the second mask image. The magnitude of the target distance is negatively correlated with the magnitude of the light angle.

[0077] S603. Subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image. Then discard the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1 to obtain the shadow image corresponding to the target object.

[0078] S601-S603 can be referred to S101-S103 above, and will not be repeated here.

[0079] S604. Blur the edges of the shadow image.

[0080] In S604, edge virtualization processing can be performed on the shadow areas in the shadow image. For example, the outline of the shadow can be blurred. By performing edge virtualization processing on the shadow areas, the realism of the shadows can be improved, making the shadows more closely resemble the presentation effect in a real scene.

[0081] S605. Add the shadow image and the first mask image pixel by pixel to obtain the third mask image.

[0082] S606. Multiply the third mask image and the target image pixel by pixel to obtain the image after stereo enhancement of the target object.

[0083] S605-S606 can be referred to S104-S105 above, and will not be repeated here.

[0084] In this embodiment, before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the shadow image is subjected to edge blurring processing, which can make the shadow presentation effect more realistic and the visual three-dimensional effect of the target object better in the final image after the target object is stereoscopically enhanced.

[0085] Optionally, in this embodiment of the disclosure, the edge blurring processing method includes Gaussian blurring or Poisson fusion. For example, Gaussian blurring can be applied to the shadow region in the shadow image, or a Poisson fusion algorithm can be used to process the shadow region in the shadow image, thereby blurring the edges of the shadow region in the shadow image. This disclosure does not limit the specific method of edge blurring processing.

[0086] In some embodiments, before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the method further includes: performing a transparency gradient processing on the shadow image.

[0087] For example, Figure 7 This is another schematic flowchart illustrating the visual stereoscopic enhancement method provided in this embodiment of the disclosure. Figure 7 As shown, the method may include:

[0088] S701. Obtain the target image and the first mask image corresponding to the target object in the target image, wherein the first mask image includes the mask of the target object.

[0089] S702. Based on the configured light direction and light angle, the mask of the target object is shifted by the target distance in the first mask image to obtain the second mask image. The magnitude of the target distance is negatively correlated with the magnitude of the light angle.

[0090] S703. Subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image. Then discard the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1 to obtain the shadow image corresponding to the target object.

[0091] S701-S703 can be referred to S101-S103 above, and will not be repeated here.

[0092] S704. Blur the edges of the shadow image.

[0093] S704 can be referred to in S604 above, and will not be repeated here.

[0094] S705. Apply a transparency gradient to the shadow image.

[0095] Similar to edge virtualization, S705 allows for transparency gradient processing of shadow areas in shadow images. For example, the transparency of different areas within a shadow can be adjusted to create a gradient effect. In real-world scenes, the transparency of different areas in a light projection is not entirely uniform; variations may exist between different areas. By applying transparency gradient processing to shadow areas, the realism of the shadows can be improved, making them more closely resemble the appearance of a real scene.

[0096] S706. Add the shadow image and the first mask image pixel by pixel to obtain the third mask image.

[0097] S707. Multiply the third mask image and the target image pixel by pixel to obtain the image after stereo enhancement of the target object.

[0098] Optionally, this disclosure does not restrict the execution order of S704 and S705 described above; for example, S705 may be executed before S704. In some examples, only one of S704 and S705 may be executed, and this is not restricted here.

[0099] S706-S707 can be referred to S104-S105 above, and will not be repeated here.

[0100] In this embodiment, before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the shadow image is subjected to a transparency gradient process. This can also make the shadow presentation effect more realistic in the final image after the target object is stereoscopically enhanced, further improving the visual stereoscopic effect of the target object.

[0101] Optionally, the transparency gradient processing method includes: linear transparency gradient or non-linear transparency gradient; wherein, after the transparency gradient processing is applied to the shadow image, the transparency of the edge area near the target object is less than the transparency of the edge area far away from the target object.

[0102] In other words, the principle for applying gradual transparency to shadow images can be: the closer to the edge of the target object, the lower the transparency of the shadow; the farther away from the edge of the target object, the higher the transparency of the shadow. For example, the transparency near the edge of the target object is 1 (completely opaque), and the transparency far from the edge of the target object is 0 (completely transparent). Transparency is represented by the alpha channel value in a PNG image.

[0103] This disclosure does not impose any restrictions on the method of transparency gradient processing.

[0104] In some embodiments, obtaining the first mask image corresponding to the target object in the target image may include: performing image matting or semantic segmentation on the target image to obtain the first mask image corresponding to the target object in the target image.

[0105] This embodiment can generate a first mask image more simply and quickly, which can then be used to obtain a stereoscopic enhanced image of the target object. For rapidly changing scenarios such as live streaming and video, this embodiment can significantly improve the processing speed of visual stereoscopic enhancement.

[0106] In an exemplary embodiment, this disclosure also provides a visual stereoscopic enhancement device, which can be used to implement the visual stereoscopic enhancement method as described in the foregoing embodiments. Figure 8 This is a schematic diagram illustrating the composition of the visual stereoscopic enhancement device provided in an embodiment of this disclosure. Figure 8 As shown, the visual stereoscopic enhancement device may include: an acquisition unit 801, a translation unit 802, a shadow generation unit 803, and a fusion unit 804.

[0107] The acquisition unit 801 is used to acquire a target image and a first mask image corresponding to a target object in the target image, wherein the first mask image includes a mask of the target object.

[0108] Translation unit 802 is used to translate the mask of the target object in the first mask image by a target distance according to the configured light direction and light angle to obtain the second mask image. The magnitude of the target distance is negatively correlated with the magnitude of the light angle.

[0109] The shadow generation unit 803 is used to subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image, and then discard the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1 to obtain the shadow image corresponding to the target object.

[0110] The fusion unit 804 is used to add the shadow image and the first mask image pixel by pixel to obtain the third mask image.

[0111] The fusion unit 804 is also used to multiply the third mask image and the target image pixel by pixel to obtain an image after stereo enhancement of the target object.

[0112] Optionally, the target distance is non-linearly related to the angle of the light rays.

[0113] Optionally, the target object includes a first target object or a second target object; when the target object is the first target object, the correlation between the target distance and the light angle is the first correlation; when the target object is the second target object, the correlation between the target distance and the light angle is the second correlation; the first correlation and the second correlation are different.

[0114] Optionally, before the fusion unit 804 adds the shadow image and the first mask image pixel by pixel to obtain the third mask image, it is also used to perform edge blurring processing on the shadow image.

[0115] Alternatively, edge blurring can be achieved through Gaussian blur or Poisson blending.

[0116] Optionally, before the fusion unit 804 adds the shadow image and the first mask image pixel by pixel to obtain the third mask image, it is also used to perform transparency gradient processing on the shadow image.

[0117] Optionally, the transparency gradient processing method includes: linear transparency gradient or non-linear transparency gradient; wherein, after the transparency gradient processing is applied to the shadow image, the transparency of the edge area near the target object is less than the transparency of the edge area far away from the target object.

[0118] Optionally, the acquisition unit 801 is specifically used to perform image matting or semantic segmentation on the target image to obtain a first mask image corresponding to the target object in the target image.

[0119] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0121] In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the above embodiments. The electronic device may be the computer or server described above.

[0122] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the above embodiments.

[0123] In an exemplary embodiment, the computer program product includes a computer program that, when executed by a processor, implements the method described in the above embodiments.

[0124] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0125] like Figure 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may also store various programs and data required for the operation of the device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0126] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0127] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the stereoscopic enhancement method. For example, in some embodiments, the stereoscopic enhancement method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the stereoscopic enhancement method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the stereoscopic enhancement method by any other suitable means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0133] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0134] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0135] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for enhancing visual stereoscopic vision, the method comprising: Obtain a target image and a first mask image corresponding to a target object in the target image, wherein the first mask image includes a mask of the target object; Based on the configured light direction and light angle, the mask of the target object is shifted by a target distance in the first mask image to obtain a second mask image, wherein the magnitude of the target distance is negatively correlated with the magnitude of the light angle; The first mask image and the second mask image are subtracted pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image. Then, the regions in the second mask image with translation increment less than 0 and the regions with translation increment greater than 1 are discarded to obtain the shadow image corresponding to the target object. The shadow image and the first mask image are added pixel by pixel to obtain the third mask image; The third mask image and the target image are multiplied pixel by pixel to obtain an image after stereo enhancement of the target object.

2. The method according to claim 1, wherein the magnitude of the target distance is non-linearly related to the magnitude of the light angle.

3. The method according to claim 1 or 2, wherein the target object includes a first target object or a second target object; When the target object is the first target object, the correlation between the target distance and the light angle is the first correlation relationship; When the target object is the second target object, the correlation between the target distance and the light angle is the second correlation relationship; The first correlation is different from the second correlation.

4. The method according to any one of claims 1-3, wherein before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the method further comprises: The shadow image is then subjected to edge blurring processing.

5. The method according to claim 4, wherein the edge blurring processing includes: Gaussian blur or Poisson blending.

6. The method according to any one of claims 1-5, wherein before adding the shadow image and the first mask image pixel by pixel to obtain the third mask image, the method further comprises: The shadow image is then subjected to a transparency gradient processing.

7. The method according to claim 6, wherein the transparency gradient processing comprises: Linear or non-linear transparency gradients; Specifically, after applying a transparency gradient to the shadow image, the transparency of the edge region closer to the target object is less than the transparency of the edge region farther from the target object.

8. The method according to any one of claims 1-7, wherein obtaining the first mask image corresponding to the target object in the target image comprises: The target image is matted or semantically segmented to obtain a first mask image corresponding to the target object in the target image.

9. A visual stereoscopic enhancement device, the device comprising: An acquisition unit is configured to acquire a target image and a first mask image corresponding to a target object in the target image, wherein the first mask image includes a mask of the target object; The translation unit is used to translate the mask of the target object in the first mask image by a target distance according to the configured light direction and light angle to obtain a second mask image, wherein the magnitude of the target distance is negatively correlated with the magnitude of the light angle; The shadow generation unit is used to subtract the first mask image and the second mask image pixel by pixel to obtain the translation increment of the second mask image relative to the first mask image, and then discard the regions in the second mask image where the translation increment is less than 0 and the regions where the translation increment is greater than 1, to obtain the shadow image corresponding to the target object. The fusion unit is used to add the shadow image and the first mask image pixel by pixel to obtain a third mask image; The fusion unit is further configured to multiply the third mask image and the target image pixel by pixel to obtain an image after stereo enhancement of the target object.

10. The apparatus according to claim 9, wherein the magnitude of the target distance is non-linearly related to the magnitude of the light angle.

11. The apparatus according to claim 9 or 10, wherein the target object includes a first target object or a second target object; When the target object is the first target object, the correlation between the target distance and the light angle is the first correlation relationship; When the target object is the second target object, the correlation between the target distance and the light angle is the second correlation relationship; The first correlation is different from the second correlation.

12. The apparatus according to any one of claims 9-11, wherein before the fusion unit adds the shadow image and the first mask image pixel by pixel to obtain the third mask image, it is further configured to perform edge blurring processing on the shadow image.

13. The apparatus according to claim 12, wherein the edge blurring processing comprises: Gaussian blur or Poisson blending.

14. The apparatus according to any one of claims 9-13, wherein before the fusion unit adds the shadow image and the first mask image pixel by pixel to obtain the third mask image, it is further configured to perform a transparency gradient processing on the shadow image.

15. The apparatus of claim 14, wherein the transparency gradient processing comprises: Linear or non-linear transparency gradients; Specifically, after applying a transparency gradient to the shadow image, the transparency of the edge region closer to the target object is less than the transparency of the edge region farther from the target object.

16. The apparatus according to any one of claims 9-15, wherein the acquisition unit is specifically configured to perform image matting or semantic segmentation on the target image to obtain a first mask image corresponding to the target object in the target image.

17. An electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images