Image rendering method, device and equipment, computer readable storage medium and product

Abstract

The embodiment of the invention provides an image rendering method, device and equipment, a computer readable storage medium and a product, and the method comprises the steps: obtaining an image rendering request which comprises an original image and a special effect identifier; performing down-sampling operation on the original image according to the image rendering request to obtain a to-be-processed image with a target resolution, the target resolution being lower than an original resolution corresponding to the original image; performing data processing on the to-be-processed image through a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result; a target rendering window matched with the target resolution is adopted to carry out rendering operation on the to-be-processed image based on the algorithm processing result, and a rendered image is obtained; and adjusting the resolution of the rendered image to the original resolution to obtain a target image. Therefore, the image quality of the generated target image can be ensured on the basis of reducing the link memory size and the calculation complexity, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular, to an image rendering method, apparatus, device, computer-readable storage medium, and product. Background Art

[0002] With the development of image processing technology, more and more image / video special effects processing application software have entered the lives of users. Users can select special effects content in image / video special effects processing application software and perform content acquisition operations based on the selected special effects content.

[0003] The resolution of videos / pictures currently collected by terminal devices is relatively high. Therefore, in the rendering process of special effects processing, processing the original videos / pictures with relatively high resolutions often takes a long time and consumes a lot of power on the terminal devices, making it impossible to quickly collect content based on special effects content, resulting in a poor user experience. Summary of the invention

[0004] The embodiments of the present disclosure provide an image rendering method, apparatus, device, computer-readable storage medium and product, which are used to solve the technical problems of low frame rate and high power consumption when existing special effects rendering methods perform rendering operations on high-resolution images.

[0005] In a first aspect, an embodiment of the present disclosure provides an image rendering method, including:

[0006] Obtaining an image rendering request, wherein the image rendering request includes an original image and a special effect identifier;

[0007] Performing a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image;

[0008] Performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result;

[0009] Using a target rendering window that matches the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image;

[0010] The resolution of the rendered image is adjusted to the original resolution to obtain a target image.

[0011] In a second aspect, an embodiment of the present disclosure provides an image rendering device, including:

[0012] An acquisition module, used for acquiring an image rendering request, wherein the image rendering request includes an original image and a special effect identifier;

[0013] A downsampling module, configured to perform a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image;

[0014] A processing module, used for performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result;

[0015] A rendering module, used to use a target rendering window matching the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image;

[0016] The adjustment module is used to adjust the resolution of the rendered image to the original resolution to obtain a target image.

[0017] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0018] The memory stores computer-executable instructions;

[0019] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image rendering method described in the first aspect and various possible designs of the first aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image rendering method described in the first aspect and various possible designs of the first aspect is implemented.

[0021] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image rendering method described in the first aspect and various possible designs of the first aspect.

[0022] The image rendering method, apparatus, device, computer-readable storage medium and product provided in this embodiment, after obtaining an image rendering request, downsamples the original image to reduce the resolution of the original image to obtain an image to be processed with a target resolution. The image to be processed is algorithmically processed by a special effect algorithm to obtain an algorithm processing result. A target rendering window matching the target resolution is pre-constructed, and a rendering operation is performed on the image to be processed based on the target rendering window according to the algorithm processing result to obtain a rendered image. By reducing the size of the rendering window, the size of the final rendering area can be reduced, the intermediate texture cache in the rendering link can be reduced, the rendering calculation complexity can be reduced, and the memory usage can be reduced. Furthermore, the resolution of the rendered image can be adjusted to the original resolution to obtain the target image. Thus, the image quality of the generated target image can be guaranteed on the basis of reducing the link memory size and the calculation complexity, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A flowchart of an image rendering method provided by an embodiment of the present disclosure;

[0025] Figure 2 A schematic diagram of image rendering provided by an embodiment of the present disclosure;

[0026] Figure 3 Another image rendering schematic diagram provided for an embodiment of the present disclosure;

[0027] Figure 4 A flowchart of an image rendering method provided by another embodiment of the present disclosure;

[0028] Figure 5 A flowchart of an image rendering method provided by another embodiment of the present disclosure;

[0029] Figure 6 A schematic diagram of the structure of an image rendering device provided by an embodiment of the present disclosure;

[0030] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0033] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0034] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0035] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0036] Glossary:

[0037] Super-resolution: It is to improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction. The core idea of ​​super-resolution reconstruction is to use time bandwidth (obtain a multi-frame image sequence of the same scene) in exchange for spatial resolution, realizing the conversion of time resolution to spatial resolution.

[0038] Downsampling: refers to the process of adjusting the size of a digital image.

[0039] In order to solve the technical problems of low frame rate and high power consumption in existing special effects rendering methods when rendering high-resolution images, the present disclosure provides an image rendering method, apparatus, device, computer-readable storage medium and product.

[0040] It should be noted that the image rendering method, apparatus, device, computer-readable storage medium and product provided by the present disclosure can be applied to any image rendering application scenario.

[0041] There are many image / video special effects processing social software on the market. Based on the image / video special effects processing software, the specific image or video input by the user can be first processed by algorithm, and then the image rendering operation can be performed according to the results obtained by the algorithm processing.

[0042] The video / picture size captured by current terminal devices is generally of large resolution, for example, common resolutions are 576p, 720p+, etc. In the special effects rendering process, the input original video / picture needs to be processed. For more complex special effects, the processing link of the video / picture will be longer.

[0043] In the special effects processing process, the two indicators that are directly related to the user experience generally include frame rate (time consumption) and power consumption. These two indicators are strongly related to the rendering process of special effects processing, and the rendering process of special effects processing is strongly related to the resolution of the user input video / picture and the specific rendering effect chain. When the rendering effect chain is constant, the higher the resolution of the user input video / picture, the lower the frame rate (longer the time consumption), the higher the power consumption, and the worse the related user experience.

[0044] In the process of solving the above technical problems, the inventors found through research that in order to ensure the image quality of the generated target image on the basis of reducing the link memory size and computational complexity, after obtaining the original image, the original image can be downsampled to obtain the image to be processed to reduce the resolution of the original image. Algorithmic processing and special effects rendering operations are performed based on the low-resolution image to be processed to obtain a rendered image. In order to obtain a clearer rendering result, the resolution of the rendered image can also be adjusted to adjust the resolution of the rendered image to the same original resolution as the original image.

[0045] Furthermore, in order to reduce the computational complexity and memory usage during the rendering process, a low-resolution rendering window may be pre-built, and the image to be processed may be rendered based on the low-resolution rendering window.

[0046] The execution subject of the embodiment of the present disclosure is an image rendering device, which can be coupled to a server. The server can be connected to a terminal device for communication. Thus, the server can obtain an image rendering request sent by the terminal device, and perform an image rendering operation based on the image rendering request by combining downsampling and super-resolution algorithms.

[0047] Alternatively, the image rendering device can be coupled to a terminal device, so as to obtain the original image collected or uploaded by the user and determine the special effect identifier selected by the user. The image rendering operation is performed based on the original image and the special effect identifier by combining downsampling and super-resolution algorithms.

[0048] Figure 1 A flowchart of an image rendering method provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method includes:

[0049] Step 101: Obtain an image rendering request, wherein the image rendering request includes an original image and a special effect identifier.

[0050] In this embodiment, the user can select a special effect identifier on the terminal device, and perform an image rendering operation on the currently generated original image based on the special effect content corresponding to the special effect identifier. The terminal device can initiate an image rendering request based on the special effect identifier and the original image.

[0051] Optionally, the image rendering device may obtain an image rendering request, wherein the image rendering request includes an original image and a special effect identifier. The original image may be collected by a user in real time or uploaded in a preset storage path, and the present disclosure does not limit this.

[0052] Step 102: downsample the original image according to the image rendering request to obtain a to-be-processed image of a target resolution, where the target resolution is lower than an original resolution corresponding to the original image.

[0053] In this embodiment, during the image rendering process, the original image needs to be processed by the special effect algorithm corresponding to the special effect identifier to obtain the algorithm processing result. The original image is rendered based on the algorithm processing result. However, since the original image has a high resolution, directly processing the original image by the algorithm and rendering the special effects may result in a large amount of calculation, a large memory usage, and a reduced frame rate, which affects the user experience.

[0054] Therefore, in order to improve the efficiency of special effects rendering and reduce the amount of calculation and content occupancy in the special effects rendering process, the original image can be downsampled according to the image rendering request to obtain a processed image of the target resolution, which is lower than the original resolution corresponding to the original image. The target resolution can be set by the user according to actual needs.

[0055] Optionally, any downsampling method may be used to implement the downsampling operation on the original image.

[0056] For example, the original image may be a 720p image, which can be downsampled based on the preset target resolution to obtain a 480p image to be processed. Performing algorithm processing and special effects rendering operations based on the low-resolution image to be processed will reduce the computational complexity of the input image itself.

[0057] Step 103: performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result.

[0058] In this embodiment, after obtaining a low-resolution image to be processed, the special effect algorithm corresponding to the special effect identifier in the image rendering request can be determined. The image to be processed is algorithmically processed based on the special effect algorithm to obtain an algorithm processing result. The algorithm processing includes but is not limited to the identification of key point information, the calculation of the rendering area, and the like.

[0059] Since the resolution of the image to be processed is lower than the original resolution, the computational complexity of the algorithm processing based on the image to be processed is low.

[0060] Step 104: Using a target rendering window that matches the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image.

[0061] In this embodiment, the target rendering window corresponding to the target resolution can be pre-set. By reducing the size of the rendering window, the size of the final rendering area can be reduced, the intermediate texture buffers in the rendering chain can be reduced, the rendering calculation complexity can be reduced, and the memory usage can be reduced.

[0062] Therefore, after obtaining the algorithm processing result, the target rendering window can be used to render the image to be processed based on the algorithm processing result to obtain a rendered image. Performing special effect rendering operations based on the low-resolution image to be processed can effectively reduce the processing time and power consumption of the special effect rendering operation.

[0063] Step 105: Adjust the resolution of the rendered image to the original resolution to obtain a target image.

[0064] In this embodiment, since the rendered image is obtained by rendering based on the low-resolution image to be processed, the resolution of the rendered image is also low. In order to provide users with clear and high-quality rendering results, the resolution of the rendered image can be adjusted to the original resolution to obtain the target image.

[0065] Among them, any method can be used to achieve super-resolution processing of the rendered image. For example, the resolution of the rendered image can be adjusted by a super-resolution algorithm, or the resolution of the rendered image can be adjusted by upsampling, etc. The present disclosure does not impose any restrictions on this.

[0066] Optionally, based on any of the above embodiments, step 105 includes:

[0067] The rendered image is processed based on a preset super-resolution algorithm to obtain the target image.

[0068] In this embodiment, a super-resolution algorithm may be preset, so that after the rendered image is acquired, image processing may be performed on the rendered image based on the preset super-resolution algorithm to obtain a target image.

[0069] Figure 2 A schematic diagram of image rendering provided by an embodiment of the present disclosure, such as Figure 2 As shown, after obtaining the original image 21, the original image 21 can be downsampled to obtain a to-be-processed image 22 of a target resolution. The to-be-processed image 22 is algorithmically processed based on the special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result. A target rendering window 23 matching the target resolution is used to render the to-be-processed image 22 based on the algorithm processing result to obtain a rendered image 24. The resolution of the rendered image 24 is adjusted to the original resolution to obtain a target image 25.

[0070] The image rendering method provided in this embodiment performs a downsampling operation on the original image after obtaining an image rendering request to reduce the resolution of the original image and obtain an image to be processed of a target resolution. The image to be processed is algorithmically processed by a special effect algorithm to obtain an algorithm processing result. A target rendering window matching the target resolution is pre-built, and a rendering operation is performed on the image to be processed based on the target rendering window according to the algorithm processing result to obtain a rendered image. By reducing the size of the rendering window, the size of the final rendering area can be reduced, the intermediate texture cache in the rendering link can be reduced, the rendering calculation complexity can be reduced, and the memory usage can be reduced. Furthermore, the resolution of the rendered image can be adjusted to the original resolution to obtain the target image. In this way, the image quality of the generated target image can be guaranteed on the basis of reducing the link memory size and the calculation complexity, thereby improving the user experience.

[0071] Further, based on any of the above embodiments, step 102 includes:

[0072] Determine whether the original image meets a preset downsampling condition.

[0073] If so, a downsampling operation is performed on the original image according to the image rendering request.

[0074] In this embodiment, in order to reduce the amount of calculation and memory pressure in the special effect rendering process, the original image may be downsampled to obtain a to-be-processed image having a resolution lower than the original resolution.

[0075] In practical applications, the original image can be a low-resolution image or a high-resolution image acquired in real time. For low-resolution images, since their resolution is low, the amount of calculation required for directly processing the special effects algorithm and performing special effects rendering operations is small, thereby ensuring the real-time nature of the rendering operation.

[0076] For various original images, after obtaining the original image, it can be determined whether it is necessary to perform downsampling operation on the original image. In order to accurately process the original image, the downsampling conditions can be established in advance. After obtaining the original image, it is determined whether the original image meets the preset downsampling conditions. If so, the original image is downsampled according to the image rendering request. Otherwise, the special effects algorithm can be directly processed and the special effects rendering operation can be performed on it.

[0077] Further, based on any of the above embodiments, determining whether the original image meets a preset downsampling condition includes:

[0078] An original resolution corresponding to the original image is determined.

[0079] Determine whether the original resolution is greater than a preset resolution threshold, or determine whether the original resolution exceeds the processing capability of the special effect processing component.

[0080] If so, it is determined that the original image meets the preset downsampling condition.

[0081] In this embodiment, the preset downsampling condition may be that the original resolution of the original image exceeds a preset resolution threshold, or that the original resolution exceeds the processing capability of the special effect processing component. The resolution threshold may be set by the user according to actual needs, and the present disclosure does not limit this.

[0082] Therefore, after obtaining the original image, the original resolution corresponding to the original image can be determined first. It is determined whether the original resolution is greater than a preset resolution threshold, or whether the original resolution exceeds the processing capability of the special effect processing component. If so, it is determined that the original image meets the preset downsampling condition.

[0083] Taking a practical application as an example, the original image may be a 480p image. The preset resolution threshold may be 720p. Since the original resolution of the original image is less than the resolution threshold, the special effects algorithm and special effects rendering operations may be directly performed on it. Conversely, if the original resolution of the original image is greater than or equal to 720p, the original image may be downsampled, and the image to be processed obtained based on the downsampling operation may be processed by the special effects algorithm and special effects rendering operations.

[0084] Further, based on any of the above embodiments, the original image is a live video frame corresponding to the live broadcast operation.

[0085] After determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component, the method further includes:

[0086] Determine whether the live video frame is a video frame generated in a microphone connection scenario.

[0087] If so, it is determined that the live video frame does not meet the downsampling condition.

[0088] If not, it is determined that the live video frame meets the downsampling condition.

[0089] In this embodiment, the original image may be a live video frame generated during the live broadcast operation. During the live broadcast process, the anchor may initiate a joint operation with other anchors according to actual needs to enter a joint scene. In the joint scene, the image frames of two anchors may be displayed simultaneously in the display interface. Therefore, the live video frame generated in the joint scene does not need to be downsampled.

[0090] Therefore, after determining whether the original resolution is greater than the preset resolution threshold, or determining whether the original resolution exceeds the processing capacity of the special effects processing component, it can be determined whether the live video frame is a video frame generated in the microphone connection scenario. If so, it is determined that the live video frame does not meet the downsampling conditions, and the special effects algorithm and special effects rendering operations can be directly performed on it. If not, it is determined that the live video frame meets the downsampling conditions, and the original image can be downsampled, and the special effects algorithm and special effects rendering operations can be performed based on the image to be processed obtained by the downsampling operation.

[0091] Figure 3 Another image rendering schematic diagram provided by the embodiment of the present disclosure is as follows: Figure 3 As shown, after acquiring the original image 31, it can be determined whether the original image 31 meets the preset downsampling condition. If so, the original image 31 can be downsampled to obtain the image to be processed 32. Special effect algorithm processing is performed based on the image to be processed 32 to obtain the algorithm processing result. Based on the algorithm processing result, the image to be processed is rendered to obtain a rendered image 33, and the rendered image 33 is super-resolved to obtain the target image 34. When it is determined that the original image 31 does not meet the preset downsampling condition, the original image 31 can be processed by a special effect algorithm to obtain the algorithm processing result. Based on the algorithm processing result, the original image 31 is rendered to obtain the target image 34.

[0092] The image rendering method provided in this embodiment sets the downsampling conditions in advance, so that after the original image is acquired, it can accurately determine whether the original image needs to be downsampled based on the downsampling conditions, thereby improving the efficiency of the image rendering operation.

[0093] Figure 4 A flowchart of an image rendering method provided by another embodiment of the present disclosure is provided. Based on any of the above embodiments, Figure 4 As shown, step 102 includes:

[0094] Step 401: Obtain a first texture map corresponding to the original image.

[0095] Step 402: Render the first texture map into a second texture map corresponding to the target resolution to obtain the image to be processed.

[0096] In this embodiment, after the original image is acquired, a first texture map of the original image may be acquired, and the first texture map may be rendered into a second texture map corresponding to the target resolution to obtain the image to be processed.

[0097] Taking a practical application as an example, if the original image is a 720p image, the first texture map corresponding to the 720p original image can be rendered to a second texture map with a preset target resolution of 480p using the LINEAR method using OpenGL operation to complete the texture downsampling process.

[0098] The image rendering method provided in this embodiment obtains the first texture map corresponding to the original image, renders the first texture map into the second texture map corresponding to the target resolution, and obtains the image to be processed. Thus, the resolution of the original image can be accurately reduced to the preset target resolution. Then, performing the image rendering operation based on the image to be processed at the target resolution can reduce the rendering time and power consumption of the special effect processing process.

[0099] Further, based on any of the above embodiments, before step 104, the method further includes:

[0100] Displays the Rendering page of presets.

[0101] In response to a setting operation triggered by the user on the rendering page, a parameter adjustment control is displayed.

[0102] The target rendering window is constructed based on the adjustment parameters associated with the target resolution generated by the user in the parameter adjustment control.

[0103] In this embodiment, an original rendering window may be pre-set, and the original rendering window is used to perform a rendering operation on a high-resolution image. In order to implement a rendering operation on an image to be processed obtained by a downsampling operation, a target rendering window may be pre-built.

[0104] Optionally, a preset rendering page may be displayed, and in response to a user triggering an edit control preset in the rendering page, a parameter adjustment control may be displayed. The user may enter an adjustment parameter associated with a target resolution in the display parameter adjustment control, so that a target rendering window can be constructed based on the target resolution to reduce the size of the original rendering window.

[0105] The image rendering method provided in this embodiment pre-establishes a target rendering window that matches the target resolution, reduces the size of the rendering window, reduces the size of the final rendering area, reduces the intermediate texture cache in the rendering chain, reduces the complexity of rendering calculations, and reduces memory usage.

[0106] Figure 5 A flowchart of an image rendering method provided by another embodiment of the present disclosure is shown below. Based on any of the above embodiments, the target image includes a portrait area. Figure 5 As shown, step 105 includes:

[0107] Step 501: Identify the portrait area in the image to be processed.

[0108] Step 502: Use the target rendering window to perform skin smoothing rendering operation and / or makeup rendering operation and / or background sticker rendering operation and / or deformation rendering operation on the portrait area.

[0109] In this embodiment, the original image may be a video frame during a live broadcast, or an image obtained by a user taking a selfie with a terminal device, and the original image may be an image including a portrait area. In the process of rendering special effects, in order to further improve rendering efficiency and reduce the amount of calculation in the rendering process, only the portrait area may be rendered.

[0110] Optionally, the special effect identifier currently selected by the user may be a special effect for rendering the portrait area. Therefore, after downsampling the original image to obtain the image to be processed, the portrait area in the image to be processed may be identified. The target rendering window is used to perform a skin smoothing rendering operation and / or a beauty rendering operation and / or a background sticker rendering operation and / or a deformation rendering operation on the portrait area.

[0111] It should be noted that if the special effect currently selected by the user is identified as a special effect for global rendering, after downsampling the original image to obtain the image to be processed, the special effect rendering operation can be performed on the entire area in the image to be processed based on the target rendering window.

[0112] The image rendering method provided in this embodiment can further reduce the computational complexity and time consumption of the rendering operation, improve the efficiency of the rendering operation, and enhance the user experience by rendering the portrait area based on the target rendering window.

[0113] Figure 6 A schematic diagram of the structure of an image rendering device provided by an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the device includes: an acquisition module 61, a downsampling module 62, a processing module 63, a rendering module 64 and an adjustment module 65. Among them, the acquisition module 61 is used to obtain an image rendering request, wherein the image rendering request includes an original image and a special effect identifier. The downsampling module 62 is used to perform a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, and the target resolution is lower than the original resolution corresponding to the original image. The processing module 63 is used to perform data processing on the image to be processed by a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result. The rendering module 64 is used to perform a rendering operation on the image to be processed based on the algorithm processing result using a target rendering window matching the target resolution to obtain a rendered image. The adjustment module 65 is used to adjust the resolution of the rendered image to the original resolution to obtain a target image.

[0114] Further, based on any of the above embodiments, the downsampling module is used to: determine whether the original image meets a preset downsampling condition, and if so, perform a downsampling operation on the original image according to the image rendering request.

[0115] Further, based on any of the above embodiments, the downsampling module is used to: determine the original resolution corresponding to the original image, determine whether the original resolution is greater than a preset resolution threshold, or determine whether the original resolution exceeds the processing capability of the special effect processing component, and if so, determine that the original image meets the preset downsampling condition.

[0116] Further, based on any of the above embodiments, the original image is a live video frame corresponding to the live broadcast operation. The downsampling module is used to: determine whether the live video frame is a video frame generated in a microphone connection scenario. If so, it is determined that the live video frame does not meet the downsampling condition. If not, it is determined that the live video frame meets the downsampling condition.

[0117] Further, based on any of the above embodiments, the downsampling module is used to: obtain a first texture map corresponding to the original image, and render the first texture map into a second texture map corresponding to the target resolution to obtain the image to be processed.

[0118] Further, based on any of the above embodiments, the device further comprises: a display module, configured to display a preset rendering page; a processing module, configured to display a parameter adjustment control in response to a setting operation triggered by a user on the rendering page; and a construction module, configured to construct the target rendering window based on the adjustment parameters associated with the target resolution generated by the user in the parameter adjustment control.

[0119] Further, based on any of the above embodiments, the adjustment module is used to: perform image processing on the rendered image based on a preset super-resolution algorithm to obtain the target image.

[0120] Further, based on any of the above embodiments, the target image includes a portrait area. The rendering module is used to: identify the portrait area in the image to be processed. Use the target rendering window to perform skin smoothing rendering operation and / or beauty rendering operation and / or background sticker rendering operation and / or deformation rendering operation on the portrait area.

[0121] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0122] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image rendering method described in any of the above embodiments is implemented.

[0123] In order to implement the above embodiments, the embodiments of the present disclosure further provide a computer program product, including a computer program, and when the computer program is executed by a processor, the image rendering method as described in any of the above embodiments is implemented.

[0124] In order to implement the above embodiment, the present disclosure also provides an electronic device, including: a processor and a memory;

[0125] The memory stores computer-executable instructions;

[0126] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the image rendering method as described in any of the above embodiments.

[0127] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as Figure 7 As shown, the electronic device 700 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (Portable Media Players, PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0128] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0129] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0130] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0131] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0132] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0133] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0134] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).

[0135] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0137] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an image rendering method, comprising:

[0140] Obtaining an image rendering request, wherein the image rendering request includes an original image and a special effect identifier;

[0141] Performing a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image;

[0142] Performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result;

[0143] Using a target rendering window that matches the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image;

[0144] The resolution of the rendered image is adjusted to the original resolution to obtain a target image.

[0145] According to one or more embodiments of the present disclosure, performing a downsampling operation on the original image according to the image rendering request includes:

[0146] Determining whether the original image meets a preset downsampling condition;

[0147] If so, a downsampling operation is performed on the original image according to the image rendering request.

[0148] According to one or more embodiments of the present disclosure, determining whether the original image meets a preset downsampling condition includes:

[0149] Determine an original resolution corresponding to the original image;

[0150] Determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component;

[0151] If so, it is determined that the original image meets the preset downsampling condition.

[0152] According to one or more embodiments of the present disclosure, the original image is a live video frame corresponding to the live broadcast operation;

[0153] After determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component, the method further includes:

[0154] Determine whether the live video frame is a video frame generated in a microphone connection scenario;

[0155] If yes, it is determined that the live video frame does not meet the downsampling condition;

[0156] If not, it is determined that the live video frame meets the downsampling condition.

[0157] According to one or more embodiments of the present disclosure, performing a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution includes:

[0158] Acquire a first texture map corresponding to the original image;

[0159] The first texture map is rendered into a second texture map corresponding to the target resolution to obtain the image to be processed.

[0160] According to one or more embodiments of the present disclosure, before the target rendering window matching the target resolution is used to render the image to be processed based on the algorithm processing result to obtain the rendered image, the method further includes:

[0161] Display the preset rendering page;

[0162] In response to a setting operation triggered by a user on the rendering page, displaying a parameter adjustment control;

[0163] The target rendering window is constructed based on the adjustment parameters associated with the target resolution generated by the user in the parameter adjustment control.

[0164] According to one or more embodiments of the present disclosure, adjusting the resolution of the rendered image to the original resolution to obtain a target image includes:

[0165] The rendered image is processed based on a preset super-resolution algorithm to obtain the target image.

[0166] According to one or more embodiments of the present disclosure, the target image includes a portrait area;

[0167] The step of using a target rendering window that matches the target resolution to render the image to be processed based on the algorithm processing result to obtain a rendered image includes:

[0168] Identifying a portrait area in the image to be processed;

[0169] The target rendering window is used to perform skin smoothing rendering operation and / or beauty makeup rendering operation and / or background sticker rendering operation and / or deformation rendering operation on the portrait area.

[0170] In a second aspect, according to one or more embodiments of the present disclosure, there is provided an image rendering device, comprising:

[0171] An acquisition module, used for acquiring an image rendering request, wherein the image rendering request includes an original image and a special effect identifier;

[0172] A downsampling module, configured to perform a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image;

[0173] A processing module, used for performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result;

[0174] A rendering module, used to use a target rendering window matching the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image;

[0175] The adjustment module is used to adjust the resolution of the rendered image to the original resolution to obtain a target image.

[0176] According to one or more embodiments of the present disclosure, the downsampling module is used to:

[0177] Determining whether the original image meets a preset downsampling condition;

[0178] If so, a downsampling operation is performed on the original image according to the image rendering request.

[0179] According to one or more embodiments of the present disclosure, the downsampling module is used to:

[0180] Determine an original resolution corresponding to the original image;

[0181] Determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component;

[0182] If so, it is determined that the original image meets the preset downsampling condition.

[0183] According to one or more embodiments of the present disclosure, the original image is a live video frame corresponding to the live broadcast operation;

[0184] The down sampling module is used for:

[0185] Determine whether the live video frame is a video frame generated in a microphone connection scenario;

[0186] If yes, it is determined that the live video frame does not meet the downsampling condition;

[0187] If not, it is determined that the live video frame meets the downsampling condition.

[0188] According to one or more embodiments of the present disclosure, the downsampling module is used to:

[0189] Acquire a first texture map corresponding to the original image;

[0190] The first texture map is rendered into a second texture map corresponding to the target resolution to obtain the image to be processed.

[0191] According to one or more embodiments of the present disclosure, the device further includes:

[0192] Display module, used to display the preset rendering page;

[0193] A processing module, configured to display a parameter adjustment control in response to a setting operation triggered by a user on the rendering page;

[0194] A construction module is used to construct the target rendering window based on the adjustment parameters associated with the target resolution generated by the user in the parameter adjustment control.

[0195] According to one or more embodiments of the present disclosure, the adjustment module is used to:

[0196] The rendered image is processed based on a preset super-resolution algorithm to obtain the target image.

[0197] According to one or more embodiments of the present disclosure, the target image includes a portrait area;

[0198] The rendering module is used to:

[0199] Identifying a portrait area in the image to be processed;

[0200] The target rendering window is used to perform skin smoothing rendering operation and / or beauty makeup rendering operation and / or background sticker rendering operation and / or deformation rendering operation on the portrait area.

[0201] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0202] The memory stores computer-executable instructions;

[0203] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image rendering method described in the first aspect and various possible designs of the first aspect.

[0204] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the image rendering method described in the first aspect and various possible designs of the first aspect is implemented.

[0205] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the image rendering method described in the first aspect and various possible designs of the first aspect.

[0206] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0207] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0208] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image rendering method, characterized in that: include: Obtaining an image rendering request, wherein the image rendering request includes an original image and a special effect identifier; Performing a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image; Performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result; Using a target rendering window that matches the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image; The resolution of the rendered image is adjusted to the original resolution to obtain a target image.

2. The method according to claim 1, characterized in that The downsampling operation on the original image according to the image rendering request comprises: Determining whether the original image meets a preset downsampling condition; If so, a downsampling operation is performed on the original image according to the image rendering request.

3. The method according to claim 2, characterized in that The determining whether the original image satisfies a preset downsampling condition comprises: Determine an original resolution corresponding to the original image; Determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component; If so, it is determined that the original image meets the preset downsampling condition.

4. The method according to claim 3, characterized in that The original image is a live video frame corresponding to the live broadcast operation; After determining whether the original resolution is greater than a preset resolution threshold, or determining whether the original resolution exceeds the processing capability of the special effect processing component, the method further includes: Determine whether the live video frame is a video frame generated in a microphone connection scenario; If yes, it is determined that the live video frame does not meet the downsampling condition; If not, it is determined that the live video frame meets the downsampling condition.

5. The method according to claim 1, characterized in that The downsampling operation is performed on the original image according to the image rendering request to obtain an image to be processed with a target resolution, including: Acquire a first texture map corresponding to the original image; The first texture map is rendered into a second texture map corresponding to the target resolution to obtain the image to be processed.

6. The method according to any one of claims 1 to 5, characterized in that: Before the target rendering window matching the target resolution is used to render the image to be processed based on the algorithm processing result to obtain the rendered image, the method further includes: Display the preset rendering page; In response to a setting operation triggered by a user on the rendering page, displaying a parameter adjustment control; The target rendering window is constructed based on the adjustment parameters associated with the target resolution generated by the user in the parameter adjustment control.

7. The method according to any one of claims 1 to 5, characterized in that: The step of adjusting the resolution of the rendered image to the original resolution to obtain a target image comprises: The rendered image is processed based on a preset super-resolution algorithm to obtain the target image.

8. The method according to any one of claims 1 to 5, characterized in that: The target image includes a portrait area; The step of using a target rendering window that matches the target resolution to render the image to be processed based on the algorithm processing result to obtain a rendered image includes: Identifying a portrait area in the image to be processed; The target rendering window is used to perform skin smoothing rendering operation and / or beauty makeup rendering operation and / or background sticker rendering operation and / or deformation rendering operation on the portrait area.

9. An image rendering device, characterized in that: include: An acquisition module, used for acquiring an image rendering request, wherein the image rendering request includes an original image and a special effect identifier; A downsampling module, configured to perform a downsampling operation on the original image according to the image rendering request to obtain an image to be processed with a target resolution, wherein the target resolution is lower than an original resolution corresponding to the original image; A processing module, used for performing data processing on the image to be processed by using a special effect algorithm corresponding to the special effect identifier to obtain an algorithm processing result; A rendering module, used to use a target rendering window matching the target resolution to perform a rendering operation on the image to be processed based on the algorithm processing result to obtain a rendered image; The adjustment module is used to adjust the resolution of the rendered image to the original resolution to obtain a target image.

10. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image rendering method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the image rendering method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image rendering method according to any one of claims 1 to 8 is implemented.

Images