Multi-target rendering method and device

Abstract

The embodiment of the invention provides a multi-target rendering method, and the method comprises the steps: obtaining a plurality of target objects corresponding to a target video frame, and each target object is associated with identification information; according to the identification information of each target object, obtaining a measurement size and on-screen information of each target object; determining a rendering size and a rendering area of each target object according to the measurement size of each target object and a preset target canvas; according to the rendering size and the rendering area of each target object, drawing the plurality of target objects into the target canvas to obtain a target texture; and according to the target texture and the on-screen information of each target object, rendering each target object to the target video frame. According to the technical scheme provided by the embodiment of the invention, the rendering time consumption and the frame dropping frequency can be greatly reduced, and the batch rendering fluency of the multiple target objects is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a multi-target rendering method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] Video bullet screens, online games and other scenes display content through real-time batch rendering. Taking video bullet screens as an example, in order to achieve the motion effect of bullet screens, the position of bullet screens on the screen needs to be updated all the time, that is, the bullet screens of each frame need to be rendered in real time. To ensure the smoothness of the movement of bullet screens, it is required to complete the rendering of all bullet screens in each frame within the time limit.

[0003] However, when too many bullet comments need to be rendered per frame, the rendering time is prolonged and frames are easily dropped, resulting in stuttering of the bullet comment movement and a decline in user experience.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] Embodiments of the present application provide a multi-target rendering method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a multi-target rendering method, the method comprising: Acquire multiple target objects corresponding to the target video frame, each of the target objects being associated with identification information; According to the identification information of each target object, obtaining the measurement size and on-screen information of each target object; Determining a rendering size and a rendering area of ​​each target object according to a measured size of each target object and a preset target canvas; According to the rendering size and rendering area of ​​each target object, the multiple target objects are drawn into the target canvas to obtain a target texture; Each of the target objects is rendered onto the target video frame according to the target texture and the on-screen information of each of the target objects.

[0007] Optionally, the identification information includes a rendering identification and a typesetting identification; Correspondingly, according to the identification information of each target object, the measured size and on-screen information of each target object is obtained, including: According to the rendering identifier of each target object, obtaining the measured size of each target object by calling a matched renderer; According to the typesetting identifier of each target object, the on-screen information of each target object is obtained by calling the matched typesetter.

[0008] Optionally, the renderer includes a special effects renderer and a text renderer; Correspondingly, the measured size of each target object is obtained by calling the matched renderer, including: In a case where the matched renderer is a special effects renderer, obtaining auxiliary rendering resources associated with the special effects renderer; In the case where the auxiliary rendering resource has been acquired, acquiring the measured size of the corresponding target object for rendering by calling the special effect renderer; In the case where the auxiliary rendering resource is not obtained, the measured size of the corresponding target object is obtained by calling a preset text renderer for degraded rendering.

[0009] Optionally, acquiring the on-screen information of each target object by calling the matched typesetter includes: Determining a target time corresponding to the target video frame; Obtaining the available space of the preset display area within the target time; The upper screen information of each target object is acquired according to the available space of the preset display area within the target time and the measured size of each target object.

[0010] Optionally, determining the rendering size and rendering area of ​​each target object according to the measured size of each target object and a preset target canvas includes: For the target object whose corresponding upper screen information is obtained, the rendering size and rendering area are determined according to the measured size and the preset target canvas.

[0011] Optionally, determining the rendering size and rendering area according to the measured size and a preset target canvas includes: Get the size and available layout space of the target canvas; Converting the measured size of the target object into a rendering size according to the size of the target canvas; Determining a target affine transformation matrix according to the available layout space of the target canvas; The target canvas is adjusted according to the target affine transformation matrix to determine a rendering area of ​​the target object.

[0012] Optionally, converting the measured size of the target object into a rendering size according to the size of the target canvas includes: If the measured size of the target object is not greater than the size of the target canvas, determining the measured size of the target object as the rendering size; When the measured size of the target object is larger than the size of the target canvas, the measured size of the target object is cropped according to the size of the target canvas to obtain a first rendering size and a second rendering size, where the first rendering size corresponds to the size of the target canvas.

[0013] Optionally, according to the rendering size and rendering area of ​​each of the target objects, the plurality of target objects are drawn into the target canvas to obtain a target texture, including: When the rendering size of the target object is one, drawing the target object into the target canvas according to the rendering size and the rendering area of ​​the target object; In the case where the rendering size of the target object is not one: Acquire an auxiliary canvas, where the auxiliary canvas is larger than or equal to the measured size of the target object; Drawing the target object into the auxiliary canvas to obtain the original texture; According to the first rendering size and the second rendering size, the original texture is cropped to obtain a first cropped texture and a second cropped texture; The first clipping texture and the second clipping texture are drawn to the target canvas according to the rendering area of ​​the target object.

[0014] Another aspect of an embodiment of the present application provides a multi-target rendering device, the device comprising: A first acquisition module is used to acquire multiple target objects corresponding to the target video frame, each of the target objects is associated with identification information; A second acquisition module, used for acquiring the measurement size and on-screen information of each target object according to the identification information of each target object; An arrangement module, used for determining a rendering size and a rendering area of ​​each target object according to a measured size of each target object and a preset target canvas; A drawing module, used for drawing the multiple target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain a target texture; A rendering module is used to render each of the target objects onto the target video frame according to the target texture and the on-screen information of each of the target objects.

[0015] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0016] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0017] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0018] The above technical solution adopted in the embodiment of the present application may have the following advantages: A unified rendering process is performed for each target object, including: obtaining the measured size and on-screen information according to the associated identification information. Determine the rendering size and rendering area according to the measured size and the preset target canvas. Based on the rendering size and the rendering area, draw the current target object onto the target canvas. Execute the aforementioned rendering process until all the target objects required to be rendered by the target video frame are drawn onto the target canvas to generate the target texture. Based on the target texture and the on-screen information of each target object, render each target object onto the target video frame. It can be seen that the embodiment of the present application can integrate multiple target objects on the same texture resource by uniformly scheduling the rendering process of multiple target objects, reduce the frequency of texture switching when drawing on the screen, greatly reduce the rendering time and the number of frame drops, improve the smoothness of batch rendering of multiple target objects, and realize the smooth display of multiple target objects on the same screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 A flowchart of a multi-target rendering method according to Embodiment 1 of the present application is schematically shown; Figure 2 Schematically shows Figure 1 Flow chart of sub-steps of step S102; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S200; Figure 4Schematically shows Figure 2 Flow chart of sub-steps of step S202; Figure 5 A flowchart of sub-steps of step S500 of the multi-target rendering method according to the first embodiment of the present application is schematically shown; Figure 6 Schematically shows Figure 5 Flow chart of sub-steps of step S602; Figure 7 Schematically shows Figure 1 Flow chart of sub-steps of step S106; Figure 8 Another flowchart of the multi-target rendering method according to the first embodiment of the present application is schematically shown; Fig. 9 A block diagram schematically shows a multi-target rendering device according to the second embodiment of the present application; and Fig.10 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0022] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0024] First, the following terms are explained: HWC (Hardware Composer): Hardware compositor, a module used to handle graphics rendering in the Android operating system.

[0025] GPU (Graphics Processing Unit): Graphics processing unit, a module in a computer system used to accelerate graphics and video rendering and processing.

[0026] CPU (Central Processing Unit): Central processing unit, the control center of the computer, a module responsible for executing program instructions and processing data.

[0027] Frame: A single static graphic in a video sequence or animation. A video or animation consists of a series of consecutive frames, each of which is a still image, which creates a visually continuous dynamic effect when played continuously at high speed.

[0028] px (pixel): screen pixel.

[0029] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: Scenes such as video bullet screens and online games display content through real-time batch rendering. Taking video bullet screens as an example, the movement of bullet screens causes their position in the screen display window to change all the time, and almost every frame is different from the previous frame. The high-speed switching of these frames can produce a visual sense of bullet screen movement. In order to keep the entire bullet screen moving smoothly without lag, the computer's CPU, GPU, HWC and other hardware are required to cooperate with each other to complete the measurement and rasterization drawing of the full-screen bullet screen within the time limit of each frame. For example, for a screen device with a refresh rate of 60Hz, the time limit for each frame is about 16ms.

[0030] The applicant understands that the relevant technology can treat each text (such as bullet comments) as an independent rendering target, measure and rasterize each text independently, and correspond to independent hardware resources in the computer, forming a fragmented texture resource pool. When the screen is refreshed, the GPU will calculate the drawing instructions, bind the drawing texture target, and repeatedly switch between different texture targets for rendering until all the texts required to be rendered in the current frame are rendered and the rendering results are submitted to the display device screen.

[0031] However, the independent texture resource pool for each text (render target) has the following disadvantages: when there are too many render targets that need to be redrawn in each frame, devices with insufficient hardware (GPU, HWC) performance cannot complete all the drawing work within the time limit of each frame, causing timeouts. Timeouts will cause the current frame to occupy the drawing time of the next frame. When the time of a subsequent frame is completely occupied, the frame will be skipped and its image will not be displayed by the display device, causing the rendering target motion animation to freeze and the user experience to deteriorate.

[0032] To this end, the embodiment of the present application provides a multi-target rendering technology solution. In this technical solution: (1) Ensure the smoothness of batch text animation effects, improve the user's barrage viewing experience while ensuring full-screen overlapping barrages, and improve the image processing and rendering efficiency on low-performance hardware devices; (2) Reduce the average time consumption per frame: control the frame consumption of more than 90% to less than half of the limited time per frame. For example, for a screen device with a refresh rate of 60Hz, the limited time for each frame is 16ms, and the average time consumption per frame is controlled within 8ms; (3) Reduce the number of frame drops per unit time: control the number of time-out rendering frames within a unit time (such as per minute) (such as a screen device with a refresh rate of 60Hz, the limited time for each frame is about 16ms, and the frame consumption exceeds 16ms) to within 1%; (4) The rendering process of each target object is uniformly scheduled, and each target object is rendered on the same canvas through target measurement, typesetting / layout, target drawing and other links. This breaks the limitation of calling GPU resources to generate corresponding textures immediately after the rendering of a single target object is completed. Only after the canvas resources are filled or the rendering of multiple targets is completed, the GPU resources need to be called once to generate the synthetic texture (target texture), which greatly reduces the system's occupation of GPU resources. At the actual moment of drawing on the screen, because the number of target textures to be drawn is reduced, the frequency of texture switching during GPU drawing is reduced. When there are many target objects to be drawn in each frame, the smoothness of animated text rendering on low-performance devices can be greatly improved.

[0033] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described here.

[0034] Embodiment 1 Figure 1 The flowchart of the multi-target rendering method according to the first embodiment of the present application is schematically shown.

[0035] like Figure 1 As shown, the multi-target rendering method may include steps S100 to S108, wherein: Step S100: acquiring a plurality of target objects corresponding to a target video frame, each of the target objects being associated with identification information.

[0036] Step S102: acquiring the measurement size and on-screen information of each target object according to the identification information of each target object.

[0037] Step S104: determining a rendering size and a rendering area of ​​each target object according to the measured size of each target object and a preset target canvas.

[0038] Step S106: drawing the multiple target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain a target texture.

[0039] Step S108: Render each of the target objects onto the target video frame according to the target texture and the on-screen information of each of the target objects.

[0040] The multi-target rendering method provided in this embodiment executes a unified rendering process for each target object, including: obtaining the measured size and on-screen information according to the associated identification information. Determine the rendering size and rendering area according to the measured size and the preset target canvas. Based on the rendering size and the rendering area, draw the current target object onto the target canvas. Execute the aforementioned rendering process until all the target objects required to be rendered by the target video frame are drawn onto the target canvas to generate the target texture. Based on the target texture and the on-screen information of each target object, render each target object onto the target video frame. It can be seen that the embodiment of the present application can integrate multiple target objects on the same texture resource by uniformly scheduling the rendering process of multiple target objects, reduce the texture switching frequency when drawing on the screen, greatly reduce the rendering time and the number of frame drops, improve the rendering smoothness of multiple target objects, and realize the smooth display of multiple target objects on the same screen.

[0041] The following combination Figure 1 and Figure 8 , each step in steps S100~S108 and other optional steps are explained in detail.

[0042] Step S100 , obtaining multiple target objects corresponding to the target video frame, each of the target objects being associated with identification information.

[0043] The target object can be various display contents in different scenes. For example, in the video bullet screen scene, the target object can be a bullet screen. For online game scenes, the target object can be game elements, such as characters, props, special effects, etc. The following will take the video bullet screen scene as an example to exemplify the multi-target rendering method provided in the embodiment of the present application.

[0044] Get multiple bullet comments corresponding to the target video frame. Bullet comments can be divided into text bullet comments and special effect bullet comments. Special effect bullet comments can be a combination of text, pictures or animations. Special effect bullet comments can also be divided into fireworks bullet comments, Lantern Festival bullet comments, emoticon bullet comments, confrontation (PK) bullet comments, etc. Each bullet comment can have a unique rendering template paradigm, which is used to render the bullet comments of this type into a specific texture effect. Each bullet comment can also have a unique upper screen distribution method, that is, the layout and movement method that the audience can feel. For example: the bullet comment can move from the right side of the screen to the left side, from bottom to top, or two bullet comments can appear from the left and right sides of the screen respectively, and disappear in the middle of the screen after colliding. According to actual conditions, identification information can be pre-configured for each bullet comment. The identification information may include a rendering identifier and a layout identifier. Among them, the rendering identifier can be used to obtain a specific renderer from a preset set of renderers. Each renderer can have different rendering template paradigms built in to render the bullet comment into different texture effects. The layout identifier is used to obtain a specific typeset in a preset set of typeset. Each layout device can have different built-in on-screen modes to determine the layout and movement of the bullet comment on the device screen.

[0045] Step S102 , according to the identification information of each target object, obtain the measured size and on-screen information of each target object.

[0046] The measured size may include the length, height, etc. of the target object. The on-screen information may be used to determine whether the corresponding target object can be displayed on the device screen. Figure 8 A set of multi-target synthesis renderers is shown to improve rendering efficiency. In some embodiments, the multi-target synthesis renderer can be arranged in the UI (user interface) thread, or it can be arranged in other threads to further reduce the blocking of the UI thread by the rendering link. After the processing is completed, the rendering result (target texture) is returned to the UI thread to organize the on-screen operation, and the target texture is synchronized between different threads. The multi-target synthesis renderer can obtain the rendering identification (rendering ID) and typesetting identification (typesetting ID) corresponding to the target object according to the identification information of the target object. According to the rendering ID and typesetting ID, a specific renderer and typeseter can be determined from the preset multiple renderers and typeseters. Among them, the preset multiple renderers can follow a unified external interface for the multi-target synthesis renderer to call, and the internal implementation of each renderer is different. Both the renderer and the typeseter can be created and destroyed by the calling module of the multi-target synthesis renderer, and injected into the multi-target synthesis renderer to work.

[0047] In this embodiment, the multi-target synthesis renderer can determine the measurement size and on-screen information of the target object by calling a specific renderer and typesetter. An exemplary embodiment is provided below.

[0048] In an optional embodiment, if Figure 2 As shown, step S102 may include: Step S200 : acquiring the measured size of each target object by calling a matched renderer according to the rendering identifier of each target object.

[0049] Step S202, according to the typesetting identifier of each target object, obtain the on-screen information of each target object by calling the matched typesetter.

[0050] Exemplarily, according to the rendering identifier, the matched renderer is called to measure the target object to obtain the measured size (length, height, etc.) of the target object. According to the layout identifier, the matched layout device is called to determine the on-screen layout mode of the target object, and the on-screen information of the target object is determined in combination with the actual display situation of the device screen (or the preset display area).

[0051] In this embodiment, by obtaining a specific renderer and typesetter through identification information, all target objects can be measured quickly and accurately, and the process is unified, which can further improve the efficiency of image rendering processing.

[0052] In an optional embodiment, the renderer may include a special effects renderer and a text renderer. Figure 3 As shown, step S200 may include: Step S300: when the matched renderer is a special effects renderer, obtain auxiliary rendering resources associated with the special effects renderer.

[0053] Step S302: when the auxiliary rendering resource has been acquired, the measurement size of the corresponding target object is acquired for rendering by calling the special effect renderer.

[0054] Step S304: if the auxiliary rendering resource is not obtained, the measured size of the corresponding target object is obtained by calling a preset text renderer for degraded rendering.

[0055] The special effect renderer can be a pure image, a pure text with special colors and font styles, or a mixed text and image. For the special effect renderer, its built-in rendering template paradigm may need to use network images and animation resources for auxiliary rendering. When these auxiliary rendering resources are not ready (not downloaded), the special effect renderer cannot work properly, resulting in extended rendering time. Therefore, when the renderer matched by the target object is a special effect renderer, you can first retrieve whether the associated auxiliary rendering resources are stored locally. If so, the multi-target synthesis renderer can directly call the special effect renderer to measure the target object. If there is no corresponding auxiliary rendering resource and it is not successfully downloaded within the preset time, it can return the result that the rendering failed but the target object is expected to be downgraded. In order to optimize the rendering effect and improve the rendering efficiency, the multi-target synthesis renderer can re-select the default built-in text renderer to measure the target object and return the measured size for downgraded rendering. Of course, you can also choose not to downgrade, and the corresponding target object directly exits the rendering process to reduce resource waste, rendering conflicts and other problems.

[0056] In this embodiment, by configuring a degradation processing strategy, reselecting a renderer for measuring the target object, or directly exiting the rendering process, unnecessary resource waste can be reduced, rendering conflicts can be avoided, and rendering efficiency can be further optimized.

[0057] In an optional embodiment, if Figure 4 As shown, step S202 may include: Step S400, determining a target time corresponding to the target video frame.

[0058] Step S402: obtaining the available space of the preset display area within the target time.

[0059] Step S404: acquiring the upper screen information of each target object according to the available space of the preset display area within the target time and the measured size of each target object.

[0060] Exemplarily, the playback time corresponding to the target video frame can be obtained (the current frame is the current moment). The available space of the preset display area (video playback area) within the playback time of the target video frame is determined. According to the available space of the preset display area and the measured size of the target object, it is determined whether there is enough space in the preset display area within the playback time of the target video frame for the target object to be displayed (on the screen), and the corresponding on-screen information is generated. The on-screen information can be used to indicate that the target object is successfully typeset.

[0061] In this embodiment, based on the available space of the preset display area and the measured size of the target object, as well as the built-in on-screen layout method of the typesetter, the layout of all target objects can be achieved quickly and accurately, and the process is unified, which can further improve the efficiency of image rendering processing.

[0062] Step S104 , determining the rendering size and rendering area of ​​each target object according to the measured size of each target object and a preset target canvas.

[0063] The multi-target synthesis renderer can specify a unique target canvas for drawing all target objects, and the right to use the target canvas will be handed over to a specific renderer during the measurement and drawing stages. Of course, if the number of target objects corresponding to the target video frame is too large, the size of the target canvas is limited, and the target canvas can also be greater than one, and each target canvas is used to draw multiple target objects. The size of the target canvas can be set according to GPU performance, device screen size, etc., for example: 2048px×2048px. Exemplarily, the measured size of the target object can be compared with the preset target canvas to determine whether the measured size needs to be cropped, rotated, etc., so as to convert the measured size into the rendering size to ensure correct rendering. Since the size and length of each target object are uncertain, the layout can also be optimized according to the resources and space usage of the target canvas, that is, the global optimal rendering area (the drawing area on the target canvas) is determined for each target object.

[0064] In this embodiment, the rendering size and rendering area of ​​the target object are determined according to the measured size of the target object and the preset target canvas, so as to accurately control the target object to be correctly drawn into the canvas.

[0065] A number of embodiments are provided below for further exemplary description.

[0066] In an optional embodiment, step S104 may include: Step S500: for the target object corresponding to the upper screen information obtained, the rendering size and rendering area are determined according to the measured size and the preset target canvas.

[0067] As mentioned above, the on-screen information can indicate that the target object is successfully laid out. For the target object that has obtained the on-screen information, the subsequent layout, drawing, and rendering processes can be performed to successfully display the successfully laid out target object on the device screen. Failure to obtain the corresponding on-screen information indicates that the layout has failed, and the corresponding target object can directly exit the rendering process to reduce the waste of canvas resources.

[0068] In this embodiment, the target object with on-screen information (successful layout) continues to be rendered, and the target object with failed layout is directly exited from the rendering process, which can reduce unnecessary resource waste, avoid layout conflicts, and further improve rendering efficiency and rendering effects.

[0069] In an optional embodiment, if Figure 5 As shown, step S500 may include: Step S600: obtaining the size and available layout space of the target canvas.

[0070] Step S602: converting the measured size of the target object into a rendering size according to the size of the target canvas.

[0071] Step S604: determining a target affine transformation matrix according to the available layout space of the target canvas.

[0072] Step S606: adjusting the target canvas according to the target affine transformation matrix to determine a rendering area of ​​the target object.

[0073] Get the size of the target canvas, which may include the length and height of the target canvas. Get the available layout space of the target canvas, that is, the remaining layout space on the target canvas. There is a size limit when the system GPU generates the target texture, and the measured size of a single target object cannot exceed the size of the target canvas. Therefore, according to the size of the target canvas, the measured size of the target object can be cropped and spliced ​​so that the target object can be drawn into the target canvas to ensure subsequent correct rendering. In practical applications, a canvas arranger can also be configured to determine the rendering area of ​​the target object through the arrangement algorithm of the canvas arranger. Exemplarily, the target canvas can be gridded first, and the rendering area information of the target object that has been rendered in the previous order can be recorded to determine the remaining available layout space. According to the available layout space, the target affine transformation matrix of the target canvas is determined. According to the target affine transformation matrix, the canvas is rotated, translated, and other operations are performed so that the target object can be accurately rendered in the desired position, that is, the rendering area of ​​the target object.

[0074] In this embodiment, the affine matrix properties of the target canvas are fully utilized, and rotation and translation operations are performed through matrix transformation, so that the correct drawing of the current target object can be accurately controlled, and the remaining space of the target canvas can be reasonably used to achieve a better layout with maximum efficiency.

[0075] In an optional embodiment, if Figure 6 As shown, step S602 may include: Step S700: When the measured size of the target object is not greater than the size of the target canvas, determine the measured size of the target object as the rendering size.

[0076] Step S702 : if the measured size of the target object is larger than the size of the target canvas, the measured size of the target object is cropped according to the size of the target canvas to obtain a first rendering size and a second rendering size, wherein the first rendering size corresponds to the size of the target canvas.

[0077] Exemplarily, if the measured size of the target object is not larger than the size of the target canvas, the measured size can be determined as the rendering size. If the measured size of the target object is larger than the size of the target canvas, the measured size of the target object can be cropped according to the size of the target canvas, that is, segmented, to obtain a first rendering size and a second rendering size. Among them, the first rendering size can correspond to the size of the target canvas, and the second rendering size is the part that exceeds the size of the target canvas. For example: if the length of the target object exceeds the length of the target canvas, the length of the target object can be segmented according to the length of the target canvas, and the overlong part is determined as the second rendering size. In practical applications, the second rendering size can be further cropped to obtain multiple smaller rendering sizes.

[0078] In this embodiment, the target object is segmented and spliced ​​into the canvas size by cutting the measured size to obtain the actual rendering size for subsequent correct rendering.

[0079] Step S106 , according to the rendering size and rendering area of ​​each of the target objects, the multiple target objects are drawn into the target canvas to obtain a target texture.

[0080] In practical applications, multiple target objects can be drawn into the target canvas one by one according to the rendering size and rendering area of ​​each target object to obtain the target texture. In this embodiment, multiple target objects are rendered onto the same canvas, breaking the limitation that GPU resources need to be called immediately to generate the corresponding texture after the rendering of a single target object is completed. Only GPU resources need to be called once to generate the synthetic texture, i.e., the target texture, after the canvas resources are filled or the multi-target rendering is completed, which greatly reduces the occupation of the system GPU resources and effectively improves the rendering efficiency. An exemplary solution will be provided below.

[0081] In an optional embodiment, if Figure 7 As shown, step S106 may include: Step S800 : when the rendering size of the target object is one, draw the target object into the target canvas according to the rendering size and rendering area of ​​the target object.

[0082] Step S802, when the rendering size of the target object is not one: obtain an auxiliary canvas, the auxiliary canvas is larger than or equal to the measured size of the target object; draw the target object into the auxiliary canvas to obtain the original texture; according to the first rendering size and the second rendering size, crop the original texture to obtain a first cropped texture and a second cropped texture; according to the rendering area of ​​the target object, draw the first cropped texture and the second cropped texture into the target canvas.

[0083] Exemplarily, when the rendering size of the target object is the measured size, the target object can be directly drawn into the target canvas through the determined renderer according to the rendering size and rendering area of ​​the target object. When the rendering size of the target object is not equal to the measured size, an auxiliary canvas (or off-screen canvas) is obtained, and the size of the auxiliary canvas is larger than the measured size of the target object, so as to draw the target object into the auxiliary canvas and obtain the original texture map. According to the first rendering size and the second rendering size, the original texture map is cropped to obtain the first cropped texture and the second cropped texture. According to the rendering area of ​​the target object, the first cropped texture and the second cropped texture are drawn into the target canvas. In some embodiments, the cropping information can also be recorded and provided to the GPU, and the cropping information can be used to instruct the GPU how to extract all the cropped textures corresponding to the target object from the target texture when it is on the screen, so as to accurately restore the actual target object.

[0084] In this embodiment, target objects smaller than the target canvas size are directly drawn to the target canvas, and target objects larger than the target canvas size are cropped and then drawn to the target canvas, so that the correct drawing of each target object can be effectively controlled.

[0085] Step S108 , rendering each of the target objects onto the target video frame according to the target texture and the on-screen information of each of the target objects.

[0086] After the aforementioned steps (measurement, typesetting, arrangement, and drawing), the multi-target synthesis renderer can combine one or more target textures and the on-screen information and provide them to the GPU for actual screen use, that is, rendering multiple target objects onto the target video frame. For example, the GPU can call one or more target textures and the on-screen information combination to generate a bullet screen layer to ensure that the bullet screen is correctly displayed on the target video frame.

[0087] In this embodiment, the integrated texture (target texture) greatly reduces the number of texture switching times when the GPU renders on the screen, reduces the rendering pressure of the GPU and HWC (on Android devices), and makes the animated text that needs to be updated in real time more fluid, effectively improving the user experience.

[0088] Embodiment 2 Fig. 9 The block diagram of the multi-target rendering device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig. 9 As shown, the device 1000 may include: a first acquisition module 1100, a second acquisition module 1200, an arrangement module 1300, a drawing module 1400, and a rendering module 1500, wherein: A first acquisition module 1100 is used to acquire a plurality of target objects corresponding to a target video frame, each of the target objects being associated with identification information; The second acquisition module 1200 is used to acquire the measurement size and on-screen information of each target object according to the identification information of each target object; Arrangement module 1300, used to determine the rendering size and rendering area of ​​each target object according to the measured size of each target object and a preset target canvas; A drawing module 1400 is used to draw the multiple target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain a target texture; The rendering module 1500 is used to render each of the target objects onto the target video frame according to the target texture and the on-screen information of each of the target objects.

[0089] As an optional embodiment, the identification information includes a rendering identification and a typesetting identification; Correspondingly, according to the identification information of each target object, the measured size and on-screen information of each target object is obtained, including: According to the rendering identifier of each target object, obtaining the measured size of each target object by calling a matched renderer; According to the typesetting identifier of each target object, the on-screen information of each target object is obtained by calling the matched typesetter.

[0090] As an optional embodiment, the renderer includes a special effect renderer and a text renderer; Correspondingly, the measured size of each target object is obtained by calling the matched renderer, including: In a case where the matched renderer is a special effects renderer, obtaining auxiliary rendering resources associated with the special effects renderer; In the case where the auxiliary rendering resource has been acquired, acquiring the measured size of the corresponding target object for rendering by calling the special effect renderer; In the case where the auxiliary rendering resource is not obtained, the measured size of the corresponding target object is obtained by calling a preset text renderer for degraded rendering.

[0091] As an optional embodiment, the acquiring the on-screen information of each target object by calling the matched typesetter includes: Determining a target time corresponding to the target video frame; Obtaining the available space of the preset display area within the target time; The upper screen information of each target object is acquired according to the available space of the preset display area within the target time and the measured size of each target object.

[0092] As an optional embodiment, determining the rendering size and rendering area of ​​each target object according to the measured size of each target object and a preset target canvas includes: For the target object whose corresponding upper screen information is obtained, the rendering size and rendering area are determined according to the measured size and the preset target canvas.

[0093] As an optional embodiment, determining the rendering size and the rendering area according to the measured size and the preset target canvas includes: Get the size and available layout space of the target canvas; Converting the measured size of the target object into a rendering size according to the size of the target canvas; Determining a target affine transformation matrix according to the available layout space of the target canvas; The target canvas is adjusted according to the target affine transformation matrix to determine a rendering area of ​​the target object.

[0094] As an optional embodiment, converting the measured size of the target object into a rendering size according to the size of the target canvas includes: If the measured size of the target object is not greater than the size of the target canvas, determining the measured size of the target object as the rendering size; When the measured size of the target object is larger than the size of the target canvas, the measured size of the target object is cropped according to the size of the target canvas to obtain a first rendering size and a second rendering size, where the first rendering size corresponds to the size of the target canvas.

[0095] As an optional embodiment, drawing the multiple target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain the target texture includes: When the rendering size of the target object is one, drawing the target object into the target canvas according to the rendering size and the rendering area of ​​the target object; In the case where the rendering size of the target object is not one: Acquire an auxiliary canvas, where the auxiliary canvas is larger than or equal to the measured size of the target object; Drawing the target object into the auxiliary canvas to obtain the original texture; According to the first rendering size and the second rendering size, the original texture is cropped to obtain a first cropped texture and a second cropped texture; The first clipping texture and the second clipping texture are drawn to the target canvas according to the rendering area of ​​the target object.

[0096] Embodiment 3 Fig.10 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing a multi-target rendering method according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.10 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program codes of a multi-target rendering method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0097] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0098] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0099] It should be pointed out that Fig.10 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0100] In this embodiment, the multi-target rendering method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0101] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-target rendering method in the embodiment are implemented.

[0102] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the multi-target rendering method in the embodiment, etc. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0103] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0104] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0105] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-target rendering method, characterized in that: The method comprises: Acquire multiple target objects corresponding to the target video frame, each of the target objects being associated with identification information; According to the identification information of each target object, obtaining the measurement size and on-screen information of each target object; Determining a rendering size and a rendering area of ​​each target object according to a measured size of each target object and a preset target canvas; According to the rendering size and rendering area of ​​each target object, the multiple target objects are drawn into the target canvas to obtain a target texture; Each of the target objects is rendered onto the target video frame according to the target texture and the on-screen information of each of the target objects.

2. The method according to claim 1, characterized in that The identification information includes a rendering identification and a typesetting identification; Correspondingly, according to the identification information of each target object, the measured size and on-screen information of each target object is obtained, including: According to the rendering identifier of each target object, obtaining the measured size of each target object by calling a matched renderer; According to the typesetting identifier of each target object, the on-screen information of each target object is obtained by calling the matched typesetter.

3. The method according to claim 2, characterized in that The renderer includes a special effect renderer and a text renderer; Correspondingly, the measured size of each target object is obtained by calling the matched renderer, including: In a case where the matched renderer is a special effects renderer, obtaining auxiliary rendering resources associated with the special effects renderer; In the case where the auxiliary rendering resource has been acquired, acquiring the measured size of the corresponding target object for rendering by calling the special effect renderer; In the case where the auxiliary rendering resource is not obtained, the measured size of the corresponding target object is obtained by calling a preset text renderer for degraded rendering.

4. The method according to claim 2, characterized in that The obtaining the on-screen information of each target object by calling the matched typesetter includes: Determining a target time corresponding to the target video frame; Obtaining the available space of the preset display area within the target time; The upper screen information of each target object is acquired according to the available space of the preset display area within the target time and the measured size of each target object.

5. The method according to any one of claims 1 to 4, characterized in that: Determining the rendering size and rendering area of ​​each target object according to the measured size of each target object and a preset target canvas includes: For the target object whose corresponding upper screen information is obtained, the rendering size and rendering area are determined according to the measured size and the preset target canvas.

6. The method according to claim 5, characterized in that Determining the rendering size and rendering area according to the measured size and the preset target canvas includes: Get the size and available layout space of the target canvas; Converting the measured size of the target object into a rendering size according to the size of the target canvas; Determining a target affine transformation matrix according to the available layout space of the target canvas; The target canvas is adjusted according to the target affine transformation matrix to determine a rendering area of ​​the target object.

7. The method according to claim 6, characterized in that According to the size of the target canvas, converting the measured size of the target object into a rendering size, including: If the measured size of the target object is not greater than the size of the target canvas, determining the measured size of the target object as the rendering size; When the measured size of the target object is larger than the size of the target canvas, the measured size of the target object is cropped according to the size of the target canvas to obtain a first rendering size and a second rendering size, where the first rendering size corresponds to the size of the target canvas.

8. The method according to claim 7, characterized in that Drawing the plurality of target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain a target texture includes: When the rendering size of the target object is one, drawing the target object into the target canvas according to the rendering size and the rendering area of ​​the target object; In the case where the rendering size of the target object is not one: Acquire an auxiliary canvas, where the auxiliary canvas is larger than or equal to the measured size of the target object; Drawing the target object into the auxiliary canvas to obtain the original texture; According to the first rendering size and the second rendering size, the original texture is cropped to obtain a first cropped texture and a second cropped texture; The first clipping texture and the second clipping texture are drawn to the target canvas according to the rendering area of ​​the target object.

9. A multi-target rendering device, characterized in that: The device comprises: A first acquisition module is used to acquire multiple target objects corresponding to the target video frame, each of the target objects is associated with identification information; A second acquisition module, used for acquiring the measurement size and on-screen information of each target object according to the identification information of each target object; An arrangement module, used for determining a rendering size and a rendering area of ​​each target object according to a measured size of each target object and a preset target canvas; A drawing module, used for drawing the multiple target objects into the target canvas according to the rendering size and rendering area of ​​each target object to obtain a target texture; A rendering module is used to render each of the target objects onto the target video frame according to the target texture and the on-screen information of each of the target objects.

10. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images