Media processing method and device, equipment and medium

Abstract

The invention provides a media processing method and device, equipment and a medium. A specific embodiment of the method comprises the following steps: acquiring spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects; storing the spatial azimuth information as auxiliary information; based on the auxiliary information and the color information, performing fuzzy rendering processing to obtain the plurality of key frames; and determining a target media based on the plurality of key frames. According to the embodiment, in the fuzzy rendering processing process, processing can be directly carried out based on the auxiliary information, the calculation amount during rendering is reduced, the efficiency of special effect processing is improved, and the user experience is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, and particularly relates to a media processing method and device, equipment and medium. BACKGROUND

[0002] With the continuous development and improvement of digital media technologies such as videos, images, animations, etc., the application of digital media has become more and more extensive. For example, digital media is widely used in advertising, film, art, and new media fields, etc. Media not only brings many conveniences to people, but also adds more interest to people's life. In order to make the media more vivid, people usually process the existing media through technical means to obtain more vivid and expressive media. For example, the existing two-dimensional picture can be processed into a picture of multiple objects moving in three-dimensional space, and the moving objects in three-dimensional space can also be blurred to obtain special effect media. Therefore, an efficient media processing method is needed. SUMMARY

[0003] The present disclosure provides a media processing method, device, equipment and medium.

[0004] According to a first aspect, a media processing method is provided, the method comprising:

[0005] obtaining spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects;

[0006] storing the spatial orientation information as auxiliary information;

[0007] performing blur rendering processing based on the auxiliary information and the color information to obtain the plurality of key frames;

[0008] determining a target media based on the plurality of key frames.

[0009] According to a second aspect, a media processing device is provided, the device comprising:

[0010] an obtaining module configured to obtain spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects;

[0011] a storage module configured to store the spatial orientation information as auxiliary information;

[0012] a processing module configured to perform blur rendering processing based on the auxiliary information and the color information to obtain the plurality of key frames;

[0013] a determination module configured to determine a target media based on the plurality of key frames.

[0014] According to a third aspect, a computer readable storage medium is provided, the storage medium storing a computer program which, when executed by a processor, implements the method of any one of the first aspect.

[0015] According to a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the method of any one of the first aspect when executing the program.

[0016] Embodiments of the present disclosure provide technical solutions that can include the following beneficial effects:

[0017] The media processing method and device provided by the embodiments of the present disclosure obtain the spatial orientation information of the target objects in the key frames and the color information of the target objects. The spatial orientation information is stored as auxiliary information. Based on the auxiliary information and the color information, the blur rendering processing is performed to obtain the key frames, and the target media is determined based on the key frames. Since the spatial orientation information of the target objects in the key frames is obtained in advance before the blur rendering processing, and the spatial orientation information is stored as auxiliary information, the processing can be directly based on the auxiliary information during the blur rendering processing, thereby reducing the calculation amount during rendering, improving the special effect processing efficiency, and improving the user experience.

[0018] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a media processing application scenario diagram according to an exemplary embodiment of the present disclosure;

[0021] Figure 2 is a flowchart of a media processing method according to an exemplary embodiment of the present disclosure;

[0022] Figure 3 is a flowchart of another media processing method according to an exemplary embodiment of the present disclosure;

[0023] Figure 4A is a media processing scenario diagram according to an exemplary embodiment of the present disclosure;

[0024] Figure 4B is another media processing scenario diagram shown according to an example embodiment of the present disclosure;

[0025] Figure 5 is a block diagram of a media processing apparatus shown according to an example embodiment of the present disclosure;

[0026] Figure 6 is a schematic block diagram of an electronic device according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable persons skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be clearly and completely described below in conjunction with the drawings in the embodiments of the specification. Obviously, the described embodiments are only some of the embodiments of the specification, not all the embodiments. Based on the embodiments in the specification, all other embodiments obtained by persons skilled in the art without creative labor should be within the protection scope of the specification.

[0028] The following description refers to the accompanying drawings. Unless otherwise indicated, same numbers in different drawings indicate same or similar elements. The following example embodiments described are not meant to represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0029] The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in the present disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0030] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used only to distinguish different sets of information from one another. For example, without departing from the scope of the present disclosure, a first information can also be termed a second information, and similarly, a second information can also be termed a first information. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination."

[0031] With the continuous development and improvement of digital media technologies such as video, image, animation and the like, the application of digital media is becoming more and more extensive. For example, digital media is widely used in advertising, film, art and new media fields. Media not only brings many conveniences to people, but also adds more interest to people's life. In order to make the media more vivid, people usually process the existing media through technical means to obtain more vivid and expressive media. For example, the existing two-dimensional picture can be processed into a picture of multiple objects moving in three-dimensional space, and the moving objects in three-dimensional space can also be blurred to obtain a more realistic special effect media.

[0032] In the related art, it is usually necessary to first model the space of the moving object, calculate the trajectory of the object in the model space when the object moves, and then render the key frame according to the trajectory of the object in the model space when the object moves, so as to generate a special effect media based on the key frame. However, when rendering directly according to the trajectory of the object in the model space, the calculation is complex and the amount of calculation is large, which makes the special effect processing efficiency low.

[0033] The media processing method provided by the present disclosure stores the spatial orientation information of a plurality of moving objects in a plurality of key frames as image-form data, performs blur rendering processing according to the image-form data, obtains a plurality of key frames, and generates a special effect media based on the plurality of key frames, thereby reducing the amount of calculation during rendering, improving the efficiency of special effect processing, and improving the user experience.

[0034] Referring to Figure 1 , a processing application scenario diagram of a media according to an example embodiment is shown.

[0035] As Figure 1 shown, the specific application scenario of the embodiment can be that, first, a model space is constructed, and a plurality of objects to be moved 101 are constructed in the model space, wherein different objects have different motion trajectories. Then, the motion trajectory of each object can be calculated, so that these objects are gathered and spliced into a connected area by rotating and moving in the model space from scattered positions scattered in the model space. In addition, a background image 102 is selected by a user, and part of the pixel points in the background image 102 are mapped onto a plurality of objects according to a preset mapping relationship to obtain objects 103 capable of displaying part of the background image.

[0036] Then, the spatial orientation information of each object in a plurality of to-be-generated key frames can be obtained based on the motion trajectory of each object, and the spatial orientation information of each object in the plurality of to-be-generated key frames is stored as image-form data. In addition, color information for rendering can be obtained based on the object 103, and the color information can include a correspondence between any point on the object and a color value.

[0037] Finally, based on the above image-form data and color information for rendering, a blur rendering process can be performed to obtain a plurality of key frames 104, and the target animation can be generated using the plurality of key frames 104. In the target animation, part of the picture in the background image 102 follows the object, moves in rotation at scattered positions scattered in the model space, and is finally spliced into a connected picture.

[0038] The present disclosure will be described in detail below with reference to specific embodiments.

[0039] Figure 2 A flowchart of a processing method of a media according to an exemplary embodiment is shown. The method can be applied in a terminal device. In this embodiment, for the convenience of understanding, the terminal device capable of installing third-party applications is taken as an example. Those skilled in the art can understand that the terminal device can include but is not limited to mobile terminal devices such as smart phones, smart wearable devices, tablet computers, desktop computers, and the like. The method can include the following steps:

[0040] As shown in FIG. 1, in step 201, spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects are obtained. Figure 2

[0041] In this embodiment, the media data to be processed can be obtained first, and the media data to be processed can include spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects. The target object can be a to-be-rendered object moving in the target media to be generated, and the key frame can be a special effect image frame used to generate the target media, each key frame corresponding to a time point on a time axis. The spatial orientation information of any target object in any key frame can include, for example, an anchor point, a scaling factor, direction information, rotation information, and the like of the target object in the model space at the time point corresponding to the key frame. The color information of any target object can be used to render the target object when the target object is displayed in a two-dimensional image. For example, the color information of the target object A can include a correspondence between points on the target object A and color values. When the target object A is displayed in a two-dimensional image, taking point a on the target object A as an example, the pixel point p corresponding to point a in the two-dimensional image can be determined, and then the color value corresponding to point a is obtained to make the pixel point p display the color value corresponding to point a.

[0042] ​Specifically, in one implementation, the media data to be processed can be obtained directly from pre-stored data or received from other devices. In another implementation, a model space can be constructed first, and a plurality of target objects to be moved can be constructed in the model space. The movement trajectory of each target object along the time axis in the model space can be simulated and calculated, and the time corresponding to each key frame on the time axis can be determined. According to the time corresponding to each key frame on the time axis and the movement trajectory, the spatial orientation information of each target object in each key frame can be determined. In addition, the color information corresponding to the points on each target object can also be determined.

[0043] In step 202, the spatial orientation information is stored as auxiliary information.

[0044] In the embodiment, the spatial orientation information can be stored as data in the form of an image to obtain an auxiliary image as auxiliary information. In one implementation, the encoding information corresponding to the spatial orientation information described above can be directly stored in the pixel value of the image to obtain an auxiliary image as auxiliary information. In another implementation, the spatial orientation information described above can be converted into transformation matrix information, and then the transformation matrix information is stored in the pixel value of the image to obtain an auxiliary image as auxiliary information. The transformation matrix information can include a target transformation matrix, and can further include an inverse matrix of the target transformation matrix, etc. The target transformation matrix can be a Model View Projection (MVP) matrix.

[0045] Further, when the spatial orientation information is stored in the pixel value of the auxiliary image, each row or each column of the auxiliary image can correspond to a key frame, and the row number of the row or the column number of the column is the same as the sequence number of the key frame. For example, the transformation matrix information corresponding to the nth key frame can be stored in the pixel value of the pixel point of the nth row or the nth column of the auxiliary image. Therefore, based on the row number or the column number of the auxiliary image, the transformation matrix information corresponding to any key frame can be quickly found and obtained to generate the key frame. Since the spatial orientation information is stored in the pixel value of the image in the embodiment, it is more conducive to the blur rendering processing of the GPU, and the processing speed of the blur rendering processing is improved.

[0046] In step 203, based on the auxiliary information and the color information described above, the blur rendering processing is performed to obtain a plurality of key frames, and in step 204, based on the plurality of key frames, the target media is determined.

[0047] In this embodiment, multiple keyframes can be generated based on auxiliary information storing the spatial orientation information of multiple target objects in multiple keyframes and the color information of multiple target objects. Specifically, a preset operation can be executed multiple times, generating a keyframe each time the preset operation is executed. For example, keyframes can be generated frame by frame in the order of playback time. Each preset operation can include: First, based on the aforementioned auxiliary information, obtaining the transformation matrix information corresponding to a keyframe. The transformation matrix information corresponding to the keyframe is obtained by converting the spatial orientation information of multiple target objects in the keyframe. For example, the transformation matrix information corresponding to the keyframe can be the target transformation matrix (such as the MVP matrix) corresponding to the keyframe. Next, based on the transformation matrix information corresponding to the keyframe, determining the renderable range of each of the multiple target objects, and performing blur rendering based on the renderable range of each of the multiple target objects and the aforementioned color information to obtain the keyframe. Finally, the target media can be generated based on the multiple keyframes obtained after blur rendering.

[0048] This disclosure provides a media processing method that acquires spatial orientation information and color information of multiple target objects in multiple keyframes. The spatial orientation information is stored as auxiliary information. Based on this auxiliary information and the color information, a blur rendering process is performed to obtain multiple keyframes, and the target media is determined based on these keyframes. Because this embodiment acquires the spatial orientation information of the target objects in multiple keyframes before the blur rendering process and stores this information as auxiliary information, the blur rendering process can be directly based on this auxiliary information, reducing the computational load during rendering, improving the efficiency of special effects processing, and enhancing the user experience.

[0049] Figure 3 This is a flowchart illustrating another media processing method according to an exemplary embodiment, which describes the process of generating a keyframe in step 203, including the following steps:

[0050] like Figure 3 As shown, in step 301, the transformation matrix information corresponding to the current keyframe is obtained based on auxiliary information.

[0051] In the embodiment, the key frame to be generated in the current preset operation can be taken as a current key frame, and the transformation matrix information corresponding to the current key frame can be converted from the spatial orientation information of the target objects in the current key frame. For example, the transformation matrix information can include a target transformation matrix (such as an MVP matrix). In one implementation, the spatial orientation information of the target objects in each key frame can be stored in the auxiliary information. A key frame can be determined as a current key frame, and the spatial orientation information of the target objects in the current key frame can be obtained from the auxiliary information, and then the spatial orientation information can be converted into the transformation matrix information corresponding to the current key frame. In another implementation, the transformation matrix information corresponding to each key frame can be directly stored in the auxiliary information, and the transformation matrix information corresponding to the current key frame can be directly obtained from the auxiliary information after a key frame is determined as the current key frame.

[0052] In step 302, the renderable ranges of the target objects are determined based on the transformation matrix information corresponding to the current key frame.

[0053] In the embodiment, the vertex coordinates of the target objects in the model space can be determined based on the target transformation matrix corresponding to the current key frame, and the vertex coordinates of the target objects in the model space are mapped into the planar image to obtain the vertex coordinates of the target objects in the current key frame. For example, as shown in FIG. 4, in the planar image, points p1, p2,..., and p5 are the vertexes of the target object 410. Then, the renderable ranges of the target objects in the current key frame can be determined according to the vertex coordinates of the target objects in the current key frame. In one implementation, the area surrounded by the vertex coordinates of each target object in the current key frame can be taken as the renderable range of the target object in the current key frame. Figure 4A

[0054] In another implementation, the renderable ranges of the target objects can also be obtained by performing a preset scale magnification operation based on the vertex coordinates of the target objects in the key frame. For example, as shown in FIG. 5, the area 411 is obtained by magnifying the vertexes of the target object 410, and the area 411 can be taken as the renderable range of the target object 410 in the current key frame. Since the area surrounded by the vertex coordinates of the target object in the key frame is magnified to obtain the renderable range of the target object, the control of the renderable range is realized, and the motion effect of the rendered image is better. Figure 4A

[0055] In step 303, the current key frame is obtained by performing the blur rendering based on the renderable ranges and the color information.

[0056] ​​In an implementation manner, the pixel value of each target object corresponding to a pixel point in the current key frame can be determined directly according to the transformation matrix information and the color information corresponding to the current key frame, and a preset blur rendering algorithm is used to perform blur rendering operation in the renderable range to obtain the current key frame.

[0057] In another implementation manner, a preset number of key frames before the current key frame and a preset number of key frames after the current key frame can also be determined as reference key frames. Based on the auxiliary information, the transformation matrix information corresponding to each reference key frame is obtained. The preset number can be any reasonable number determined according to experience, and the embodiment is not limited in the specific value of the preset number. Then, based on the transformation matrix information corresponding to each reference key frame, the renderable range and the color information, the ghosting information of the plurality of target objects in the renderable range in the current key frame is determined. Finally, based on the transformation matrix information corresponding to the current key frame, the color information and the ghosting information, the plurality of target objects are rendered respectively.

[0058] Specifically, in an optional implementation manner, the transformation matrix information can also include the inverse matrix of the target transformation matrix, and the inverse matrix of the target transformation matrix corresponding to each reference key frame can be directly obtained from the auxiliary information. Then, the target point on the target object to which the pixel point in the renderable range is mapped is calculated according to the inverse matrix of the target transformation matrix corresponding to each reference key frame. The ghosting information is determined based on the target point and the color information. Compared with using the target transformation matrix to determine the ghosting information, using the inverse matrix of the target transformation matrix to determine the ghosting information has higher calculation efficiency.

[0059] For example, for any pixel point in the renderable range in the current key frame, the target point on the target object to which the pixel point is mapped can be calculated based on the inverse matrix of the target transformation matrix corresponding to any reference key frame. Then, the color value corresponding to the target point is determined according to the color information as the ghosting color value contributed by the reference key frame to the pixel point. Taking the pixel point S1 in the renderable range b in the current key frame a as an example, the point S1' on the target object to which the pixel point S1 is mapped can be calculated according to the inverse matrix of the target transformation matrix corresponding to the reference key frame c. Then, based on the color information, the color value corresponding to the coordinate S1' is determined as the ghosting color value contributed by the reference key frame c to the pixel point S1.

[0060] Finally, after determining the ghost color value contributed by each reference keyframe to each pixel within the renderable range of the current keyframe, the ghost color values ​​contributed by each reference keyframe to that pixel can be weighted and superimposed for each pixel within the renderable range of the current keyframe to obtain the ghost information of each pixel within the renderable range of the current keyframe.

[0061] Since this embodiment determines the ghosting information based on the transformation matrix information corresponding to the reference keyframes before and after the current keyframe, the ghosting information is closer to the effect of the target object's motion, thus making the blur rendering have a better effect.

[0062] Optionally, a mask image can be generated in advance for the media data to be processed. A portion of the pixels in this mask image have a one-to-one mapping relationship with points on multiple target objects. Specifically, pixels corresponding to different target objects in the mask image have different pixel values, while pixels corresponding to the same target object have the same pixel value. Therefore, this mask image is used to distinguish the target objects corresponding to pixels based on their pixel values.

[0063] like Figure 4B As shown, image 420 is a mask image pre-generated for the media data to be processed. The media data to be processed includes five different target objects: target object d1, target object d2, target object d3, target object d4, and target object d5. Region 421 in the mask image corresponds to target object d1 (i.e., the pixels mapped from points on target object d1 to pixels in the mask image are located in region 421), region 422 corresponds to target object d2, region 423 corresponds to target object d3, region 424 corresponds to target object d4, and region 425 corresponds to target object d5. Regions 421, 422, 423, 424, and 425 each have different pixel values ​​(different patterns in the image represent different pixel values). For example, any two points on target object d1 have the same pixel value mapped to pixels in the mask image, while any point on target object d1 and any point on target object d2 have different pixel values ​​mapped to pixels in the mask image.

[0064] Before determining the ghost information in the current key frame, a pre-generated mask image can be obtained. Then, based on the mask image, valid target points can be selected. Specifically, according to the inverse matrix of the target transformation matrix corresponding to each reference key frame, the target point on the target object to which each pixel point in the renderable range of the current key frame is mapped can be calculated. Wherein, the points on the target objects can be mapped into the object space as a whole, and each point in the object space can correspond to an object space coordinate, and different points in the object space correspond to different object space coordinates. Therefore, according to the inverse matrix of the target transformation matrix corresponding to any reference key frame, the model space coordinate corresponding to any pixel point in the renderable range of the current key frame mapped into the model space can be calculated, and then the model space coordinate is mapped into the object space to obtain the object space coordinate, so as to determine the target point on the target object corresponding to the object space coordinate. According to the above mask image, a plurality of valid target points can be selected from a plurality of target points.

[0065] For example, the pixel point S2 is a pixel point in the renderable range of the target object c in the current key frame. Based on the inverse matrix G1 of the target transformation matrix corresponding to the previous key frame of the current key frame, the coordinate (x, y, z) of the pixel point S2 mapped into the model space at t1 time (t1 time is the time corresponding to the previous key frame of the current key frame on the time axis) can be calculated. Then, according to the matrix G1, the target point m corresponding to the coordinate (x, y, z) in the model space at t1 time mapped into the object space is determined. According to the mapping relationship between the points on the target object and the pixel points in the mask image, the pixel point YS1 in the mask image corresponding to the target point m is determined. The pixel value corresponding to the pixel point YS1 in the mask image is obtained, and if the pixel value corresponding to the pixel point YS1 is associated with the target object c, the target point m can be selected as a valid target point. If the pixel value corresponding to the pixel point YS1 is not associated with the target object c (for example, the pixel value corresponding to the pixel point YS1 is associated with other target objects, or is not associated with any target object), the target point m can be determined as an invalid target point, and the invalid target point can be ignored.

[0066] Finally, the ghost information can be determined according to the valid target points. Specifically, according to the color information, the color value corresponding to each valid target point can be obtained as the ghost color value contributed by the reference key frame. For example, for the valid target point m, the color value corresponding to the valid target point m can be obtained as the ghost color value contributed by the reference key frame according to the correspondence between the points in the model space and the color values. Then, for each pixel point in the renderable range, the color value of the valid target point corresponding to the pixel point is weighted and calculated to obtain the ghost information corresponding to the pixel point.

[0067] By selecting valid target points based on a pre-generated mask image, invalid target points that affect the blurring effect can be discarded, further improving the effect of the special effects.

[0068] This disclosure provides a media processing method that obtains transformation matrix information corresponding to the current keyframe based on auxiliary information, determines the renderable range of multiple target objects based on the transformation matrix information of the current keyframe, and performs blur rendering based on the renderable range and color information to obtain the keyframe. Since this embodiment first determines the renderable range of multiple target objects in the keyframe, and then performs blur rendering based on the renderable range of each target object in the keyframe, there is no need to perform blur rendering operations on areas outside the renderable range during the blur rendering process. This further reduces the computational load during rendering, helps improve the efficiency of blur rendering processing, and enhances the user experience.

[0069] It should be noted that although the operations of the methods of this disclosure embodiment are described in a specific order in the above embodiments, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0070] Corresponding to the aforementioned media processing method embodiments, this disclosure also provides embodiments of media processing apparatus.

[0071] like Figure 5 As shown, Figure 5 This is a block diagram of a media processing apparatus according to an exemplary embodiment of the present disclosure. The apparatus may include: an acquisition module 501, a storage module 502, a processing module 503, and a determination module 504.

[0072] The acquisition module 501 is used to acquire the spatial orientation information of multiple target objects in multiple keyframes and the color information of multiple target objects.

[0073] Storage module 502 is used to store spatial orientation information as auxiliary information.

[0074] The processing module 503 is used to perform blur rendering based on auxiliary information and color information to obtain multiple keyframes.

[0075] The determination module 504 is used to determine the target media based on multiple keyframes.

[0076] In some embodiments, the storage module 502 is configured to convert the spatial orientation information into transformation matrix information, store the transformation matrix information into pixel values of the image, and obtain an auxiliary image as the auxiliary information.

[0077] In some other embodiments, the processing module 503 can perform the blur rendering processing by performing the preset operation multiple times, and perform the preset operation once to obtain a key frame. The preset operation includes: obtaining transformation matrix information corresponding to the current key frame based on the auxiliary information, the transformation matrix information corresponding to the current key frame being converted from the spatial orientation information of the plurality of target objects in the current key frame. Based on the transformation matrix information corresponding to the current key frame, the rendering range of each of the plurality of target objects is determined, and the blur rendering is performed based on the rendering range and the color information to obtain the current key frame.

[0078] In some other embodiments, the transformation matrix information includes a target transformation matrix, and the processing module 503 can determine the rendering range of each of the plurality of target objects based on the transformation matrix information corresponding to the current key frame by: determining vertex coordinates of the plurality of target objects in the model space based on the target transformation matrix corresponding to the current key frame, mapping the vertex coordinates to the planar image to obtain vertex coordinates of the plurality of target objects in the current key frame, and performing a preset scale magnification operation based on the vertex coordinates of the plurality of target objects in the current key frame to obtain the rendering range of each of the plurality of target objects.

[0079] In some other embodiments, the processing module 503 performs the blur rendering based on the rendering range and the color information by: determining a preset number of key frames before the current key frame and a preset number of key frames after the current key frame as reference key frames. Based on the auxiliary information, the transformation matrix information corresponding to each of the reference key frames is obtained, and based on the transformation matrix information corresponding to each of the reference key frames, the rendering range, and the color information, the ghost information in the rendering range is determined. Based on the transformation matrix information corresponding to the current key frame, the color information, and the ghost information, the blur rendering is performed for each of the plurality of target objects.

[0080] In some other embodiments, the transformation matrix information includes an inverse matrix of the target transformation matrix, and the processing module 503 is configured to obtain the inverse matrix of the target transformation matrix corresponding to the reference key frame.

[0081] In some other embodiments, the processing module 503 can determine the ghost information in the rendering range by: determining target points on the plurality of target objects to which pixel points in the rendering range are mapped based on the inverse matrix of the target transformation matrix corresponding to each of the reference key frames, and determining the ghost information based on the target points and the color information.

[0082] In some embodiments, the processing module 503 can determine the ghosting information based on the target points and the color information by: obtaining a pre-generated mask image, and the pixel points in the mask image have a one-to-one mapping relationship with the points on the target objects. The pixel points corresponding to different target objects in the mask image have different pixel values, and the pixel points corresponding to the same target object have the same pixel value. Based on the mask image, the effective target points in the target points are selected, and the ghosting information is determined based on the effective target points and the color information.

[0083] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts are described in the part of the method embodiments. The device embodiments described above are only illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected to achieve the purpose of the embodiments of the present disclosure according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0084] Some embodiments of the present disclosure provide an electronic device. The electronic device includes a processor and a memory, which can be used to implement a client or a server. The memory is used to non-transiently store computer executable instructions (for example, one or more computer program modules). The processor is used to run the computer executable instructions, and when the computer executable instructions are run by the processor, one or more steps of the media processing method described above can be executed, and the media processing method described above is implemented. The memory and the processor can be interconnected through a bus system and / or other forms of connection mechanism (not shown).

[0085] For example, the processor can be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capability and / or program execution capability. For example, the central processing unit (CPU) can be X86 or ARM architecture, etc. The processor can be a general-purpose processor or a special-purpose processor, and can control other components in the electronic device to perform the desired functions.

[0086] For example, the memory can include any combination of one or more computer program products which can include various forms of computer-readable storage media, for example, volatile memory and / or non-volatile memory. Volatile memory, for example, can include random access memory (RAM), and / or cache memory, etc. Non-volatile memory, for example, can include read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules can be stored on the computer-readable storage media, and the processor can execute the one or more computer program modules to implement various functions of the electronic device. Various application programs and various data used and / or generated by the application programs, etc. can also be stored in the computer-readable storage media.

[0087] It should be noted that, in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device can refer to the description of the media processing method in the foregoing description, which will not be repeated here.

[0088] Figure 6 A schematic block diagram of an electronic device is provided for some embodiments of the present disclosure. The electronic device 920 is suitable for implementing the media processing method provided by the embodiments of the present disclosure, for example. The electronic device 920 can be a terminal device, etc., and can be used to implement a client or a server. The electronic device 920 can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a vehicle terminal (such as a vehicle navigation terminal), a wearable electronic device, etc., and a fixed terminal such as a digital TV, a desktop computer, a smart home device, etc. It should be noted that, Figure 6 The electronic device 920 shown is only an example, which does not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0089] As Figure 6 shown, the electronic device 920 can include a processing device (such as a central processor, a graphics processor, etc.) 921, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 922 or programs loaded from a storage device 928 into a random access memory (RAM) 923. Various programs and data required for the operation of the electronic device 920 are also stored in the RAM 923. The processing device 921, the ROM 922, and the RAM 923 are connected to each other through a bus 924. An input / output (I / O) interface 925 is also connected to the bus 924.

[0090] Generally, the following devices can be connected to the I / O interface 925: input devices 926, including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 927, including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 928, including, for example, a tape, a hard disk, and the like; and communication devices 929. The communication devices 929 can allow the electronic device 920 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 6 The electronic device 920 is shown with various devices, but it is understood that not all of the shown devices are required to be implemented or present, and the electronic device 920 can instead be implemented or present more or fewer devices.

[0091] For example, according to embodiments of the disclosure, the above-described method of processing media can be implemented as a computer software program. For example, embodiments of the disclosure include a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program including program code for executing the above-described method of processing media. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 929, or installed from the storage devices 928, or installed from the ROM 922. When the computer program is executed by the processing devices 921, the functions defined in the method of processing media provided by embodiments of the disclosure can be implemented.

[0092] Some embodiments of the disclosure provide a storage medium. For example, the storage medium can be a non-transitory computer readable storage medium, for storing non-transitory computer executable instructions. When the non-transitory computer executable instructions are executed by a processor, the method of processing media described by embodiments of the disclosure can be implemented, for example, when the non-transitory computer executable instructions are executed by a processor, one or more steps of the method of processing media according to the above can be performed.

[0093] For example, the storage medium can be applied in the above-described electronic device, for example, the storage medium can include a memory in the electronic device.

[0094] For example, the storage medium can include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a compact disc read only memory (CD-ROM), a flash memory, or any combination of the above storage mediums, or other applicable storage medium.

[0095] For example, the description about the storage medium can refer to the description about the memory in the embodiment of the electronic device, and the repeated parts will not be described herein. The specific functions and technical effects of the storage medium can refer to the description about the processing method of the medium in the foregoing, and will not be described herein.

[0096] It should be noted that, in the context of the present disclosure, the computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency), or the like, or any suitable combination of the above.

[0097] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure as come within known use or custom in the art. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0098] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A media processing method, the method comprising: obtaining spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects; storing the spatial orientation information as auxiliary information; performing blur rendering processing based on the auxiliary information and the color information to obtain the plurality of key frames; determining a target media based on the plurality of key frames.

2. The method of claim 1, wherein, The storing of the spatial orientation information as auxiliary information comprises: converting the spatial orientation information into transformation matrix information; storing the transformation matrix information into pixel values of an image to obtain auxiliary information.

3. The method of claim 1, wherein the performing the blur rendering process based on the auxiliary information and the color information comprises performing a preset operation for a plurality of times, and wherein one key frame is obtained by performing the preset operation once. The preset operation comprises: obtaining transformation matrix information corresponding to a current key frame based on the auxiliary information; the transformation matrix information corresponding to the current key frame is converted from the spatial orientation information of the plurality of target objects in the current key frame; determining a renderable range of each of the plurality of target objects based on the transformation matrix information corresponding to the current key frame; performing blur rendering based on the renderable range and the color information to obtain the current key frame.

4. The method of claim 3, wherein, The transformation matrix information includes a target transformation matrix; the determining of the renderable range of each of the plurality of target objects based on the transformation matrix information corresponding to the current key frame comprises: determining vertex coordinates of the plurality of target objects in a model space based on the target transformation matrix corresponding to the current key frame, and mapping the vertex coordinates to a planar image to obtain vertex coordinates of the plurality of target objects in the current key frame; performing a preset scale magnification operation based on the vertex coordinates of the plurality of target objects in the current key frame to obtain the renderable range of each of the plurality of target objects.

5. The method of claim 3, wherein, The performing of the blur rendering based on the renderable range and the color information comprises: determining a preset number of key frames before the current key frame and a preset number of key frames after the current key frame as reference key frames; obtaining transformation matrix information corresponding to the reference key frames based on the auxiliary information; determining ghost information in the renderable range based on the transformation matrix information corresponding to each of the reference key frames, the renderable range, and the color information; performing blur rendering on the plurality of target objects respectively based on the transformation matrix information corresponding to the current key frame, the color information, and the ghost information.

6. The method of claim 5, wherein, The transformation matrix information includes an inverse matrix of a target transformation matrix; wherein the obtaining of the transformation matrix information corresponding to the reference key frames comprises: obtaining the inverse matrix of the target transformation matrix corresponding to the reference key frames; wherein the determining of the ghost information in the renderable range comprises: determining target points on which pixel points in the renderable range are mapped to the plurality of target objects respectively based on the inverse matrix of the target transformation matrix corresponding to each of the reference key frames; determining the ghost information based on the target points and the color information.

7. The method of claim 6, wherein, The determining of the ghost information based on the target points and the color information comprises: obtaining a pre-generated mask image; the mask image has a one-to-one mapping relationship between part of the pixel points and the points on the plurality of target objects; wherein the pixel points corresponding to different target objects in the mask image have different pixel values, and the pixel points corresponding to the same target object have the same pixel value; selecting effective target points in the target points based on the mask image; determining the ghost information based on the effective target points and the color information.

8. A media processing apparatus, the apparatus comprising: an obtaining module configured to obtain spatial orientation information of a plurality of target objects in a plurality of key frames and color information of the plurality of target objects; a storage module configured to store the spatial orientation information as auxiliary information; a processing module configured to perform blur rendering processing based on the auxiliary information and the color information to obtain the plurality of key frames; a determination module configured to determine a target media based on the plurality of key frames.

9. A computer readable storage medium having stored thereon a computer program, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.

10. An electronic device comprising a memory and a processor, the memory having stored therein executable code, the processor implementing the method of any one of claims 1-7 when executing the executable code.

Images