Image data processing method, device, readable storage medium and program product

Abstract

Embodiments of the present disclosure provide an image data processing method and device, a readable storage medium and a program product. The method comprises: obtaining a first video, the first video comprising a plurality of images, the first video being a slow video; dividing the first video into a first video subset and a second video subset; dropping frames of the first video subset; applying a special effect to the first video subset after frame dropping, and recording special effect application information of the special effect; previewing and displaying the first video subset after applying the special effect; and generating a material package comprising the second video subset and the special effect application information.

Description

Technical Field

[0001] This disclosure relates to the field of computer and image data processing, and more particularly to an image data processing method, apparatus, readable storage medium, and program product. Background Technology

[0002] Slow-motion video is needed in many applications, such as when analyzing motion details or enhancing visual effects.

[0003] To further enhance visual effects, there is a need to apply special effects to slow-motion videos. However, due to the high resource consumption, it is not possible to preview the effects when generating a material package for applying special effects to slow-motion videos. Summary of the Invention

[0004] This disclosure provides an image data processing method, apparatus, readable storage medium, and program product to overcome the problem that when generating material packages for applying special effects to slow-moving videos, the large resource consumption makes it impossible to preview the effects.

[0005] In a first aspect, embodiments of this disclosure provide an image data processing method, including:

[0006] Acquire a first video, which includes multiple frames of images and is a slow-motion video;

[0007] The first video is divided into a first video subset and a second video subset;

[0008] Frames are dropped from the first subset of videos;

[0009] Apply special effects to the first video subset after frame loss, and record the effect application information of the special effects;

[0010] The preview displays the first subset of videos after the applied effects.

[0011] Generate a media package that includes the second video subset and the special effects application information.

[0012] In a second aspect, embodiments of this disclosure provide an image data processing apparatus, comprising:

[0013] The acquisition unit is used to acquire a first video, which includes multiple frames of images and is a slow-motion video.

[0014] A partitioning unit is used to divide the first video into a first video subset and a second video subset;

[0015] A frame dropping unit is used to drop frames from the first video subset.

[0016] The special effects application unit is used to apply special effects to the first video subset after frame loss and to record the special effects application information.

[0017] The display unit is used to preview the first video subset after applying special effects;

[0018] The generation unit is used to generate a material package that includes the second video subset and the special effects application information.

[0019] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0020] The memory stores computer-executed instructions;

[0021] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the image data processing method as described in the first aspect and various possible designs of the first aspect.

[0022] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the image data processing method described in the first aspect and various possible designs of the first aspect.

[0023] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the image data processing method described in the first aspect and various possible designs of the first aspect.

[0024] The image data processing method, device, readable storage medium, and program product provided in this embodiment acquire a first video, which includes multiple images and is a slow-motion video; divide the first video into a first video subset and a second video subset; drop frames from the first video subset; apply special effects to the first video subset after frame dropping and record the effect application information; preview and display the first video subset after applying the special effects; and generate a material package including the second video subset and the effect application information. Because the frames are dropped in advance in the first video subset used for preview, the number of images requiring special effects is reduced, thus reducing the resources required for previewing; simultaneously, the material package is generated according to the original images in the second video subset, and the effect application information is encapsulated in the material package, reducing the resources required to generate the slow-motion video material package with applied special effects; thereby, it is possible to achieve a smooth preview of the slow-motion video after applying special effects and simultaneously create a slow-motion video material package with applied special effects. The effect application information in the aforementioned material package is used to display the slow-motion video with applied special effects when the video file is subsequently opened. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 Flowchart of the image data processing method provided in the embodiments of this disclosure Figure 1 ;

[0027] Figure 2 Flowchart of the image data processing method provided in the embodiments of this disclosure Figure 2 ;

[0028] Figure 3 This is a structural block diagram of an image data processing device provided in an embodiment of the present disclosure;

[0029] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] Slow-motion video (also known as slow motion or slow-motion video) refers to the technique of slowing down the playback speed of video to show more detailed motion than at normal speed. Slow-motion video typically uses a higher frame rate to capture image sequences, and then plays these sequences back at the normal playback frame rate, thus achieving the effect of slowing down time. For example, if a video is shot at 240 frames per second and played back at 30 frames per second, it can achieve an effect equivalent to 8 times slower motion.

[0032] In some application scenarios, users can apply effects to slow-motion videos and preview how the effects will appear in the slow-motion video while adding them. However, because slow-motion videos have a large data volume, applying effects frame by frame and previewing them would consume a lot of system resources. It is usually not possible to simultaneously display the effect of applying effects to a slow-motion video and generate a media package that includes the applied effects.

[0033] The solution disclosed herein splits slow-motion video into segments, applies effects preview to one segment after frame dropping, and generates a media package based on the other segment and the effects application information. This reduces the resources required for previewing and displaying the applied effects on the slow-motion video, as well as the resources required for generating the applied effects media package. It can solve the problem of not being able to simultaneously display the applied effects on the slow-motion video and generate a media package containing the applied effects.

[0034] Please refer to Figure 1 , Figure 1 A schematic diagram of the image data processing method provided in this disclosure is shown. Figure 1 ,like Figure 1 As shown, the method includes the following steps:

[0035] S101: Obtain the first video, which includes multiple images and is a slow-motion video.

[0036] The image data processing method in this embodiment can be executed by a terminal device, specifically an application running on the terminal device. In some application scenarios, the aforementioned application can be an application used to call an image acquisition device for acquisition and to edit images.

[0037] The first video may be pre-captured by other image acquisition devices using a high frame rate (e.g., 120 frames / second, 240 frames / second). The first video may include multiple images arranged sequentially according to their acquisition time.

[0038] In some application scenarios, the aforementioned first video can also be a video captured in real time using an image acquisition device on a terminal device.

[0039] In these application scenarios, users can initiate slow-motion video capture commands on terminal devices, such as triggering preset slow-motion video recording controls in the application's display interface.

[0040] The image acquisition device can continuously acquire multiple frames of images at a frame rate higher than the display frame rate, according to a slow-motion video acquisition command. These continuously acquired frames form the first video. For example, the frame rate could be 120 frames per second, 240 frames per second, etc.

[0041] S102: Divide the first video into a first video subset and a second video subset.

[0042] In one example, the aforementioned execution entity can split the first video into a first video subset and a second video subset. Specifically, the first video can be copied or distributed into two independent video subsets; that is, after receiving each frame, a copy of that frame is created and sent to two different processing paths respectively. One of the video subsets can be considered the first video subset, and the other video subset can be considered the second video subset.

[0043] S103: Drop frames from the first video subset.

[0044] Frame dropping can be performed on the first video subset. For example, frames can be dropped from the first video subset according to the preview display frame rate. Specifically, the frame dropping strategy can be determined based on the image acquisition frame rate and the preview display frame rate of the first video, and the image frames in the first video subset can be discarded according to the above frame dropping strategy.

[0045] To illustrate, assuming the image acquisition frame rate is 240 frames per second and the preview display frame rate is 30 frames per second, one frame can be retained every eight frames in the first video subset, and the remaining seven frames can be discarded. After discarding frames from the first video subset according to the above frame-dropping strategy, the resulting first video subset can be obtained.

[0046] Apply effects to the first subset of the video after frame drops. These effects can include one or more types of effects, and each type can include more than one effect. Examples of these effects include beautification filters, stickers, and audio.

[0047] After applying effects to each image frame in the first video subset after frame loss, the first video subset with applied effects can be sent to the display unit for preview display.

[0048] S104: Apply effects to the first subset of video after frame loss and record the effect application information.

[0049] S105: Preview displays the first subset of the video after applying effects.

[0050] When processing image sequences captured at high frame rates (e.g., 240 frames per second), if each frame requires complex image processing operations (such as beautification effects), the system may be unable to complete the processing of all frames within the allotted time due to limitations in computing resources and time, resulting in frame backlog. Over time, this backlog leads to increasingly larger processing delays, ultimately causing the output image to lag significantly behind the actual events. Furthermore, if slow-moving video is being captured simultaneously, resource constraints may prevent the system from capturing images at the same frame rate as the slower video.

[0051] Applying effects to the first video subset after frame drops reduces the number of image frames that need to have effects applied, which in turn reduces the resource consumption required to preview the video content after the effects are applied. The first video subset with the effects applied can be previewed and displayed more smoothly.

[0052] It is understandable that each image in the first video has its own timestamp, with the first frame having a timestamp of 00:00:00. The timestamp for each image could, for example, correspond to the start time.

[0053] When applying effects to the first subset of video after frame loss, the effect application information for the first subset of video is recorded. This effect information may include the effect identifier (or effect data) and effect application settings for at least one applied effect. For example, which effects were used in multiple time periods within the first subset of video. Another example is the start and end times of each of the multiple effects applied to the first subset of video.

[0054] S106: Generate a media package that includes a second video subset and information on special effects.

[0055] For the second video subset, in order to save resources, no special effects are applied to the second video subset when generating the material package. Instead, the material package is generated based on multiple original images in the second video subset, and the recorded special effects application information is encapsulated as metadata in the material package.

[0056] A resource pack typically refers to a collection of various resources used for creation or design. These resource collections can include multimedia elements such as images, audio, video, 3D models, fonts, and icons. Resource packs provide ready-made design elements and multimedia resources, allowing creators to quickly find the materials they need, thus saving time spent creating these elements from scratch. This can improve creative efficiency.

[0057] When playing a video file, the special effects application information in the video file can be read, and special effects can be applied to the second video subset according to the special effects application settings in the special effects application information.

[0058] In this embodiment, a first video, comprising multiple images, is acquired and is a slow-motion video. The first video is then divided into a first video subset and a second video subset. Frames are dropped from the first video subset. Special effects are applied to the first video subset after frame dropping, and the effect application information is recorded. The first video subset after applying the special effects is previewed. A media package including the second video subset and the effect application information is generated. Because frames are dropped in advance in the first video subset used for preview, the number of images requiring special effects is reduced, thus lowering the resources required for previewing. Simultaneously, a media package is generated based on the original images in the second video subset, and the effect application information is encapsulated within the media package, further reducing the resources required to generate the slow-motion video media package with applied effects. Therefore, a smooth preview of the slow-motion video after applying special effects can be achieved, while simultaneously creating the slow-motion video media package with applied effects. The effect application information in the aforementioned media package is used to display the slow-motion video with applied effects when the video file is subsequently opened.

[0059] In some embodiments, applying effects to the first video subset after frame loss in step S103 includes the following steps:

[0060] First, the frame dropping strategy is determined based on the preview display frame rate and the image acquisition frame rate corresponding to the first video.

[0061] Secondly, the first video subset is subjected to frame dropping based on a frame dropping strategy.

[0062] Finally, effects are applied to the first subset of the video after frame loss.

[0063] In some examples, the preset display frame rate can be set by the user; in other examples, it can be automatically set by the terminal device's system. The preset display frame rate can be 3 frames per second, 60 frames per second, etc.

[0064] The image acquisition frame rate can be higher than the normal display frame rate of the terminal device, such as 120 frames / second or 240 frames / second. The above-mentioned frame dropping strategies include, but are not limited to: fixed-interval frame dropping, time-based frame dropping, and content-based frame dropping.

[0065] The aforementioned fixed-interval frame dropping can drop frames at fixed intervals. Taking an image acquisition frame rate of 120 frames / second and a display frame rate of 30 frames / second as an example, one frame is dropped every four frames. This ensures that the intervals between the remaining frames are relatively uniform, which helps to maintain the basic smoothness of the preview.

[0066] The aforementioned time-based frame dropping can drop frames according to a set time interval, such as setting it to retain only 30 frames per second and discard the rest.

[0067] The aforementioned content-based frame dropping determines which frames to discard by analyzing the content of image frames. For example, if the difference between two frames is very small, the latter frame can be discarded. This frame dropping strategy can balance the quality of the preview image with the amount of data.

[0068] Understandably, different frame dropping strategies can also be combined to drop frames from the first subset of the video.

[0069] By dropping frames from the first video subset using the aforementioned frame dropping strategy, a frame-dropped first video subset can be obtained. The aforementioned execution entity can then apply special effects to the images retained in the frame-dropped first video subset. By determining the frame dropping strategy based on the preview display frame rate and the image acquisition frame rate of the image acquisition device, and dropping frames from the first video subset according to the frame dropping strategy, the resources required for previewing are reduced while maintaining smooth preview playback.

[0070] In some embodiments, step S103 includes:

[0071] Based on a preset effect application strategy for applying multiple effects, each image in the first video subset after frame loss is applied with its own corresponding effect. Among them, the effects applied to multiple images are the same or different according to the preset effect application strategy.

[0072] In one example, the same effect can be applied to multiple frames of images according to the preset effects strategy described above.

[0073] In another example, based on the preset application strategies of the above-mentioned multiple effects, different effects can be applied to multiple frames of images, including effect switching strategies, combination strategies of different effects, etc.

[0074] The aforementioned special effects switching strategies may include switching effects based on time intervals or switching effects based on image content. Switching effects based on time intervals could include automatically changing the effect every N seconds, where N can be any value greater than 0.

[0075] The aforementioned content-based effect selection includes parsing the content of each frame of an image (e.g., object detection, face recognition) to determine the effects corresponding to each frame. Specifically, the target image content corresponding to each effect can be pre-set. For any image, the execution entity can detect the image content in the image, identify whether the image content includes a target object, and if it includes a target object, determine the target effect that matches the target object and apply the target effect to that frame of the image.

[0076] In this example, the effects applied to the multiple frames of the first video subset after frame loss using the aforementioned preset effect application strategy can be different.

[0077] In these embodiments, by applying multiple special effects to the first video subset after frame loss according to a preset special effects application strategy, corresponding special effects can be applied to each image in the first video subset, thereby improving the visual effect of the generated video content.

[0078] In some embodiments, step S104 includes:

[0079] The second video subset is compressed and encoded to generate a media package that includes the compressed and encoded second video subset.

[0080] Specifically, an encoder (such as H.264 / AVC, H.265 / HEVC, etc.) can be selected, and then the selected encoder is used to compress and encode the second video subset according to preset encoding parameters. These encoding parameters include, but are not limited to: resolution, Group of Pictures (GOP) structure, quantization parameters, and preset modes. The resolution mentioned above includes the resolution of the output video. The GOP structure includes the distribution of I-frames, P-frames, and B-frames. Quantization parameters affect compression quality and file size. The preset modes balance encoding speed and quality. The executing entity can call a video file generation tool to compress and encode the second video subset, and encapsulate the compressed and edited video data into a preset container (e.g., MP4), thereby obtaining a media package.

[0081] In this embodiment, by compressing and encoding the second video subset, redundant information between images is reduced, resulting in a smaller data package, which is beneficial for the storage and transmission of the data package.

[0082] In some embodiments, step S104 further includes:

[0083] Speed ​​information corresponding to the first video.

[0084] The aforementioned speed information could be, for example, a slow-motion factor, which can be determined by the ratio between the image capture frame rate and the display frame rate. The display frame rate can also be determined based on the image capture frame rate and the slow-motion factor.

[0085] The aforementioned speed adjustment information may include, for example, 8x speed adjustment, 4x speed adjustment, etc. The speed adjustment information can be set according to the specific application scenario.

[0086] In addition, when recording slow-motion video, it is not necessary to directly specify the slow-motion playback multiplier. The slow-motion effect that can be achieved during subsequent playback can be indirectly determined by selecting different image capture frame rates.

[0087] In these embodiments, the aforementioned material package may also encapsulate the speed change information of the slow-motion video. In this way, when the user opens the material package for playback, the second video subset in the material package can be played according to the speed change information to achieve automatic slow-motion playback.

[0088] Please refer to Figure 2 , Figure 2 Flowchart of the image data processing method provided in this disclosure Figure 2 ,like Figure 2 As shown, the method includes, in addition to, with Figure 1 In addition to the same steps S201-S206 as steps S101-S106 in the illustrated embodiment, the method further includes the following steps:

[0089] S207: In response to the loading instruction of the media package, decode the media package to obtain the second video subset.

[0090] In this embodiment, the entity executing the image data processing method can be a terminal device used by the user, specifically an application running on the terminal device that can process video image data.

[0091] In some examples, the executing entity for steps S205 to S206 can be the same as the terminal device executing steps S201 to S206, or it can be a different executing entity. For example, a user can obtain a material package on terminal device A through steps S201 to S206, and then send the material package to terminal device B, where the material package can be loaded by an application.

[0092] In one example, a user can issue the above loading command from an application running on a terminal device for editing video files. For instance, a user can trigger a control for loading a media package in the application and select the identifier of the media package from the file list displayed after triggering the control, thereby issuing a loading command for the media package to the terminal device.

[0093] Upon receiving the aforementioned loading instruction, the terminal device can decode the media package to obtain multiple frames of the second video subset.

[0094] S208: Apply special effects to multiple frames of images in the decoded second video subset based on the special effects application information encapsulated in the material package.

[0095] In one example, the aforementioned effect application information may include identifiers for multiple effects. These effect identifiers are used to distinguish different effects. In one example, the corresponding effect can be retrieved from other electronic devices (e.g., servers) based on the effect identifier. In another example, the aforementioned effect information may also include effect content, where the effect content can present the corresponding effect when loaded and displayed.

[0096] The aforementioned effects can include different categories of effects, such as one or more of the following categories: filters, filtering effects, scale transformations, text and graphics, dynamic elements, beautification, masking and keying, etc.

[0097] In some implementations, the aforementioned special effects information includes a preset special effects application strategy for applying the same or different special effects to multiple frames of images.

[0098] In some application scenarios, the aforementioned preset effect application strategy may include, for example, an effect application strategy that applies the same effect to multiple frames of images.

[0099] In these application scenarios, the aforementioned execution entity can apply the same effect to multiple frames of images decoded from the target video file according to the aforementioned preset effect application strategy, thereby obtaining multiple frames of images with the effect applied.

[0100] In other application scenarios, the above-mentioned preset effect application strategies may also include strategies for applying different effects to multiple frames of images.

[0101] In these application scenarios, the aforementioned preset effect application strategies can include, for example, switching effects based on time intervals or switching effects based on image content.

[0102] Based on these preset effect application strategies, different effects can be applied to multiple frames of video images. This can improve the richness of the displayed video content and enhance the visual effect.

[0103] S209: Displays multiple frames of images with applied effects.

[0104] The aforementioned executing entity can send multiple frames of images with applied special effects to the display unit, which then displays the multiple frames sequentially according to their respective timestamps.

[0105] In this embodiment, by generating a material package containing multiple frames of images of a second video subset including slow-motion video and corresponding special effects application information, the resources required to generate a material package of slow-motion video with applied special effects are saved. When loading the material package, the corresponding special effects are applied to the multiple frames of images decoded from the material package according to the special effects application information encapsulated in the material package, thereby displaying the slow-motion video with applied special effects.

[0106] In some optional implementations of this embodiment, step S207 includes:

[0107] Based on the display frame rate corresponding to the speed change information encapsulated in the material package, multiple frames of video images are displayed.

[0108] The timestamps are preserved from the image attributes in the second video subset.

[0109] In one example, the frame rate of slow-motion video can be determined based on the timestamps corresponding to multiple frames. In another example, the frame rate can be recorded in the header of the first video file; in this case, the frame rate can be read from the header of the first video file.

[0110] Before receiving instructions from the user to modify the speed change information, the aforementioned execution entity can use the speed change information and image acquisition frame rate encapsulated in the target video file to determine the display frame rate, and then control the display unit to display multiple frames of video images according to the aforementioned display frame rate to show the slow motion effect.

[0111] To illustrate, if the image acquisition frame rate is 240 frames per second and the speed information is 8 times slow motion, then the display frame rate is 240 / 8 = 30 frames per second.

[0112] Understandably, users can adjust the slow motion multiplier when loading or displaying the video content of the media package. If the user adjusts the slow motion multiplier, the display frame rate can be determined based on the adjusted slow motion multiplier, and multiple frames of images can be displayed based on that display frame rate.

[0113] In these implementations, the display frame rate is determined based on the speed information encapsulated in the media package, and multiple frames of images decoded from the media package are displayed according to the display frame rate, thereby automatically displaying slow-motion effects based on the speed information.

[0114] Corresponding to the image data processing method in the above embodiments, Figure 3 This is a structural block diagram of an image processing apparatus provided according to embodiments of the present disclosure. For ease of explanation, only the parts relevant to embodiments of the present disclosure are shown. (Refer to...) Figure 3 The device 30 includes: an acquisition unit 301, a partitioning unit 302, a frame dropping unit 303, a special effects application unit 304, a first display unit 305, and a generation unit 306.

[0115] Acquisition unit 301 is used to acquire a first video, the first video including multiple frames of images, and the first video is a slow-motion video;

[0116] The partitioning unit 302 is used to partition the first video into a first video subset and a second video subset;

[0117] Frame dropping unit 303 is used to drop frames from the first video subset;

[0118] Special effects application unit 304 is used to apply special effects to the first video subset after frame loss and record the special effects application information;

[0119] The first display unit 305 is used to preview the first subset of video after applying special effects;

[0120] The generation unit 306 is used to generate a material package that includes a second video subset and special effects application information.

[0121] The device provided in this embodiment acquires a first video, which includes multiple frames and is a slow-motion video, through an acquisition unit. A segmentation unit divides the first video into a first video subset and a second video subset. A frame dropping unit drops frames from the first video subset. A special effects application unit applies special effects to the first video subset after frame dropping and records the special effects application information. A display unit previews and displays the first video subset after applying the special effects. A generation unit generates a media package including the second video subset and the special effects application information. Because pre-dropping frames in the first video subset used for previewing reduces the number of images requiring special effects, and the media package is created from the original images of the video subset used to generate the media package, while encapsulating the special effects application information within the media package, the resources required for previewing the effect of applying special effects to the slow-motion video and generating the slow-motion video media package with applied special effects are reduced. This allows for a smooth preview of the slow-motion video after applying special effects and the simultaneous creation of the slow-motion video media package with applied special effects. The special effects application information in the media package is used to display the slow-motion video with applied special effects when the video file is subsequently opened.

[0122] In one embodiment of this disclosure, the frame dropping unit 303 is further configured to:

[0123] The frame dropping strategy is determined based on the preview display frame rate and the image acquisition frame rate corresponding to the first video.

[0124] The first video subset is subjected to frame dropping based on a frame dropping strategy.

[0125] In summary, further, by applying the aforementioned frame-dropping strategy to the first video subset, a frame-dropped first video subset can be obtained. The aforementioned execution entity can then apply special effects to the images retained in the frame-dropped first video subset. By determining the frame-dropping strategy based on the preview display frame rate and the image acquisition frame rate of the image acquisition device, and by applying the frame-dropping strategy to the first video subset, the resources required for previewing are reduced while maintaining the smoothness of the preview.

[0126] In one embodiment of this disclosure, the generation unit 306 is further configured to:

[0127] The second video subset is compressed and encoded to generate a media package that includes the compressed and encoded second video subset.

[0128] In summary, by further compressing and encoding the second video subset, redundant information between images is reduced, resulting in a smaller data package, which is beneficial for the storage and transmission of the data package.

[0129] In one embodiment of this disclosure, the material package further includes: speed change information corresponding to the first video.

[0130] In summary, furthermore, the aforementioned media package can also encapsulate the speed change information of slow-motion videos. In this way, when users open the media package for playback, they can play the second subset of videos in the media package based on the speed change information to achieve automatic slow-motion playback.

[0131] In one embodiment of this disclosure, the special effects application unit 304 is further configured to: apply corresponding special effects to each image in the first video subset after frame loss according to a preset special effects application strategy for applying multiple special effects, wherein the special effects applied to the multiple images according to the preset special effects application strategy are the same or different.

[0132] In summary, by applying multiple special effects to the first video subset after frame loss according to a preset special effects application strategy, corresponding special effects can be applied to each image in the first video subset, thereby improving the visual effect of the generated video content.

[0133] In one embodiment of this disclosure, device 30 further includes a second display unit (not shown in the figures), the second display unit being used for:

[0134] In response to the loading command for the media package, the media package is decoded to obtain the second video subset;

[0135] Based on the special effects application information encapsulated in the material package, special effects are applied to multiple frames of images in the decoded second video subset;

[0136] The display shows multiple frames of images with applied special effects. In one embodiment of this disclosure, the special effects application information includes a preset special effects application strategy for applying the same or different special effects to multiple frames of video images.

[0137] In summary, by generating a media package containing multiple frames of images from a second subset of the slow-motion video and corresponding special effects application information, the resources required to generate a media package of slow-motion video with applied special effects are saved. When loading the media package, the corresponding special effects are applied to the multiple frames decoded from the media package according to the special effects application information encapsulated in the media package, thereby displaying the slow-motion video with applied special effects.

[0138] In one embodiment of this disclosure, the first display unit 305 is further configured to:

[0139] Based on the display frame rate corresponding to the speed change information encapsulated in the material package, display multiple frames of images.

[0140] In summary, by determining the display frame rate based on the speed change information encapsulated in the material package, and displaying multiple frames of images decoded from the material package according to the display frame rate, the slow-motion effect can be automatically displayed based on the speed change information.

[0141] To implement the above embodiments, this disclosure also provides an electronic device.

[0142] refer to Figure 4 The diagram illustrates a structural schematic of an electronic device 400 suitable for implementing embodiments of the present disclosure. The electronic device 400 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers, portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0143] like Figure 4 As shown, electronic device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for the operation of electronic device 400. The processing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0144] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0145] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0146] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0147] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0148] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0149] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0150] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0151] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0152] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0153] In this embodiment, a first video is acquired, comprising multiple frames, and is a slow-motion video. A segmentation unit is used to divide the first video into a first video subset and a second video subset. Frames are dropped from the first video subset. Special effects are applied to the first video subset after frame dropping, and the effect application information is recorded. The first video subset after applying the special effects is previewed and displayed. A media package including the second video subset and the effect application information is generated. Because the pre-dropping of frames in the first video subset used for preview reduces the number of images requiring special effects, and the media package is created based on the original images of the video subset used to generate the media package, while encapsulating the effect application information in the media package, the resources required for previewing the effect of applying special effects to the slow-motion video and generating the slow-motion video media package with applied special effects are reduced. This allows for a smooth preview of the slow-motion video after applying special effects and the simultaneous creation of the slow-motion video media package with applied special effects. The effect application information in the media package is used to display the slow-motion video with applied special effects when the video file is opened subsequently.

[0154] In a first aspect, according to one or more embodiments of the present disclosure, an image data processing method is provided, comprising:

[0155] Acquire the first video, which includes multiple frames of images and is a slow-motion video;

[0156] The first video is divided into a first video subset and a second video subset;

[0157] Frames were dropped from the first subset of the video.

[0158] Apply effects to the first subset of video after frame loss and record the effect application information;

[0159] The preview displays the first subset of the video after the effects are applied;

[0160] Generate a media package that includes a second subset of videos and information on special effects.

[0161] According to one or more embodiments of this disclosure, dropping frames in a first subset of video includes:

[0162] The frame dropping strategy is determined based on the preview display frame rate and the image acquisition frame rate corresponding to the first video.

[0163] The first video subset is subjected to frame dropping based on a frame dropping strategy.

[0164] According to one or more embodiments of this disclosure, generating a media package including a second subset of video and special effects application information includes:

[0165] The second video subset is compressed and encoded to generate a media package that includes the compressed and encoded second video subset.

[0166] According to one or more embodiments of this disclosure, the material package further includes:

[0167] Speed ​​information corresponding to the first video.

[0168] According to one or more embodiments of this disclosure, applying effects to a first subset of video after frame loss includes:

[0169] Based on a preset effect application strategy for applying multiple effects, each image in the first video subset after frame loss is applied with its own corresponding effect. Among them, the effects applied to multiple images are the same or different according to the preset effect application strategy.

[0170] According to one or more embodiments of this disclosure, the method further includes:

[0171] In response to the loading command for the media package, the media package is decoded to obtain the second video subset;

[0172] Based on the special effects application information encapsulated in the material package, special effects are applied to multiple frames of images in the decoded second video subset;

[0173] Displays multiple frames of images with applied effects.

[0174] According to one or more embodiments of this disclosure, the special effects application information includes preset special effects application strategies for applying the same or different special effects to multiple frames of images.

[0175] According to one or more embodiments of this disclosure, displaying multi-frame video images with applied special effects includes:

[0176] Based on the display frame rate corresponding to the speed change information encapsulated in the material package, display multiple frames of images.

[0177] Secondly, according to one or more embodiments of the present disclosure, an image data processing apparatus is provided, comprising:

[0178] The acquisition unit is used to acquire a first video, which includes multiple frames of images and is a slow-motion video.

[0179] A partitioning unit is used to divide the first video into a first video subset and a second video subset;

[0180] The frame dropping unit is used to drop frames from the first subset of video.

[0181] The special effects application unit is used to apply special effects to the first subset of video after frame loss and to record the special effects application information.

[0182] The display unit is used to preview the first subset of the video after applying special effects;

[0183] The generation unit is used to generate a media package that includes a second video subset and special effects application information.

[0184] According to one or more embodiments of this disclosure, the frame dropping unit is further configured to:

[0185] The frame dropping strategy is determined based on the preview display frame rate and the image acquisition frame rate corresponding to the first video.

[0186] The first video subset is subjected to frame dropping based on a frame dropping strategy.

[0187] According to one or more embodiments of this disclosure, the generating unit is further configured to:

[0188] The second video subset is compressed and encoded to generate a media package that includes the compressed and encoded second video subset.

[0189] According to one or more embodiments of this disclosure, the material package further includes: speed change information corresponding to the first video.

[0190] According to one or more embodiments of this disclosure, the special effects application unit is further configured to: apply corresponding special effects to each image in the first video subset after frame loss according to a preset special effects application strategy for applying multiple special effects, wherein the special effects applied to the multiple images according to the preset special effects application strategy are the same or different.

[0191] According to one or more embodiments of this disclosure, the device further includes a second display unit, the second display unit being used for:

[0192] In response to the loading command for the media package, the media package is decoded to obtain the second video subset;

[0193] Based on the special effects application information encapsulated in the material package, special effects are applied to multiple frames of images in the decoded second video subset;

[0194] Displays multiple frames of images with applied effects.

[0195] According to one or more embodiments of this disclosure, the special effects application information includes preset special effects application strategies for applying the same or different special effects to multiple frames of video images.

[0196] According to one or more embodiments of this disclosure, the first display unit is further configured to:

[0197] Based on the display frame rate corresponding to the speed change information encapsulated in the material package, display multiple frames of images.

[0198] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0199] The memory stores instructions that the computer executes;

[0200] At least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the image data processing method as described in the first aspect above and various possible designs of the first aspect.

[0201] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed by a processor, implement the image data processing method as described in the first aspect and various possible designs of the first aspect.

[0202] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the image data processing method as described in the first aspect and various possible designs of the first aspect.

[0203] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0204] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0205] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image data processing method, comprising: Acquire a first video, which includes multiple frames of images and is a slow-motion video; The first video is divided into a first video subset and a second video subset; Frames are dropped from the first subset of videos; Apply special effects to the first video subset after frame loss, and record the effect application information of the special effects; The preview displays the first subset of videos after the applied effects. Generate a media package that includes the second video subset and the special effects application information.

2. The method according to claim 1, characterized in that, The frame dropping of the first video subset includes: The frame dropping strategy is determined based on the preview display frame rate and the image acquisition frame rate corresponding to the first video. The first video subset is subjected to frame dropping based on the frame dropping strategy.

3. The method according to claim 1, characterized in that, The generation includes a material package comprising the second video subset and the special effects application information, including: The second video subset is compressed and encoded to generate a material package that includes the compressed and encoded second video subset.

4. The method according to claim 1, characterized in that, The resource package also includes: Speed ​​information corresponding to the first video.

5. The method according to any one of claims 1, characterized in that, The application of effects to the first video subset after frame loss includes: According to a preset effect application strategy for applying multiple effects, each image in the first video subset after frame loss is applied with its own corresponding effect, wherein the effects applied to multiple images according to the preset effect application strategy may be the same or different.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: In response to the loading instruction of the media package, the media package is decoded to obtain the second video subset; Based on the special effects application information encapsulated in the material package, special effects are applied to multiple frames of images in the decoded second video subset; The multi-frame images with the special effects applied are displayed.

7. The method according to claim 6, characterized in that, The special effects application information includes preset special effects application strategies for applying the same or different special effects to multiple frames of images.

8. The method according to claim 6, characterized in that, The display of the multi-frame video images with applied special effects includes: The multi-frame images are displayed according to the display frame rate corresponding to the speed change information encapsulated in the material package.

9. An image data processing device, comprising: The acquisition unit is used to acquire a first video, which includes multiple frames of images and is a slow-motion video. A partitioning unit is used to divide the first video into a first video subset and a second video subset; A frame dropping unit is used to drop frames from the first video subset. The special effects application unit is used to apply special effects to the first video subset after frame loss and to record the special effects application information. The display unit is used to preview the first video subset after applying special effects; The generation unit is used to generate a material package that includes the second video subset and the special effects application information.

10. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the image data processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the image data processing method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image data processing method as described in any one of claims 1 to 8.

Images