Image processing method, device, electronic device, and readable storage medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device, and readable storage medium, where the method includes traversing a plurality of multimedia materials and determining multimedia materials of a non-target color display standard contained in the plurality of multimedia materials based on a target color display standard; converting the multimedia materials of the non-target color display standard into multimedia materials of a target color display standard; and image rendering the plurality of multimedia materials based on the target color display standard, so that the image rendering results can be presented or exported as a video file of a specific format according to the image rendering results.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority from Chinese Patent Application No. 202111658426.2, filed on December 30, 2021, the entire disclosure of which is incorporated herein by reference in its entirety.

[0002] [Technical field] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, and readable storage medium. [Background technology]

[0003] When editing a video, multiple multimedia materials are always edited to generate a video that meets the expected display effect. Summary of the Invention

[0004] The present disclosure provides an image processing method, apparatus, electronic device, and readable storage medium.

[0005] According to a first aspect, the present disclosure provides an image processing method, the method comprising: determining, based on the target color display standard, multimedia materials of a non-target color display standard included in the plurality of multimedia materials; converting the non-target color display standard multimedia material into the target color display standard multimedia material; The method includes image-rendering the plurality of multimedia materials based on the target color display standard, and presenting the plurality of multimedia materials on a display screen of an electronic device according to the image rendering result, or exporting the plurality of multimedia materials as a video file in a specific format according to the image rendering result.

[0006] In some embodiments, before determining, based on the target color display standard, multimedia material of a non-target color display standard included in the plurality of multimedia materials, the method further comprises: The method further includes determining the target color display standard.

[0007] In some embodiments, determining the target color display standard comprises: determining the target color display standard based on color display standards corresponding to each of the plurality of multimedia materials;

[0008] In some embodiments, determining the target color display standard based on color display standards corresponding to each of the plurality of multimedia materials comprises: The method includes determining the target color display standard based on a color display standard with the widest color space among the color display standards corresponding to each of the plurality of multimedia materials.

[0009] In some embodiments, determining the target color display standard includes determining the target color display standard based on a pre-specified color display standard.

[0010] In some embodiments, converting the multimedia material in the non-target color display standard to the multimedia material in the target color display standard comprises: converting the multimedia material in the non-target color display standard into the multimedia material in the target color display standard if the multimedia material in the non-target color display standard was generated based on a video frame image at a freeze-frame position of another multimedia material in the non-target color display standard among the plurality of multimedia materials; or This includes not converting the multimedia material of the non-target color display standard if the multimedia material of the non-target color display standard was generated based on a video frame image at the freeze frame position of the multimedia material of the target color display standard among the plurality of multimedia materials.

[0011] In some embodiments, the target color display standard is a perceptually quantized (PQ) color display standard, and the non-target color display standard comprises a standard dynamic range (SDR) color display standard and / or a hybrid log gamma curve (HLG) color display standard; or The target color display standard is an SDR color display standard, and the non-target color display standard includes an HLG color display standard and / or a PQ color display standard; or The target color display standard is the HLG color display standard, and the non-target color display standard includes the SDR color display standard and / or the PQ color display standard.

[0012] In some embodiments, before said image rendering of said plurality of multimedia materials based on said target color display standard, said method further comprises: The method further includes processing the plurality of multimedia materials according to a clip manipulation scheme corresponding to each of one or more clip types.

[0013] According to a second aspect, the present disclosure provides an image processing apparatus, the apparatus comprising: an identification module for determining, based on the target color display standard, multimedia materials of a non-target color display standard included in the plurality of multimedia materials; a conversion module for converting the non-target color display standard multimedia material into the target color display standard multimedia material; a rendering module for graphically rendering the plurality of multimedia materials; and a control module for controlling the electronic device to present the plurality of multimedia materials on a display screen according to the image rendering result, or for controlling the electronic device to export the plurality of multimedia materials as a video file in a specific format according to the image rendering result.

[0014] According to a third aspect, the present disclosure provides an electronic device, the electronic device including a memory and a processor; the memory configured to store computer program instructions; The processor is configured to execute the computer program instructions to cause the electronic device to implement the image processing method according to the first aspect.

[0015] According to a fourth aspect, the present disclosure provides a readable storage medium, the readable storage medium comprising computer program instructions: At least one processor of the electronic device executes the computer program instructions to implement the image processing method according to the first aspect.

[0016] According to a fifth aspect, the present disclosure provides a computer program product, which, when executed by a computer, causes the computer to implement the image processing method according to the first aspect.

[0017] According to a sixth aspect, the present disclosure provides a computer program comprising instructions that, when executed by a processor, cause the processor to implement the image processing method according to the first aspect.

[0018] The present disclosure provides an image processing method, apparatus, electronic device, and readable storage medium, wherein the method includes traversing a plurality of multimedia materials and determining, based on a target color display standard, multimedia materials of a non-target color display standard contained in the plurality of multimedia materials; converting the multimedia materials of the non-target color display standard into multimedia materials of the target color display standard; and image-rendering the plurality of multimedia materials based on the target color display standard, so that the image-rendered results can be presented or exported as a video file of a specific format according to the image-rendered results. [Brief explanation of the drawings]

[0019] The drawings herein are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the specification serve to explain the principles of the present disclosure.

[0020] In order to more clearly explain the technical solutions in the embodiments of the present disclosure or related technologies, the following briefly introduces drawings that need to be used in the embodiments or related technology descriptions, and it is obvious that those skilled in the art can also derive other drawings based on these drawings without exerting any creative efforts. [Figure 1] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. [Figure 2A] 10 is a flowchart of an image processing method according to another embodiment of the present disclosure. [Figure 2B] 10 is a flowchart of an image processing method according to another embodiment of the present disclosure. [Figure 3] 10 is a flowchart of an image processing method according to another embodiment of the present disclosure. [Figure 4] 1 is a structural schematic diagram of an image processing device according to an embodiment of the present disclosure; [Figure 5] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0021] In order to make the above-mentioned objects, features and advantages of the present disclosure more clearly understood, the present disclosure will be further described below. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0022] In the following description, numerous specific details are set forth to facilitate a thorough understanding of the present disclosure; however, the present disclosure may be implemented in other ways different from those described herein, and it is apparent that the embodiments in the specification are merely some embodiments of the present disclosure, but not all of the embodiments.

[0023] If the color display standards of multiple multimedia materials are the same, the video obtained by editing can meet the expected display effect. However, since the color space, luminance range, and other parameters of different color display standards are different, if the color display standards of multiple multimedia materials are not completely the same, the composite video obtained by editing and the preview during editing cannot achieve the expected display effect if they are displayed according to the original color display standards of each multimedia material. For example, the first multimedia material is multimedia material of the standard dynamic range (SDR) color display standard, and the second multimedia material is multimedia material of the high dynamic range (HDR) color display standard. The color space of the SDR color display standard is bt.709 and the brightness range is 0-100nit, while the color space of the HDR color display standard is bt.2020 and the brightness range is 0-10000nit. Therefore, after editing, if the first multimedia material is displayed according to the SDR color display standard and the second multimedia material is displayed according to the HDR color display standard, the visual difference between the two multimedia materials will be relatively large, the expected display effect will not be achieved, and the user experience will be greatly reduced.

[0024] Based on this, the present disclosure provides an image processing method, an apparatus, an electronic device, a readable storage medium, and a computer program product, in which the method converts multimedia materials of a non-target color display standard among a plurality of multimedia materials into multimedia materials of a target color display standard, thereby matching the color display standards of the plurality of multimedia materials, and in the composite video obtained thereby, each multimedia material is presented according to the target color display standard, matching the visual effects, and thereby realizing that the display effect of the multimedia materials meets expectations.

[0025] For example, the image processing method according to the present disclosure may be performed by an image processing device according to the present disclosure, and the image processing device may be implemented in any software and / or hardware manner. For example, the image processing device may include, but is not limited to, a tablet PC, a mobile phone (e.g., a foldable screen mobile phone, a large screen mobile phone, etc.), a laptop, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an Internet of Things (IoT) device such as a smart TV, a smart screen, a high-definition TV, a 4K TV, a smart projector, a server, a server cluster, a cloud server, etc., and the present disclosure does not impose any limitations on the specific type of electronic device.

[0026] The embodiments of the present disclosure use an electronic device as an example, in which a video editing application program (abbreviated as Application 1) is installed and multiple multimedia materials can be edited using Application 1. The image processing method of the present disclosure will be described in detail in conjunction with drawings and application scenarios.

[0027] The image processing method according to the present disclosure can be used in the following scenarios:

[0028] Scenario 1: Application 1 is launched and multimedia material is introduced for the first time.

[0029] Scenario 2: When you are editing multiple multimedia materials and want to add a new multimedia material.

[0030] Scenario 3: Inserting specified multimedia material at a freeze frame position using the freeze frame function provided by application 1. Here, the "freeze frame function" refers to saving a video frame image at a specified position as an image, and inserting a corresponding number of these images at the specified position according to the playback rate.

[0031] Of course, the method according to the present disclosure is not limited to being used in the above scenarios, and the above scenarios 1 to 3 are merely examples.

[0032] 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. Referring to FIG. 1, the method according to this embodiment includes the following steps:

[0033] S101, determining multimedia materials of non-target color display standards included in a plurality of multimedia materials according to a target color display standard;

[0034] The present disclosure does not limit the type of multimedia material, for example, the multimedia material may be any one of photos, images, videos, etc. The present disclosure does not limit the number of multimedia materials.

[0035] In addition, the color display standards corresponding to the plurality of multimedia materials may or may not be exactly the same, and the present disclosure is not limited thereto. For example, there are five multimedia materials, three of which are multimedia materials of the PQ color display standard, one of which is multimedia material of the HLG color display standard, and one of which is multimedia material of the SDR color display standard.

[0036] The present disclosure is not limited to the target color display standard and the non-target color display standard. Illustratively, the target color display standard is the PQ color display standard, and the non-target color display standard includes the SDR color display standard and / or the HLG color display standard, or the target color display standard is the SDR color display standard, and the non-target color display standard includes the HLG color display standard and / or the PQ color display standard, or the target color display standard is the HLG color display standard, and the non-target color display standard includes the SDR color display standard and / or the PQ color display standard.

[0037] It should be noted that the target color display standard and the non-target color display standard are not limited to the above cases, for example, the target color display standard may be a color display standard with a wider color space and a higher luminance range.

[0038] The present disclosure does not limit the implementation manner in which the electronic device acquires multiple multimedia materials. For example, the electronic device may launch Application 1 and acquire multiple multimedia materials from the local storage space of the electronic device through related entries provided by Application 1. Furthermore, while the electronic device acquires the multimedia materials, it also acquires related attribute information of each multimedia material, and determines a target color display standard corresponding to the multimedia materials based on the related attribute information of the multimedia materials.

[0039] For example, the related attribute information of the multimedia material may include color space information, color description information (color transfer information), etc., and the electronic device can determine the color display standard corresponding to the multimedia material based on the color description information corresponding to the multimedia material.

[0040] For example, assume that the format of color description information corresponding to multimedia material is as follows:

[0041] "color_transfer":"smpte2840" or "color_transfer": "arib-std-b67".

[0042] According to "smpte2840" or "arib-std-b67", it can be determined that the color display standard corresponding to this multimedia material is HDR.

[0043] Based on this, the application 1 can determine the color display standard corresponding to each of the multimedia materials among the plurality of multimedia materials by traversing the related information of the plurality of multimedia materials, and can determine the multimedia materials of non-target color display standards included in the plurality of multimedia materials by comparing the color display standard corresponding to each multimedia material with the target color display standard, where the number of multimedia materials of non-target color display standards may be one or more.

[0044] For example, if there are five multimedia materials, three of which are multimedia materials of the PQ color display standard, one of which is multimedia material of the HLG color display standard, and one of which is multimedia material of the SDR color display standard, and the target color display standard is the PQ color display standard, the multimedia materials of the non-target color display standard include multimedia materials of the HLG color display standard and multimedia materials of the SDR color display standard.

[0045] In some cases, when inserting an image at a specified position using the freeze frame function provided by application 1, if the color display standard of the original multimedia material of the acquired image differs from the color display standard of the image to be saved, a color difference will occur in the inserted image. For example, if the original multimedia material corresponding to the specified position is of HDR color display standard, but the electronic device only supports saving the image at the specified position as a JPEG format image, but the JPEG format image is of SDR color display standard, a color difference will occur between the inserted image and the original multimedia material.

[0046] Therefore, when the freeze frame function saves an image extracted from the original multimedia material, a description file can be generated for the image, and the description file can store image-related attribute information, such as information on the color display standard corresponding to the image source (i.e., the original multimedia material). When inserting the image, if it is determined based on the description file that the image originates from multimedia material of the target color display standard, it is possible to not convert the image even though it is saved as a non-target color display standard; and if it is determined based on the description file that the image originates from multimedia material of the non-target color display standard, it is determined that the image is multimedia material of the non-target color display standard that requires conversion.

[0047] That is, when one of the plurality of multimedia materials is generated based on another of the plurality of multimedia materials, the color display standard of the original multimedia material is used to determine whether the multimedia material needs to be converted.

[0048] S102, converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard.

[0049] In some embodiments, the implementation of step S102 may include the following steps.

[0050] Step a: Decode the multimedia material in the non-target color display standard to obtain color values ​​in the non-target color display standard of each pixel point of the multimedia material, that is, obtain a first set of color values.

[0051] Step b: converting each color value in the first color value set into an optical signal in the non-target color display standard according to an electro-optical transfer function (EOTF) corresponding to the non-target color display standard to obtain a first optical signal set.

[0052] Here, the electrical-to-optical conversion function is a function for describing the relationship between the non-linear color values ​​of the input display (digitally encoded pixel values, electrical signals) and the linear color values ​​(optical signals) displayed by the display.

[0053] Since what is stored and transmitted in the device is an electrical signal (i.e., color value), and the electrical-to-optical conversion functions corresponding to different color display standards are different, when converting the color values ​​of each pixel point of the multimedia material in the non-target color display standard to the color values ​​of the target color display standard, in order to realize the conversion of color values ​​of different color spaces in linear space, it is necessary to first convert the color values ​​of each pixel point of the multimedia material in the non-target color display standard into optical signals.

[0054] Step c, converting each optical signal in the first set of optical signals according to an optical-to-optical transfer function (OOTF) to obtain a second set of optical signals in the target color display standard.

[0055] There is a specific conversion function (which may be understood as a conversion matrix), i.e., a light-to-light conversion function, between optical signals of different color display standards, and the light-to-light conversion function can be used to convert optical signals of non-target color display standards into optical signals of the target color display standard.

[0056] Step d: converting each optical signal in the second set of optical signals into a color value according to an optical-to-electrical transfer function (OETF) corresponding to the target color display standard to obtain a second set of color values ​​in the target color display standard of the multimedia material.

[0057] By converting optical signals in the target color display standard into electrical signals (color values) in the target color display standard, it is possible to obtain color values ​​in the target color display standard for each pixel point of the multimedia material.

[0058] In the above conversion process, the multimedia material may be converted frame by frame.

[0059] For example, the target color display standard is the PQ color display standard, and the non-target color display standard is the HLG color display standard.

[0060] Here, multimedia materials conforming to the HLG color display standard are abbreviated as HLG materials, and multimedia materials conforming to the PQ color display standard are abbreviated as PQ materials.

[0061] First, the HLG material is decoded to obtain the RGB values ​​in the HLG color display standard of each pixel point of the HLG material. Then, based on the EOTF of the HLG color display standard, each RGB value in the HLG color display standard is converted into an optical signal in the HLG color display standard. Based on the optical-to-electrical conversion function between the HLG color display standard and the PQ color display standard, each optical signal in the HLG color display standard is converted into an optical signal in the PQ color display standard. Furthermore, based on the optical-to-electrical conversion function of the PQ color display standard, each optical signal in the PQ color display standard is converted into an RGB value in the PQ color display standard, thereby realizing conversion of the HLG material into SDR material.

[0062] It should be noted that when the target color display standard and the non-target color display standard are in other situations, the implementation methods are similar, and the difference lies in the differences in the adopted electrical-to-optical conversion function, optical-to-optical conversion function, and optical-to-electrical conversion function, and no examples will be given here to explain them one by one.

[0063] In some possible embodiments, after converting each optical signal of the non-target color display standard into an optical signal of the target color display standard by the above step c, a gamut mapping process may be performed to align color values ​​in different color display standards. The present disclosure is not limited to a specific implementation method of the gamut mapping process.

[0064] For example, if the target color display standard is the SDR color display standard and the non-target color display standard is the PQ color display standard, the same RGB values ​​represent different colors in the PQ color display standard and the SDR color display standard. Therefore, after converting an optical signal of the PQ color display standard into an optical signal of the SDR color display standard, a color gamut mapping process may be performed on each optical signal to align the color values ​​in the different color display standards.

[0065] For example, referring to the embodiment shown in FIG. 2B, when converting multimedia material in the SDR color display standard into multimedia material in the PQ color display standard, each optical signal in the SDR color display standard may be converted into an optical signal in the PQ color display standard using an optical-to-optical conversion function, and then a color gamut mapping process may be performed on each optical signal. Then, the optical signal obtained by the color gamut mapping process in the PQ color display standard may be converted into an electrical signal in the PQ color display standard.

[0066] Similarly, a color gamut mapping process may be performed when converting multimedia material in the PQ color display standard into multimedia material in the SDR color display standard, and when converting multimedia material in the HLG color display standard into multimedia material in the SDR color display standard, the implementation of which is similar to that shown in Figure 2B, and for the sake of brevity, will not be further described here.

[0067] In practical application, the application 1 can achieve step S102 by calling an application programming interface (API) provided by a function module that realizes the conversion between the different color display standards.

[0068] For example, a functional module that realizes conversion from multimedia material of the HLG color display standard to multimedia material of the PQ color display standard provides an API with the API name "TECoreHLG2PQCore", and application 1 can convert multimedia material of the HLG color display standard to multimedia material of the PQ color display standard by calling the API with the name "TECoreHLG2PQCore".

[0069] The functional module for realizing the conversion from multimedia material of the SDR color display standard to multimedia material of the PQ color display standard provides an API with the API name "TECoreSDR2PQCore", and by calling the API "TECoreSDR2PQCore", application 1 can realize the API to convert multimedia material of the SDR color display standard into multimedia material of the PQ color display standard.

[0070] In other situations where the target color display standard and the non-target color display standard are different, the manner in which the conversion is realized by the API is similar to this example, and for the sake of brevity, it will not be further described here.

[0071] S103, image-rendering the plurality of multimedia materials according to a target color display standard, and presenting the plurality of multimedia materials on a display screen of an electronic device according to the image-rendering result, or exporting the plurality of multimedia materials as a video file of a specific format according to the image-rendering result.

[0072] Visual rendering of multimedia material may be accomplished using any currently existing rendering technology, and this disclosure is not limited to any particular implementation for visual rendering.

[0073] In a scenario where multimedia materials are presented on a display screen of an electronic device, the electronic device can convert the RGB color values ​​of the pixel points of each multimedia material into optical signals for presentation according to an electro-optical conversion function and image rendering result corresponding to a target color display standard, and can also present some or all of the multiple multimedia materials according to different scenarios.

[0074] For example, when a user is editing a plurality of multimedia materials using application 1, when the user is positioned at a specific video frame position, the multimedia materials corresponding to this specific position can be displayed according to the target color display standard.

[0075] For example, when a user edits multiple multimedia materials using application 1, if playback of the multiple multimedia materials is triggered, the multiple multimedia materials can be played in sequence according to the order of the target color display standard.

[0076] When the multiple multimedia materials after color display standard conversion are exported as video files of a specific format, they can be encoded according to RGB color value information corresponding to the multimedia materials of the target color display standard based on the image rendering results and saved as video files of a specific format, for example, mp4 format.

[0077] The method according to this embodiment traverses a plurality of multimedia materials, determines multimedia materials of non-target color display standards contained in the plurality of multimedia materials according to a target color display standard, converts the multimedia materials of non-target color display standards into multimedia materials of the target color display standard, and performs image rendering of the plurality of multimedia materials according to the target color display standard, thereby presenting the plurality of multimedia materials on a display screen of an electronic device according to the image rendering result, or exporting the plurality of multimedia materials as a video file of a specific format according to the image rendering result, thereby realizing the display effect of the multimedia materials meeting expectations.

[0078] Based on the embodiment shown in FIG. 1, the multimedia materials may be clipped before being image-rendered based on a target color display standard, for example, according to a clip operation method corresponding to each of one or more clip types provided by an application program.

[0079] Here, the clip type includes, for example, a filter, an effect, a sticker, etc. The clip operation manner is a specific clip manner provided by the clip type. For example, the filter may include a landscape filter, a gourmet filter, etc.

[0080] 3 is a flowchart of an image processing method according to an embodiment of the present disclosure. Referring to FIG. 3, the method according to this embodiment includes the following steps:

[0081] S301, determining the target color display standard;

[0082] In some embodiments, the target color display standard may be determined based on an adaptation policy, i.e., the adaptation policy is a policy that represents changes in the target color display standard with respect to target color display standards corresponding to each of the plurality of multimedia materials. Exemplarily, the target color display standard may be a color display standard with the widest color space among the color display standards corresponding to each of the plurality of multimedia materials.

[0083] For example, when application 1 introduces three multimedia materials, the color display standards corresponding to the three multimedia materials are the same, and all three multimedia materials are multimedia materials of the SDR color display standard. During the video editing process, if a new multimedia material is introduced and this newly introduced multimedia material is multimedia material of the PQ color display standard, it can be determined based on the adaptation policy that the target color display standard is the PQ color display standard, that is, the three previously introduced multimedia materials need to be converted to the PQ color display standard.

[0084] In another possible embodiment, the target color display standard may be a pre-specified color display standard, for example, a color display standard specified by the application 1.

[0085] Here, when determining the target color display standard, it is necessary to take into account the color display standard supported by the electronic device to avoid errors. Therefore, before determining the target color display standard using the above-mentioned adaptation policy or a method specified by the application program, the color display standard supported by the screen display of the electronic device may be detected.

[0086] For example, the target color display standard that an electronic device supports for top screen display can be determined by checking related parameters in a specific API. For example, when determining whether an electronic device using the Android system supports the top screen display HDR color display standard, the display parameters supported by the screen of the electronic device are checked and the graphics API is checked. If both checks are successful, the electronic device is determined to support the top screen display HDR color display standard. This is merely an example, and a specific implementation method is not limited to checking the display parameters supported by the screen of the electronic device and checking the graphics API.

[0087] S302, determining multimedia materials of non-target color display standards included in the plurality of multimedia materials according to the target color display standard;

[0088] S303, converting the multimedia material of the non-target color display standard into the multimedia material of the target color display standard.

[0089] S304, image-rendering the plurality of multimedia materials based on the target color display standard, and presenting the plurality of multimedia materials on a display screen of an electronic device according to the image-rendering result, or exporting the plurality of multimedia materials as a video file of a specific format according to the image-rendering result.

[0090] Steps S302 to S304 in this embodiment are similar to steps S101 to S103 in the embodiment shown in FIG. 1, respectively, and reference can be made to the detailed description of the embodiment shown in FIG. 1, and for the sake of brevity, they will not be further described here.

[0091] The method of this embodiment traverses multiple multimedia materials, determines multimedia materials with non-target color display standards contained in the multiple multimedia materials according to a target color display standard, converts the non-target color display standard multimedia materials to multimedia materials with the target color display standard, and performs image rendering of the multiple multimedia materials according to the target color display standard. The image rendering results can be displayed on a screen or exported as a video file in a specific video format, thereby solving the problem of abnormal composite video display effects obtained by editing multimedia materials with different color display standards. Furthermore, during application, the target color display standard can be determined according to an adaptive policy or a specified manner, thereby improving the flexibility of image processing.

[0092] Illustratively, the present disclosure further provides an image processing device.

[0093] 4 is a structural schematic diagram of an image processing device according to an embodiment of the present disclosure. Referring to FIG. 4, the image processing device 400 according to this embodiment includes:

[0094] The identification module 401 is used to determine, based on the target color display standard, multimedia materials of non-target color display standards contained in the plurality of multimedia materials.

[0095] The conversion module 402 is used to convert the multimedia material in the non-target color display standard into multimedia material in the target color display standard.

[0096] The rendering module 403 is used to perform image rendering of the plurality of multimedia materials based on the target color display standard to obtain an image rendering result.

[0097] The image processing device 400 may further include a control module 404 for controlling to display the plurality of multimedia materials on a display screen of an electronic device according to the image rendering result, or to export the plurality of multimedia materials as a video file of a specific format according to the image rendering result.

[0098] In some embodiments, the identification module 401 is also used to determine a target color display standard before determining multimedia materials of non-target color display standards contained in the plurality of multimedia materials based on the target color display standard.

[0099] In some embodiments, the identification module 401 is specifically used to determine the target color display standard based on the color display standards corresponding to each of the plurality of multimedia materials.

[0100] In some embodiments, the identification module 401 is specifically used to determine the target color display standard based on the color display standard with the widest color space among the color display standards corresponding to each of the plurality of multimedia materials.

[0101] In some embodiments, the identification module 401 is specifically used to determine the target color display standard based on a pre-specified color display standard.

[0102] In some embodiments, the conversion module 402 is specifically used to convert multimedia material of the non-target color display standard into multimedia material of the target color display standard if the multimedia material of the non-target color display standard was generated based on a video frame image at a freeze frame position of another multimedia material of the non-target color display standard among the plurality of multimedia materials, or not convert the multimedia material of the non-target color display standard if the multimedia material of the non-target color display standard was generated based on a video frame image at the freeze frame position of another multimedia material of the target color display standard among the plurality of multimedia materials.

[0103] In some embodiments, the target color display standard is a PQ color display standard and the non-target color display standards include an SDR color display standard and / or an HLG color display standard, or the target color display standard is an SDR color display standard and the non-target color display standard includes an HLG color display standard and / or a PQ color display standard, or the target color display standard is an HLG color display standard and the non-target color display standard includes an SDR color display standard and / or a PQ color display standard.

[0104] In some embodiments, the image processing device 400 may further include a clip processing module for processing the plurality of multimedia materials according to a clip manipulation scheme corresponding to each of one or more clip types.

[0105] During the process of clipping the multimedia material, the rendering module 403 can perform real-time image rendering according to the processing result of the clip operation mode, and preview the clip effect on the display screen of the electronic device.

[0106] The image processing device according to this embodiment is used to implement any one of the above method embodiments, and based on this, its realization principles and technical effects are similar, and reference can be made to the detailed description of the above method embodiment, and for the sake of brevity, no further description will be given here.

[0107] Illustratively, the present disclosure further provides an electronic device.

[0108] 5 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Referring to FIG. 5, an electronic device 500 according to this embodiment includes a memory 501 and a processor 502.

[0109] Here, the memory 501 may be an independent physical unit, or may be connected to the processor 502 via a bus 503. The memory 501 and the processor 502 may be integrated and realized by hardware or the like.

[0110] The memory 501 is used to store program instructions, and the processor 502 calls the program instructions to execute the technical solutions of any one of the above method embodiments.

[0111] Alternatively, when some or all of the methods of the above embodiments are realized by software, the electronic device 500 may include only the processor 502. A memory 501 for storing a program is located outside the electronic device 500, and the processor 502 is connected to the memory via circuits / wires to read and execute the program stored in the memory.

[0112] The processor 502 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP.

[0113] The processor 502 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0114] The memory 501 may include volatile memory, such as random-access memory (RAM), or may include non-volatile memory, such as flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), or may further include a combination of the above types of memory.

[0115] The present disclosure further provides a readable storage medium, the readable storage medium including computer program instructions, The computer program instructions, when executed by at least one processor of an electronic device, implement the image processing method as set forth in any one of the method embodiments above.

[0116] The present disclosure further provides a computer program product, which, when executed by a computer, causes the computer to implement the image processing method as set forth in any one of the method embodiments above.

[0117] The present disclosure further provides a computer program, which includes instructions that, when executed by a processor, cause the processor to implement an image processing method as set forth in any one of the method embodiments above.

[0118] It should be explained that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply the existence of any such actual relationship or order between those entities or operations. Furthermore, the terms "comprise," "include," or any other variation thereof are intended to cover a non-exclusive "comprise," whereby a process, method, article, or device comprising a set of elements not only includes those elements, but also other elements not expressly listed, or further elements inherent in such process, method, article, or device. In the absence of further limitations, an element defined by the language "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes said element.

[0119] The foregoing are merely specific embodiments of the present disclosure that will enable those skilled in the art to understand or realize the present disclosure. Various modifications to these examples will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other examples without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the examples described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for hybrid editing of a plurality of multimedia materials having different color display standards, comprising: determining, based on the target color display standard, multimedia materials of a non-target color display standard included in the plurality of multimedia materials; converting the non-target color display standard multimedia material into the target color display standard multimedia material; rendering the plurality of multimedia materials into images, and presenting the plurality of multimedia materials on a display screen of an electronic device according to the image rendering result, or exporting the plurality of multimedia materials as a video file in a specific format according to the image rendering result; converting the multimedia material in the non-target color display standard into multimedia material in the target color display standard includes: not converting the multimedia material of the non-target color display standard when the multimedia material of the non-target color display standard is generated based on a video frame image at a freeze-frame position of the multimedia material of the target color display standard among the plurality of multimedia materials; The method, wherein the non-target color display standard multimedia material has a description file, and the description file stores information of the target color display standard corresponding to the video frame image at the freeze-frame position.

2. The method for hybrid editing a plurality of multimedia materials having different color display standards, before determining multimedia materials having non-target color display standards included in the plurality of multimedia materials based on the target color display standard, comprises: The method of claim 1 , further comprising determining the target color display standard.

3. determining the target color display standard includes: The method of claim 2 , further comprising determining the target color display standard based on a color display standard with the widest color space among color display standards corresponding to each of the plurality of multimedia materials.

4. determining the target color display standard includes:

3. The method of claim 2, further comprising determining the target color display standard based on a pre-specified color display standard.

5. converting the multimedia material in the non-target color display standard into multimedia material in the target color display standard includes:

2. The method of claim 1, further comprising converting the multimedia material of the non-target color display standard into multimedia material of the target color display standard if the multimedia material of the non-target color display standard was generated based on a video frame image at the freeze frame position of another multimedia material of the non-target color display standard among the plurality of multimedia materials.

6. converting the multimedia material in the non-target color display standard into multimedia material in the target color display standard includes: decoding the multimedia material in the non-target color display standard to obtain a first set of color values ​​in the non-target color display standard for each pixel point of the multimedia material; converting each color value in the first color value set into an optical signal in the non-target color display standard based on an electrical-optical conversion function corresponding to the non-target color display standard to obtain a first optical signal set; converting each optical signal in the first set of optical signals based on an optical-to-optical conversion function to obtain a second set of optical signals in the target color display standard; 2. The method of claim 1, further comprising: converting each optical signal in the second set of optical signals into a color value based on an opto-electrical conversion function corresponding to the target color display standard to obtain a second set of color values ​​in the target color display standard of the multimedia material.

7. 7. The method of claim 6, wherein after converting each optical signal in the first set of optical signals based on the optical-to-optical conversion function to obtain a second set of optical signals in the target color display standard, the method further comprises performing a color gamut mapping process to align color values ​​in different color display standards.

8. the target color display standard is a perceptually quantized (PQ) color display standard, and the non-target color display standard includes a standard dynamic range (SDR) color display standard and / or a hybrid log gamma curve (HLG) color display standard; or the target color display standard is an SDR color display standard, and the non-target color display standards include an HLG color display standard and / or a PQ color display standard; or The method of claim 1 , wherein the target color display standard is the HLG color display standard, and the non-target color display standards include the SDR color display standard and / or the PQ color display standard.

9. The method for hybrid editing of the plurality of multimedia materials having different color display standards before image rendering of the plurality of multimedia materials comprises:

10. The method of claim 1, further comprising processing the plurality of multimedia materials according to a clip manipulation scheme corresponding to each of one or more clip types.

10. An apparatus for hybrid editing of a plurality of multimedia materials having different color display standards, comprising: an identification module for determining, based on the target color display standard, multimedia materials of non-target color display standards included in the plurality of multimedia materials; a conversion module for converting the non-target color display standard multimedia material into the target color display standard multimedia material; a rendering module for graphically rendering the plurality of multimedia materials; a control module for controlling the electronic device to present the plurality of multimedia materials on a display screen according to the image rendering result, or for controlling the electronic device to export the plurality of multimedia materials as a video file in a specific format according to the image rendering result; The conversion module: When the multimedia material of the non-target color display standard is generated based on a video frame image at a freeze frame position of the multimedia material of the target color display standard among the plurality of multimedia materials, the multimedia material of the non-target color display standard is not converted; The apparatus, wherein the multimedia material of the non-target color display standard has a description file, and the description file stores information of the target color display standard corresponding to the video frame image at the freeze-frame position.

11. An electronic device, a memory and a processor, the memory configured to store computer program instructions; 10. An electronic device, wherein the processor is configured to execute the computer program instructions to cause the electronic device to perform the method of any one of claims 1 to 9.

12. A readable storage medium, comprising: Contains computer program instructions, A readable storage medium, the computer program instructions of which, when executed by at least one processor of an electronic device, cause the electronic device to implement the method of any one of claims 1 to 9.

13. A computer program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 9.

Images