Video display method, electronic device, computer program product and storage medium

By using SurfaceView and OpenGL in video editing applications of electronic devices, combined with MediaCodec, the problem of not being able to display HDR videos is solved, and the preview and editing functions of HDR videos are realized, the device's image processing capabilities are improved and the video type switching is supported.

CN117692714BActive Publication Date: 2024-11-08HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310852349.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-11
Publication Date
2024-11-08
Estimated Expiration
2043-07-11

AI Technical Summary

Technical Problem

Existing electronic devices cannot display HDR videos in the preview area of ​​the video editing application, limiting users' editing functions for HDR videos.

Method used

By using SurfaceView as a preview area control in the video editing application and setting its layer attributes to the BT2020 color gamut, combining OpenGL and MediaCodec, the decoding and format conversion of HDR videos can be realized to ensure the correct display and editing of the video in the preview area. .

Benefits of technology

It realizes the preview and editing functions of HDR videos in video editing applications, improves the image processing capabilities of electronic devices, can display HDR videos in the preview area, and supports switching between HDR videos and SDR videos.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117692714B_ABST
    Figure CN117692714B_ABST
Patent Text Reader

Abstract

The present application provides a video display method and electronic device. The method can be applied to electronic devices with image processing capabilities, such as smart phones, tablet computers, etc. When the video editing application cannot display the video frames of the HDR video normally, the electronic device uses SurfaceView as the display control of the preview area of ​​the video editing application, and sets the property of the layer to which SurfaceView belongs to BT2020, so that the HDR video can be displayed in the user interface of the video editing application.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a video display method and an electronic device. Background Art

[0002] With the development of electronic technology, electronic devices such as mobile phones and tablets can support the shooting, storage and playback of various types of videos, such as Standard Dynamic Range (SDR) video, High-Dynamic Range (HDR) video, etc. Compared with SDR video, HDR video can include richer color effects and record more image details, so that HDR video can present an excellent viewing effect.

[0003] For SDR video, in response to a video editing operation for a certain Standard Dynamic Range (SDR) video, the electronic device can perform video editing on the SDR video, such as cropping, adding text, etc. The video editing application on the electronic device can edit the SDR video, but the video editing application cannot edit the HDR video, and thus the HDR video cannot be displayed in the preview area of ​​the video editing application. Summary of the invention

[0004] The present application provides a video display method and an electronic device, which can display HDR video in a preview area of ​​a video editing application, thereby helping to improve the image processing capability of the electronic device.

[0005] In a first aspect, the present application provides a video display method, which is applied to an electronic device, and the method includes: displaying a first user interface, the first user interface including a first display window and an editing control for displaying a first video, and the video type of the first video is an HDR video; detecting a first operation acting on the editing control, and in response to the first operation: creating a SurfaceView, and applying for a first Surface for the SurfaceView; creating a decoder; applying for a second Surface, and binding the first Surface to the second Surface; decoding the first video into N first video frames through the decoder; format conversion of the N first video frames through OpenGL to obtain N second video frames output by OpenGL; calling a graphics processor (Graphics Processing Unit, GPU) converts the format of N second video frames to obtain N third video frames, and outputs the N third video frames to the second Surface; based on the binding between the first Surface and the second Surface, outputs the N third video frames on the second Surface to the first Surface; sets the attribute of the layer to which SurfaceView belongs to BT2020, and based on the N third video frames on the first Surface and the attribute of the layer to which SurfaceView belongs, synthesizes a preview video of the first video, and the video type of the preview video is HDR video; displays a second user interface, the second user interface includes a second display window, the second display window is used to display the preview video, and the display control of the second display window is SurfaceView.

[0006] By implementing the video display method provided in the first aspect, the electronic device uses SurfaceView as a display control in the preview area of ​​the video editing application, and sets the property of the layer to which SurfaceView belongs to BT2020, so that HDR video can be displayed in the preview area of ​​the video editing application, thereby realizing the editing of HDR video, which helps to improve the image processing capability of the electronic device.

[0007] In combination with the method provided in the first aspect, in some embodiments, the format of the first video frame includes: the color encoding format is a YUV format, the data type of the color value is an integer, and the color gamut is BT2020; the format of the first video frame can be expressed as (YUV, INT, BT2020);

[0008] The format of the second video frame includes: the color encoding format is RGB format, the data type of the color value is floating point type, and the color gamut is BT2020; the format of the second video frame can be expressed as (RGB, FLOAT, BT2020);

[0009] The format of the third video frame includes: the color encoding format is the YUV format, the data type of the color value is an integer, and the color gamut is BT2020; the format of the third video frame can be expressed as (YUV, INT, BT2020).

[0010] Optionally, the first video frame, the second video frame and the third video frame are all video frames encoded by a perceptual quantizer (PQ), and thus the format of the first video frame can be expressed as (YUV, INT, BT2020, PQ), the format of the second video frame can be expressed as (RGB, FLOAT, BT2020, PQ), and the format of the third video frame can be expressed as (YUV, INT, BT2020, PQ).

[0011] In combination with the method provided in the first aspect, in some embodiments, the second user interface also includes a video type control, which is used to indicate that the video type of the preview video is an HDR video, so that the user knows that the preview video displayed in the second user interface is an HDR video.

[0012] In combination with the method provided in the first aspect, in some embodiments, the video type control in the second user interface can not only indicate the video type of the preview video, but also, when a second operation acting on the video type control is detected, a video type adjustment window is output in the second user interface in response to the second operation;

[0013] The video type adjustment window includes at least one of the following options: export format, video type, resolution, and frame rate; the export format includes video format options and GIF format options; the video type includes HDR video type options and common video type options; the resolution includes 1080P options and 2K / 4K options; the frame rate includes 24 options, 25 options, 30 options, 50 options, and 60 options. Common video type is SDR video type.

[0014] The video type adjustment window provides users with a variety of options so that users can adjust the video type, resolution, frame rate, etc. of the preview video according to their needs, which can further improve the image processing capabilities of electronic devices.

[0015] In combination with the method provided in the first aspect, in some embodiments, after the second user interface outputs the video type adjustment window, the method further includes: detecting a switching operation of the video type in the video type adjustment window, and in response to the switching operation: converting the preview video from the HDR video to the SDR video; and displaying the SDR video of the first video in the second display window. Thus, the preview video is switched from the HDR video to the SDR video, so that the preview area of ​​the video editing application can display both the HDR video and the SDR video, and can also switch between the two. For example, the HDR video type is selected before the switching operation, and the switching operation may be an operation of clicking on the ordinary video type; for another example, the ordinary video type is selected before the switching operation, and the switching operation may be an operation of clicking on the HDR video type.

[0016] In combination with the method provided in the first aspect, in some embodiments, converting the preview video from an HDR video to an SDR video includes: converting the HDR nonlinear electrical signal of the third video frame n into an HDR linear optical signal through an electro-optical conversion function EOTF; performing color space conversion on the HDR linear optical signal; performing tone mapping on the HDR linear optical signal after the color space conversion to obtain an SDR nonlinear optical signal; converting the SDR linear optical signal into an SDR nonlinear electrical signal through an optoelectronic conversion function to obtain an SDR nonlinear electrical signal of the third video frame n; wherein the third video frame n is any third video frame among the N third video frames. This process can quickly convert the preview video from an HDR video to an SDR video to reduce the waiting time of the user.

[0017] In combination with the method provided in the first aspect, in some embodiments, the EOTF is related to the brightness parameter of the third video frame n, and the brightness parameter of the third video frame n is the maximum brightness of the third video frame n. That is to say, when the electro-optical conversion is performed on each third video frame, the brightness parameter is different, which can reflect the difference in conversion. During the electro-optical conversion, considering the brightness parameter can improve the contrast between the HDR video frame and the SDR video frame.

[0018] In combination with the method provided in the first aspect, in some embodiments, before converting the HDR nonlinear electrical signal of the third video frame n into an HDR linear optical signal, the method further includes: dividing the third video frame n into L grouping areas, and calculating the maximum brightness of each grouping area in the L grouping areas to obtain L maximum brightness values; M is a positive integer; selecting the first K maximum brightness values ​​from the L maximum brightness values ​​in order from large to small, K is an integer greater than 1 and less than M; calculating the average of the K maximum brightness values, the average value being the maximum brightness of the third video frame n. The brightness parameter determined in this way helps to improve the contrast between the HDR video frame and the SDR video frame.

[0019] In combination with the method provided in the first aspect, in some embodiments, the electronic device includes a video editing application, an application architecture layer and a memory; creating a SurfaceView and applying for a first Surface for the SurfaceView may specifically include: the video editing application sends a first request to the application architecture layer, the first request is used to request the creation of a SurfaceView; the application architecture layer responds to the first request, creates a SurfaceView, and sends a second request to the memory, the second request is used to request the memory to allocate a first Surface; the memory responds to the second request, allocates the first Surface, and sends a second response to the application architecture layer, the second response includes identification information of the first Surface; the application architecture layer sends a first response to the video editing application, the first response includes identification information of the first Surface.

[0020] In the process of creating SurfaceView, the video editing application obtains the identification information of the Surface in SurfaceView from the memory, that is, the identification information of the first Surface, and the first Surface is used to display the HDR color gamut, that is, the BT2020 color gamut.

[0021] In combination with the method provided in the first aspect, in some embodiments, the electronic device also includes an open graphics library OpenGL; applying for a second Surface, and binding the first Surface to the second Surface, may specifically include: a video editing application sends a first instruction to OpenGL, the first instruction is used to instruct OpenGL to apply for the second Surface from the memory; OpenGL responds to the first instruction and sends a third request to the memory, the third request is used to request the memory to allocate the second Surface; the memory responds to the third request, allocates the second Surface, and sends a third response to OpenGL, the third response includes identification information of the second Surface; OpenGL binds the first Surface to the second Surface.

[0022] OpenGL binds the first Surface to the second Surface so that OpenGL controls the GPU to transfer the video frame on the second Surface to the first Surface, thereby displaying the HDR video frame or HDR video through SurfaceView.

[0023] In combination with the method provided in the first aspect, in some embodiments, the electronic device also includes MediaCodec, which is used to create a decoder; after creating the decoder, the above method also includes: the video editing application sends a fourth request to MediaCodec, and the fourth request is used to request a third Surface; the third Surface is used to store the N first video frames output by the decoder; MediaCodec responds to the fourth request and sends a fifth request to the memory, and the fifth request is used to request the memory to allocate a third Surface; the memory responds to the fifth request, allocates the third Surface, and sends a fifth response to MediaCodec, and the fifth response includes identification information of the third Surface; MediaCodec sends a fourth response to the video editing application, and the fourth response includes identification information of the third Surface.

[0024] In combination with the method provided in the first aspect, in some embodiments, the method also includes: the decoder created by MediaCodec sends N first video frames to the memory; the memory writes the N first video frames to the third Surface; the N first video frames are formatted through OpenGL to obtain N second video frames output by OpenGL, including: the video editing application sends a second instruction to OpenGL, and the second instruction is used to instruct OpenGL to read the N first video frames from the third Surface; OpenGL responds to the second instruction, reads the N first video frames from the third Surface, and formats the N first video frames to obtain N second video frames output by OpenGL; OpenGL stores the N second video frames in the first video memory.

[0025] In combination with the method provided in the first aspect, in some embodiments, calling the GPU of the electronic device to perform format conversion on N second video frames to obtain N third video frames, and outputting the N third video frames to the second Surface, specifically includes: OpenGL sends a third instruction to the GPU, the third instruction is used to instruct the GPU to perform format conversion on the N second video frames; the GPU responds to the third instruction, performs format conversion on the N second video frames stored in the first video memory, to obtain N third video frames; OpenGL sends a fourth instruction to the GPU, the fourth instruction is used to instruct the GPU to output the N third video frames to the second Surface; the fourth instruction includes identification information of the second Surface; the GPU responds to the fourth instruction, and outputs the N third video frames to the second Surface based on the identification information of the second Surface.

[0026] In combination with the method provided in the first aspect, in some embodiments, the third instruction or the fourth instruction includes identification information of the first Surface bound to the second Surface;

[0027] Based on the binding between the first Surface and the second Surface, the N third video frames on the second Surface are output to the first Surface, specifically including: the GPU determines the binding between the first Surface and the second Surface based on the third instruction or the fourth instruction, and based on the binding between the first Surface and the second Surface, outputs the N third video frames on the second Surface to the first Surface.

[0028] In combination with the method provided in the first aspect, in some embodiments, the electronic device further includes a hardware abstraction layer; the layer to which SurfaceView belongs is set to a BT2020 attribute, and a preview video of the first video is synthesized based on the N third video frames and the BT2020 attribute on the first Surface, specifically including: the application architecture layer monitors data changes on the first Surface, and sets the layer to which SurfaceView belongs to a BT2020 attribute; the application architecture layer sends the layer set to the BT2020 attribute to the hardware abstraction layer; the hardware abstraction layer synthesizes a preview video of the first video based on the N third video frames and the BT2020 attribute on the first Surface. Thus, the second display window of the second user interface can display HDR video.

[0029] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0030] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0031] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0032] It can be understood that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in the first aspect of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A-1F is a set of user interface schematic diagrams provided in an embodiment of the present application;

[0034] Figure 2 is a software architecture diagram of an electronic device provided in an embodiment of the present application;

[0035] Figure 3 is a flow chart of a video display method provided in an embodiment of the present application;

[0036] Figure 4 is a flow chart of initializing a video editing environment of an electronic device provided by an embodiment of the present application;

[0037] Figure 5 is a flowchart of an electronic device providing an embodiment of the present application generating an HDR video frame of an HDR video to be edited;

[0038] Figure 6 is a flowchart of an electronic device provided in an embodiment of the present application converting an HDR video frame into an SDR video frame;

[0039] Figure 7 It is a hardware structure diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0040] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0041] 1. SurfaceView and TextureView

[0042] SurfaceView inherits from the View class. It is essentially a View, but it has its own Surface. Therefore, SurfaceView has a corresponding WindowState in the System Management Service (Window Manager Service, WMS) and a layer in SurfaceFlinger. The rendering of SurfaceView can be done in a separate thread, and it can have its own GL context when rendering. Because SurfaceView does not affect the main thread's response to time, it can be drawn in an independent thread without affecting the main thread, and the double buffering mechanism is used, so the picture is smoother when playing videos. In other words, the rendering of SurfaceView can be placed in a separate thread instead of the main thread.

[0043] TextureView also inherits from the View class. It is also a View in essence. It can project content streams directly into the View and can be used to implement Live preview and other functions. TextureView does not create a separate window in WMS, but is a normal View in the View hierarchy. Therefore, it can be moved, rotated, scaled, animated, and other changes like other normal Views. Because TextureView is drawn in the View hierarchy, TextureView is generally rendered in the main thread.

[0044] Since SurfaceView's rendering can be done in a separate thread, SurfaceView can pull out the video player's screen for rendering separately, while TextureView generally renders in the main thread, so TextureView cannot pull out the video player's screen for rendering separately.

[0045] In view of the advantages of SurfaceView, the embodiment of the present application adopts SurfaceView in the application framework layer.

[0046] 2. Perceptual Quantizer (PQ)

[0047] PQ is an efficient way to encode HDR brightness information. In the entire dynamic range, the difference between each pair of adjacent coded values ​​is slightly smaller than the perceptible difference, making the code value utilization extremely high. PQ-encoded signals can be decoded on devices that support HDR. The HDR videos involved in the embodiments of this application are all PQ-encoded HDR videos.

[0048] 3. Color gamut

[0049] Color gamut refers to the range of colors that can be displayed when the video is encoded. SDR videos use the BT709 color gamut; HDR videos use the BT2020 color gamut. Therefore, compared to SDR videos, HDR videos can use more types of colors, a wider range of colors, and a higher range of displayed brightness. Furthermore, HDR videos can support richer image color expressions and more vivid image detail expressions. This also enables HDR videos to provide users with viewing effects, thereby improving the user experience.

[0050] Not limited to BT2020 color gamut and BT709 color gamut, HDR video and SDR video may also use other types of color gamut. But in general, the color gamut used by HDR video is wider than that used by SDR video, and the colors are richer and the details are richer.

[0051] Generally, the type of video supported by electronic devices such as mobile phones and tablet computers (hereinafter referred to as electronic devices 100) is SDR video. With the development of shooting technology and image technology, the electronic device 100 not only supports shooting SDR video, but also supports shooting HDR video. In this way, the user's demand for editing HDR video also emerges.

[0052] Currently, the video editing application used by the electronic device 100 to edit HDR videos is an SDR video editing application. The SDR video editing application cannot normally display the color gamut of the HDR video, nor can it edit the HDR video.

[0053] In order to solve the above problem, an embodiment of the present application provides a video display method. The method can be applied to an electronic device with the image processing capability, such as a mobile phone, a tablet computer, etc.

[0054] By implementing the video display method provided in the embodiment of the present application, the electronic device 100 can use SurfaceView as a preview area control of a video editing application, and set the layer to which SurfaceView belongs to the BT2020 color gamut, so that the electronic device 100 can display HDR video in the user interface of the video editing application, specifically, can display HDR video in the video preview area of ​​the user interface.

[0055] Furthermore, when the electronic device 100 detects a video editing operation acting on the editing control in the user interface, it can respond to the video editing operation and add filters to the HDR video. Video editing operations include but are not limited to adding filters. For example, video editing operations can also include cropping, image inversion, scaling, adding text, adding filters, adding titles (or endings or other pages), adding video watermarks or stickers, etc.

[0056] In implementing the video display method provided in the embodiment of the present application, the user interface may further include a video type control, the electronic device 100 detects a user operation acting on the video type control, and in response to the user operation, outputs a video type adjustment window on the user interface, the video type adjustment window including the video type. The electronic device 100 detects a switching operation acting on the video type, and in response to the switching operation, converts the HDR video to the SDR video, and switches the HDR video displayed in the video preview area of ​​the user interface to the SDR video.

[0057] The electronic device 100 is not limited to mobile phones and tablet computers. It can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0058] Figure 1A-1F An exemplary embodiment of a group of user interfaces on the electronic device 100 is shown below. Figure 1A-1F The application scenarios of the video display method provided by the embodiments of the present application are specifically introduced.

[0059] first, Figure 1A The user interface of the electronic device 100 displaying the installed applications, i.e., the home page, is shown as an example. Figure 1A As shown, one or more application icons are displayed in the main page, such as a "clock" application icon, a "calendar" application icon, a "weather" application icon, and the like.

[0060] The one or more application icons include a "Gallery" application (hereinafter referred to as "Gallery") icon, namely, icon 111. The electronic device 100 can detect a user operation on icon 111. The operation is, for example, a click operation. In response to the operation, the electronic device 100 can display Figure 1B The user interface shown.

[0061] Figure 1B The main interface of the "Gallery" when the "Gallery" is run on the electronic device 100 is exemplarily shown. The interface may display one or more pictures or videos. Among them, the one or more videos include HDR videos, LOG videos, and other types of videos, such as SDR videos. The above-mentioned LOG video refers to a low-saturation, low-brightness video shot in LOG gray mode, which can also be called a LOG gray film.

[0062] like Figure 1BAs shown, the video indicated by icon 121 may be a LOG video; the video indicated by icon 122 may be an HDR video; and the video indicated by icon 123 may be an SDR video. When the electronic device 100 displays an HDR video or a LOG video, the icon indicating the video may display the type of the video. In this way, the user can understand the type of the video through the information displayed in the icon. For example, LOG is displayed in the lower left corner of icon 121; HDR is displayed in the lower left corner of icon 122. Figure 1B Videos not marked as HDR or LOG are SDR videos.

[0063] The electronic device 100 may detect a user operation on the icon 122, and in response to the operation, the electronic device 100 may display Figure 1C The user interface shown. Figure 1C A user interface for the electronic device 100 to specifically display a certain picture or video.

[0064] like Figure 1C As shown, the user interface may include a window 131. Window 131 may be used to display the video selected by the user for viewing. Figure 1B In the example, the video selected for browsing by the user is the HDR video (“Video M”) indicated by icon 122 . Then, “Video M” may be displayed in window 131 .

[0065] The user interface also includes an icon 132 and a control 133. The icon 132 may be used to indicate the type of video displayed in the window 131. For example, the "HDR" currently displayed in the icon 132 may indicate that the "Video M" is a video of the HDR type.

[0066] The control 133 may be used to receive a user's operation of editing a video (or picture), and upon detecting the operation, the electronic device 100 may respond to the operation and display a user interface for editing the video (or picture). Generally, if the video editing application of the electronic device 100 does not support processing HDR videos, the electronic device 100 will not provide the user with a function of editing HDR videos. Figure 1C Control 133 is generally not included, that is, the electronic device 100 will not provide the user with controls for editing the video, because the electronic device 100 cannot output and save the edited HDR video.

[0067] In the embodiment of the present application, the electronic device 100 can display HDR video and can provide the user with the function of editing HDR video to meet the user's editing needs and improve the user experience. Figure 1C In the user interface shown, the electronic device 100 can display the control 133 and can respond to the user operation acting on the control 133.

[0068] The user interface may also include a control 134 , a share control 135 , a favorite control 136 , a delete control 137 , and the like.

[0069] The control 134 may be used to display detailed information of the video, such as shooting time, shooting location, color encoding format, bit rate, frame rate, pixel size, etc.

[0070] The sharing control 135 can be used to send the video M to other applications for use. For example, after detecting a user operation on the sharing control, in response to the operation, the electronic device 100 can display one or more application icons, and the one or more application icons include the icon of the social software 1. After detecting an application icon acting on the social software 1, in response to the operation, the electronic device 100 can send the video M to the social software 1, and further, the user can share the video with friends through the social software.

[0071] The favorites control 136 can be used to mark a video. Figure 1C In the user interface shown, when a user operation acting on the favorite control is detected, in response to the operation, the electronic device 100 may mark the video M as a favorite video of the user. The electronic device 100 may generate an album for displaying the videos marked as favorite by the user. In this way, when the video M is marked as a favorite video of the user, the user can quickly view the video M through the album displaying the favorite videos of the user.

[0072] The delete control 137 can be used to delete the video M.

[0073] When a user operation on the control 133 is detected, the electronic device 100 may display Figure 1D The user interface shown. Figure 1D The user interface of a video editing application is shown as an example. Figure 1D As shown, the user interface may include window 141 , window 142 , operation bar 143 , and operation bar 144 .

[0074] Window 141 can be used to display a preview video of the video to be edited or the video being edited. Window 141 is a preview area of ​​the user interface of the video editing application. Generally, window 141 will display the cover video frame of the video to be edited. When a user operation acting on the play button 145 is detected, window 141 can display the video frame stream of the video in sequence, that is, play the video. In an embodiment of the present application, window 141 can be used to display a preview video of the HDR video to be edited or the HDR video being edited, that is, window 141 can be used to display a preview video of the HDR type.

[0075] Window 142 may be used to display a video frame stream of the edited video. The user may drag window 142 to adjust the video frame displayed in window 141. Specifically, Figure 1D A ruler 147 is also displayed. The electronic device 100 can detect a user operation of sliding left or right on the window 142. In response to the user operation, the position of the video frame stream where the ruler 147 is located is different. At this time, the electronic device 100 can display the video frame where the current ruler 147 is located in the window 141.

[0076] The operation bar 143 and the operation bar 144 may display icons of multiple video editing operations. Generally, an icon displayed in the operation bar 143 indicates a major category of editing operations. The operation bar 144 may display video editing operations belonging to the major category of operations currently selected in the operation bar 143. For example, the operation bar 143 includes "Clip". The bold "Clip" may indicate that the type of video editing operation currently selected by the user is "Clip". At this time, the operation bar 144 displays some operations belonging to the "Clip" category, such as "Split", "Cut", "Volume", "Frame", etc.

[0077] For example, the electronic device 100 may detect a user operation on a "split" control, and in response to the operation, the electronic device 100 may display one or more operation controls for splitting the video. The electronic device 100 may record the user's splitting operation, such as the start time and end time of the first video segment, the start time and end time of the second video segment, and the like.

[0078] For another example, the electronic device 100 may detect a user operation on a "frame" control. In response to the operation, the electronic device 100 may record the size of the video screen set by the user, and then crop the original video frame.

[0079] The operation bar 144 corresponding to the "edit" operation also includes other editing controls belonging to the "edit" category. In response to user operations acting on the above controls, the electronic device 100 can record and execute the video editing operations corresponding to the above controls, which are not listed one by one here.

[0080] The user interface also includes an export control 146. When a user operation acting on the export control 146 is detected, in response to the operation, the electronic device 100 can export the video in the current state to a corresponding file directory. The video in the current state can be a video with an editing operation attached, or a video without an editing operation. The export control 146 can also be a save control for saving the video in the current state.

[0081] In the embodiment of the present application, the user interface also includes a video type control 148. When a user operation acting on the control 133 is detected, the video type control 148 may display "HDR" by default (e.g. Figure 1D141 is an HDR video. Alternatively, when a user operation on control 133 is detected, video type control 148 may display "HDR|1080P" by default, indicating that the video type displayed in window 141 is an HDR video with a resolution of 1080P. Alternatively, when a user operation on control 133 is detected, video type control 148 may display "HDR|2K / 4K" by default, indicating that the video type displayed in window 141 is an HDR video with a resolution of 2K / 4K.

[0082] The video type control 148 may provide options for export format, video type, resolution, frame rate, etc.

[0083] The electronic device 100 may detect a user operation on the video type control 148, and in response to the operation, the electronic device 100 may output a video type control 148 in the user interface of the video editing application. Figure 1E The video type adjustment window is shown. Figure 1E The electronic device 100 outputs a video type adjustment window in the user interface of the video editing application. Figure 1E The window 149 in the video may completely cover the window 141 or may cover part of the content of the window 141. The size of the window depends on the displayed content. For example, the window may display the content of the export format, video type, resolution and frame rate, thereby providing the user with options to adjust these contents.

[0084] Among them, the export format can be divided into video format and GIF animation format. Figure 1E Take selecting a video format as an example.

[0085] Video types can be divided into HDR video types and ordinary video types. Figure 1E Taking the selection of HDR video type as an example, the normal video type is the SDR video type. Optionally, when the video type is the HDR video type, the window 149 may also display the text of "High Dynamic Range"; when the video type is the normal video type, the window 149 may also display the text of "Standard Dynamic Range" so that the user can select the video type according to the needs. The specific text display position may be located above the "Normal Video" text and flush with the "Video Type" text.

[0086] The resolution can be divided into 1080P and 2K / 4K. Figure 1ETake 2K / 4K as an example. Optionally, when the resolution is 1080P, window 149 may also display the text "poor image restoration effect"; when the resolution is 2K / 4K, window 149 may also display the text "extreme image restoration, large space occupancy" so that the user can select the resolution according to needs. The specific text display position may be located above the "2K / 4K" text and flush with the "resolution" text.

[0087] Frames Per Second (FPS) can be divided into 24, 25, 30, 50 and 60. Figure 1E Take 30 as an example. FPS is a definition in the field of images, which refers to the number of frames transmitted per second. Generally speaking, it refers to the number of frames of animation or video. The higher the frame rate, the smoother the playback. Optionally, window 149 can also display the text "The higher the frame rate, the smoother the playback", so that the user can select the frame rate according to needs. The specific text display position can be located above the text "50 60", flush with the "Frame Rate (FPS)".

[0088] Window 149 may also display the size of the video, which may be related to the selected export format, video type, resolution, and frame rate.

[0089] Figure 1E The content displayed by the video type control 148 in the video type control 148 may change as the video type and resolution change. For example, if the video type is an HDR video type and the resolution is 2K / 4K, the video type control 148 may display "HDR|2K / 4K". For another example, if the video type is a normal video type and the resolution is 1080P, the video type control 148 may display "SDR|1080P".

[0090] The electronic device 100 may detect a switching operation (or described as a video type switching operation) acting on the “video type” in the window 149, i.e., switching from the HDR video type to the normal video type. In response to the operation, the electronic device 100 may convert the HDR video into an SDR video and display the video. Figure 1F The user interface shown. Figure 1F An example of a user interface for editing an "SDR|2K / 4K" video is shown. Figure 1F The video type displayed in the window 141 is SDR video, and the content displayed in the video type control 148 is "SDR|2K / 4K".

[0091] The electronic device 100 can detect the switching operation of the "video type" and the switching operation of the "resolution" in the window 149, that is, switching from the HDR video type to the ordinary video type and from 2K / 4K to 1080P. Then, in response to these two switching operations, the electronic device 100 can convert the HDR video into an SDR video and adjust the resolution to 1080P, so that the content displayed by the video type control 148 is "SDR|1080P".

[0092] The electronic device 100 can detect the Figure 1F In response to a user operation of the video type control 148 in the video editing application, a video type adjustment window is output in the user interface of the video editing application. If the electronic device 100 detects a switching operation of the “video type” in the video type adjustment window, i.e., switching from a normal video type to an HDR video type, in response to the operation, the SDR video can be converted to an HDR video, and the content displayed by the video type control 148 can be adjusted to “HDR|2K / 4K”.

[0093] By using the embodiment of the present application, the electronic device 100 can achieve Figure 1D-1F In the user interface shown, the preview area in the user interface of the video editing application on the electronic device 100 can display HDR video, and can also switch the HDR video displayed in the preview area to SDR video according to the video type switching operation. Thus, the preview area can support the display of HDR video so as to implement video editing of HDR video. The electronic device 100 also supports converting HDR video to SDR video, thereby displaying SDR video.

[0094] The following describes in detail the electronic device 100 Figure 1D-1F The specific process of the user interface is shown.

[0095] first, Figure 2 The software architecture of the electronic device 100 is exemplarily shown.

[0096] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0097] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, the hardware abstraction layer (HAL), and the kernel layer.

[0098] The application layer can include a series of application packages. Figure 2 As shown, the application package may include applications such as camera, gallery, video, music, navigation, calendar, map, WLAN, etc. In the embodiment of the present application, the application layer also includes a video editing application. The video editing application has video data processing capabilities and can provide users with video editing functions, including cropping, rendering, and other video data processing. The video editing application in the embodiment of the present application can realize the editing of HDR videos, and can also realize the editing of SDR videos. Figure 1D-1F The user interface shown can be regarded as a user interface provided by the above-mentioned video editing application. The video editing application can also be described as a video editor.

[0099] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0100] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0101] The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the display interface of an application. The display interface can be composed of one or more views. For example, the display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0102] In an embodiment of the present application, the view system can create a SurfaceView, and SurfaceView can pull out the screen of the video player separately for rendering. The view system can also apply for a Surface from the memory, and the Surface is used to display the color gamut of the HDR video, such as the BT2020 color gamut. The view system can also set the layer to which the SurfaceView belongs to the attribute of BT2020 when it detects that there is a data change on the Surface, and send the layer to which the set SurfaceView belongs to the hardware abstraction layer so that the hardware abstraction layer can synthesize the HDR video frame. The phone manager is used to provide communication functions for the electronic device 100. For example, the management of call status (including connecting, hanging up, etc.). The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0103] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0104] In the embodiment of the present application, the application framework layer also includes a media framework. The media framework provides multiple tools for editing video and audio. Among them, the above tools include MediaCodec. MediaCodec is a class provided by Android for encoding and decoding audio and video. It includes an encoder and a decoder.

[0105] Among them, the encoder can convert one form of video or audio input to the encoder into another form through compression technology, and the decoder performs the reverse process of encoding and can convert one form of video or audio input to the decoder into another form through decompression technology.

[0106] For example, the video input to the encoder may be an HDR video. The HDR video is composed of N HDR video frames with a color gamut of BT2020. The N is an integer greater than 1. After receiving the HDR video, the decoder may split the video composed of the N HDR video frames with a color gamut of BT2020 into N independent HDR video frames, so that the subsequent electronic device 100 can perform image processing on each HDR video frame.

[0107] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0108] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0109] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (such as OpenGL or OpenGL ES, etc.), 2D graphics engine (such as SGL), etc. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0110] The Open Graphics Library (OpenGL) provides multiple image rendering functions that can be used to draw simple graphics to complex three-dimensional scenes. The OpenGL provided by the system library may include a GLES rendering module for executing the OpenGL ES pipeline rendering process. In the OpenGL ES pipeline rendering process, vertex coordinates, texture coordinates, and texture images can be input from the CPU to the GPU, and parameters can also be output to the GPU. The entire rendering process inputs data (i.e., vertex coordinates, texture coordinates, and texture images) from the CPU to the GPU, and then executes them in turn in each component module of the GPU, and finally outputs a block of cached data. In an embodiment of the present application, the GLES rendering module can output video frames in PQ-encoded (BT2020, RGB) format.

[0111] In the embodiment of the present application, the system library also includes EGL. EGL is an interface, i.e., a bridge, between a graphics rendering API (e.g., OpenGL ES) and a native platform (e.g., Android) view system. EGL can provide a mechanism for creating a rendering Surface, and can also provide synchronization between native platform rendering (e.g., window rendering) and OpenGL ES rendering.

[0112] In the embodiment of the present application, the system library also includes SurfaceFlinger. Surface represents the producer in the buffer queue, SurfaceFlinger represents the consumer, and SurfaceFlinger receives data buffers from multiple sources, combines them, and sends them to the hardware display layer.

[0113] The Hardware Abstraction Layer (HAL) provides a standard interface that can be used to control the actions of the hardware. The HAL includes multiple library modules, each of which implements a set of interfaces for a specific type of hardware component, such as a WLAN module, a Bluetooth module, etc.

[0114] In the embodiment of the present application, HAL also includes hwcComposer and OpenGL ES modules. hwcComposer is a HAL module for Layer synthesis and display in Android, providing hardware support for SurfaceFlinger service. OpenGL ES module is used to provide hardware support for OpenGL ES.

[0115] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to perform low-level functions such as threads and low-level memory management. The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, sensor drivers, GPU drivers, etc.

[0116] Figure 3 The flowchart of the electronic device 100 for displaying HDR video is shown as an example. Figure 1A-1F The user interface shown and Figure 2 The software architecture of the electronic device 100 is shown. Below, the embodiment of the present application will specifically introduce the process of the electronic device 100 to display HDR video.

[0117] S101, the electronic device 100 detects a user operation on an editing control in a user interface of an HDR video to be edited.

[0118] Among them, the color encoding format used by the HDR video is the YUV format, the data type of the color value of the color channel is an integer (INT), and the color gamut is BT2020. The HDR video to be edited refers to the HDR video that the user wants the electronic device 100 to edit. The user interface of the HDR video to be edited refers to the user interface for displaying the HDR video to be edited, and the user interface may include a browsing window, an editing control, and a video type icon of the HDR video to be edited. The video type icon can be used to indicate that the video type in the browsing window is an HDR video, that is, the video to be edited is an HDR video. The user interface may also include other operation controls, such as sharing controls, collection controls, deletion controls, and more controls.

[0119] When the electronic device 100 detects a click operation on a certain HDR video in the gallery user interface, the electronic device 100 may display the user interface of the HDR video in response to the click operation, and the electronic device 100 may use the HDR video as the HDR video to be edited. Figure 1B When the icon 122 in the display is clicked, the display Figure 1C The user interface shown is the user interface of the HDR video indicated by icon 122, that is, the user interface of the HDR video to be edited.

[0120] The electronic device 100 detects a user operation on an editing control in a user interface of a to-be-edited HDR video, for example, Figure 1C In other words, the electronic device 100 detects whether a touch instruction for the editing control in the user interface is received when outputting the user interface of the HDR video to be edited.

[0121] S102: The electronic device 100 detects the user operation and initializes the video editing environment in response to the user operation.

[0122] The electronic device 100 detects a user operation on the editing control and can initialize the video editing environment in response to the user operation. Initializing the video editing environment means creating or applying for tools and storage space required for video editing so that the electronic device 100 can perform data processing for video editing.

[0123] Initializing the video editing environment includes: creating an encoder, creating a decoder, OpenGL rendering, applying for user memory to cache video frames and video memory provided by the GPU. The decoder can be used to split the video to be edited into a sequence of video frames; the encoder can be used to combine the edited video frames into a video. OpenGL can be used to adjust the video frame, and / or modify the pixels in the video frame, thereby changing the image content included in the video, that is, rendering the video frame. The above-mentioned adjustment of the video frame includes adjusting to increase or decrease the video frame and modifying the size of the video frame. The initialization of the video editing environment in the embodiment of the present application may not include the creation of an encoder. After the HDR video is displayed in the preview area of ​​the video editing application, an encoder can be created to combine the edited HDR video frames into an HDR video. Before creating the decoder, the embodiment of the present application may include creating a SurfaceView so that the preview area control of the video editing application is implemented through the SurfaceView.

[0124] The above memory includes Surface and BufferQueue. Surface can be used to cache the rendered video frames output by the GPU. BufferQueue can be used to cache the video to be edited input by the video editing application. The decoder can split the video to be edited stored in BufferQueue into a sequence of video frames to be edited.

[0125] Specifically, Figure 4 The flowchart of the electronic device 100 initializing the video editing environment is shown as an example. Figure 4 As shown, APP can be used to represent a video editing application.

[0126] First, in step (1), the electronic device 100 may detect a user operation on an editing control in a user interface of the HDR video to be edited. Figure 1C In the user interface shown, the user operation acting on the edit control 133 can be called a user operation of clicking the edit control.

[0127] In response to the user operation in step (1), the APP initializes the interface and requests the Framework layer to create a SurfaceView. Since the APP learns that the video to be edited is an HDR video, the APP requests the Framework layer to create a SurfaceView. Then, in step (2), the APP sends a request to the Framework layer to create a SurfaceView. Specifically, the APP sends a request to the view system in the Framework layer to create a SurfaceView. Step (3) The Framework layer creates a SurfaceView in response to the request in step (2). When creating a SurfaceView, dataSpace and / or transfer information may be generated in the SurfaceFlinger layer. The dataSpace and / or transfer information may indicate that the attributes of the layer are BT2020, or (BT2020, PQ). SurfaceView can render the video player screen separately, and can also be used to generate a layer with BT2020 attributes, that is, the format of the Surface in SurfaceView is set to BT2020. In an embodiment of the present application, SurfaceView serves as a preview area window for a video editing application.

[0128] After creating SurfaceView, the Framework layer applies for a piece of memory from the memory, such as Surface C. The memory can provide multiple Surfaces. Each Surface carries an identity (ID) indicating the Surface. For any Surface, the ID of the Surface corresponds one-to-one to the address of the Surface. For example, suppose the ID of Surface-01 is 01; the address is 0011-0100. When the ID of a Surface is identified as 01, the electronic device 100 can determine that the above Surface is Surface-01, and can also determine that the address of the above Surface is 0011-0100; conversely, when the address used by a Surface is identified as 0011-0100, the electronic device 100 can determine that the Surface is Surface-01.

[0129] Step (4) The Framework layer sends a request for applying for Surface to the memory. Step (5) The memory responds to the request in step (4) by allocating Surface C, which is a piece of unformatted memory. Then, step (6) the memory returns the ID and / or address of Surface C to the Framework layer. The Framework layer receives the ID and / or address of Surface C, and can bind Surface C to SurfaceView based on the ID and / or address. In other words, the Surface in SurfaceView is Surface C. Then, step (7) the Framework layer returns a successful creation response to the APP in response to the request in step (2). The successful creation response can carry the ID and / or address of Surface C. In other words, the Framework layer applies for a piece of memory from the memory, and executes a successful creation callback when the memory allocates Surface C.

[0130] Step (2) to step (7) can be understood as the process of interface initialization.

[0131] Step (8) APP may send a request to MediaCodec to create a decoder. The request may carry attribute information of the HDR video. The attribute information may be used to indicate the color gamut and color encoding format of the video to be edited. Since the video to be edited in this application is an HDR video, the attribute information carried by the request is BT2020 color gamut and YUV color encoding format.

[0132] In response to the request in step (8), in step (9), MediaCodec can create a decoder. When creating a decoder, MediaCodec does not need to specify the type of video that the decoder supports decoding. The decoder can determine the type of video to be decoded after receiving the video to be decoded input by the APP. In an embodiment of the present application, the video to be edited is an HDR video, so the above decoder can be used to decode HDR video. After MediaCodec creates the decoder, in step (10), the decoder can send a request to the memory for a storage space (BufferQueue). BufferQueue can be used to receive the video to be decoded input by the APP. Step (11) In response to the request in step (10), the memory can allocate a storage BufferQueue for the decoder. Subsequently, in step (12), the memory returns the address of the above BufferQueue to the decoder.

[0133] After receiving the address of the BufferQueue returned by the memory, the decoder can locate the available BufferQueue in the memory according to the address of the BufferQueue. Then, in step (13), the decoder can return a response indicating that the decoder has been successfully created to the APP.

[0134] After receiving a response indicating successful creation of the decoder, in step (14), the APP may send a request to create a surface to the above decoder. Surface is a memory storage space with a specific data structure. Generally, Surface is dedicated to caching decoded video frames. In step (15), the decoder may send a request to the memory to apply for a Surface, such as a request to apply for Surface A, in response to the request in step (14). In step (16), the memory may allocate a storage space, such as Surface A, for use by the decoder in response to the application in step (15). Surface A is used to carry and store BT2020 and YUV video frames, that is, it is used to carry and store HDR video frames.

[0135] After completing the allocation of Surface A to the decoder, the memory in step (17) may return the ID and / or address of Surface A to the decoder. After receiving the above return information, the decoder may confirm that the application for Surface is successful and determine the ID and / or address of the available Surface A. Furthermore, in step (18), the decoder may return the ID and / or address of Surface A to the APP. In this way, the APP can determine that the decoder has completed the process of applying for Surface from the memory, and can determine the ID and / or address of the available Surface A applied by the decoder.

[0136] Then, in step (19), the APP may send an initialization request to OpenGL. The initialization request may carry information of Surface A. The information of Surface A may be used to indicate to OpenGL: the ID and / or address of Surface A in the decoder for receiving the decoded video frame.

[0137] In step (20), based on the information of Surface A carried in the initialization request, OpenGL can determine Surface A used by the decoder. That is, when outputting the video frame after performing the calculation processing of the video frame, OpenGL can determine which cache (Surface) to write the video frame to. The embodiment of the present application takes writing to Surface A as an example.

[0138] In addition, after receiving the initialization request, in step (21), OpenGL will also apply for a video memory from the GPU, which is recorded as video memory a. Video memory a can be used to cache the video frames to be edited. The video memory a can be a texture (texture) or a frame buffer object (Frame BufferObject) in OpenGL.

[0139] In response to step (21), the GPU will allocate a storage space for OpenGL as the video memory (video memory a) requested by OpenGL. Then, in step (22), the GPU can return the address of video memory a to OpenGL. After receiving the address of video memory a, OpenGL can locate video memory a through the above address, and then OpenGL can use video memory a. Then in step (23), OpenGL switches the video memory format to 10 bits so that OpenGL can retain the accuracy of the HDR video image during data transmission, so that HDR video can be displayed in the preview area. Then in step (24), OpenGL can return initialization completion information to the APP. The initialization completion information can indicate to the APP: OpenGL has completed initialization.

[0140] After confirming that OpenGL has completed initialization, in step (25), APP may send an instruction to OpenGL, which is used to instruct OpenGL to apply for a Surface from the memory, such as Surface B. The instruction may carry the ID and / or address of Surface C. In step (26), OpenGL may apply for a surface from the memory in response to the instruction in step (25). In step (27), the memory may allocate a storage space, such as Surface B, in response to the application in step (26), to receive the YUV output of OpenGL, that is, to receive the screen data after OpenGL rendering is completed. In other words, Surface B is used to receive the video frames in the YUV color encoding format output by OpenGL. In step (28), the memory may return the ID and / or address of Surface B to OpenGL. Furthermore, in step (29), OpenGL may bind Surface B to Surface C. Surface B may also be described as EglSurface.

[0141] Figure 4 The process shown in steps (1) to (29) in FIG. 1 shows the process of initializing the video editing environment of the electronic device 100. After completing the initialization of the editing environment, the electronic device 100 can display the HDR video in the preview area of ​​the video editing application.

[0142] S103: The electronic device 100 generates a preview video of the HDR video to be edited, where the preview video is an HDR video.

[0143] After completing the process of initializing the video editing environment, the electronic device 100 can use the above video editing environment to generate a preview video of the HDR video to be edited.

[0144] Figure 5 Steps (1) to (14) show the specific process of the electronic device 100 generating HDR video frames of the HDR video to be edited.

[0145] First, in step (1), the APP can input the HDR video to be edited into the decoder. Specifically, according to the editing environment initialization process, the APP can determine the address of the BufferQueue applied by the decoder for caching the video to be decoded. After determining the above address, the APP can write the HDR video to be edited into the above BufferQueue. At this time, the color encoding format used by the HDR video to be edited is the YUV format, the data type of the color value of the color channel is integer (INT), and the color gamut is BT2020.

[0146] Step (2) When it is detected that a video is written into the BufferQueue, the decoder can decode the video stored in the BufferQueue to obtain a video frame sequence of the video. Therefore, after writing the HDR video to be edited into the BufferQueue, the decoder can output the video frames of the HDR video to be edited, that is, N HDR video frames (HDR video frames to be edited). At this time, the color encoding format, data type and color gamut of the HDR video frames to be edited are still YUV, INT, and BT2020.

[0147] It is understandable that a video may also include audio. Therefore, the decoder also includes an audio decoder. In the video display method provided in the embodiment of the present application, the processing related to audio is a prior art and will not be repeated here.

[0148] In the case of including audio data, after decoding by the decoder, the electronic device 100 can obtain N HDR video frames and audio data of the HDR video to be edited. It can be understood that when the HDR video to be edited does not include audio data, the electronic device 100 does not need to perform audio decoding on the HDR video to be edited, so the data obtained after decoding does not include audio data.

[0149] After decoding is completed in step (3), the decoder can send the decoded HDR video frames to be edited (YUV, INT, BT2020) to the memory in sequence. Correspondingly, in step (4), the memory can successively receive N HDR video frames to be edited (YUV, INT, BT2020), and write the received HDR video frames to be edited to Surface A.

[0150] After decoding is completed, the APP can learn from MediaCodec that the HDR video to be edited has been decoded. (5) The APP sends instruction 1 to OpenGL. Instruction 1 is used to instruct OpenGL to read the HDR video frame to be edited (YUV, INT, BT2020) from Surface A.

[0151] Step (6) OpenGL reads the HDR video frame to be edited (YUV, INT, BT2020) from Surface A, and can change the color encoding format of the HDR video frame to be edited and the data type of the color value in the color channel. In an embodiment of the present application, OpenGL sets the color encoding format of the HDR video frame to be edited to RGB, and the data type of the color value in the color channel to FLOAT, that is, changes the original (YUV, INT) format of the HDR video frame to be edited to the (RGB, FLOAT) format of the HDR video frame to be edited. This is because the color encoding format supported by OpenGL when drawing and / or rendering video frames is RGB, and the data type of the color value in the color channel is floating point.

[0152] Therefore, after the above-mentioned N HDR video frames to be edited are input into OpenGL, their color encoding format is changed to RGB, and the data type of the color value in the color channel is changed to floating point. At this time, the color gamut of the HDR video frame is still BT2020. That is to say, OpenGL converts the HDR video frames to be edited in the format of (YUV, INT, BT2020) into HDR video frames to be edited in the format of (RGB, FLOAT, BT2020). In other words, what OpenGL renders and outputs are HDR video frames to be edited in the format of (RGB, FLOAT, BT2020), and the HDR video frames to be edited can be PQ-encoded video frames. Secondly, OpenGL can store the HDR video frames to be edited in the format of (RGB, FLOAT, BT2020) in video memory a. Video memory a is OpenGL through Figure 4 Steps (21) and (22) shown are obtained from the GPU.

[0153] Step (7) OpenGL sends instruction 2 to the GPU, and instruction 2 is used to instruct the GPU to convert the format of the HDR video frame to be edited, specifically instructing the GPU to convert the HDR video frame to be edited in the (RGB, FLOAT, BT2020) format stored in the video memory a into the HDR video frame to be edited in the (YUV, INT, BT2020) format. Step (8) The GPU responds to instruction 2 and converts the HDR video frame to be edited in the (RGB, FLOAT, BT2020) format stored in the video memory a into the HDR video frame to be edited in the (YUV, INT, BT2020) format. Steps (7) and (8) can be understood as OpenGL instructing the GPU to convert the HDR video frame to be edited in the (RGB, FLOAT, BT2020) format of the OpenGL rendering output stored in the video memory a into the HDR video frame to be edited in the (YUV, INT, BT2020) format.

[0154] Step (9) OpenGL sends instruction 3 to the GPU, and instruction 3 is used to instruct the GPU to output the HDR video frame to be edited to Surface B, specifically instructing the GPU to output the HDR video frame to be edited in the (YUV, INT, BT2020) format to Surface B. Step (10) The GPU responds to instruction 3 and outputs the HDR video frame to be edited in the (YUV, INT, BT2020) format to Surface B. Instruction 3 may carry the ID and / or address of Surface B to instruct the GPU to output the HDR video frame to be edited to Surface B. Instruction 3 may also carry the ID and / or address of Surface C bound to Surface B so that the GPU executes step (11).

[0155] Step (11) The GPU outputs the HDR video frame to be edited in the (YUV, INT, BT2020) format on Surface B to Surface C of SurfaceView. Based on the binding relationship between Surface B and Surface C, the GPU can output the HDR video frame to be edited in the (YUV, INT, BT2020) format on Surface B to Surface C of SurfaceView.

[0156] Step (9) to step (11) can be understood as, based on the binding relationship between Surface B and Surface C in SurfaceView, using Surface B as the rendering output location of OpenGL. When OpenGL calls SwapBuffer, the HDR video frame to be edited after rendering is completed will be swapped from the backBuffer of Surface B to SurfaceView, so that the attributes of Surface C are (BT2020, PQ).

[0157] Then, in step (12), the Framework layer monitors the data change of Surface C of SurfaceView and sets the layer to which SurfaceView belongs to the BT2020 attribute. That is, the Framework layer monitors the data in the backBuffer of Surface B and exchanges it to Surface C, and the data format on Surface C is (YUV, INT, BT2020), so the Framework layer sets the layer to which SurfaceView belongs to the BT2020 attribute. Optionally, the layer to which SurfaceView belongs is set to the (BT2020, PQ) attribute.

[0158] Step (13) The Framework layer sends the layer set to the BT2020 attribute to the hardware abstraction layer.

[0159] Step (14) The hardware abstraction layer synthesizes the HDR video frame. In response to the received layer with the attribute BT2020, the hardware abstraction layer can integrate the layers in the preview area of ​​the video editing application, and place the layer with the attribute BT2020 on the top layer to cover the layers below, and then calculate the brightness displayed on the screen according to the attribute BT2020, and perform tone mapping. The hardware abstraction layer can synthesize the preview video based on multiple HDR video frames.

[0160] Figure 5 The process shown in steps (1) to (14) in FIG. 1 shows the process of generating a preview video by the electronic device 100. After generating the preview video, the electronic device 100 can display the HDR video in the preview area of ​​the video editing application.

[0161] S104, the electronic device 100 displays a preview video of the HDR video to be edited on the user interface of the video editing application, where the preview video is an HDR video.

[0162] The electronic device 100 displays a preview video of the HDR video to be edited in the user interface of the video editing application. Specifically, the electronic device 100 can display the preview video in the preview area in the user interface of the video editing application, for example, Figure 1D , the cover video frame of the preview video is displayed in the window 141. When a user operation on the play button 145 is detected, the window 141 can display the video frame stream of the preview video in sequence.

[0163] In the embodiment of the present application, the video type of the preview video in S104 is an HDR video. That is, when the electronic device 100 detects a user operation on an editing control in a user interface of an HDR video to be edited, it responds to the user operation, switches to a user interface of a video editing application, and displays a preview video in the user interface of the video editing application, where the preview video is an HDR video, so as to enable editing of the HDR video.

[0164] The above S101 to S104 briefly introduce the operation of the electronic device 100 displaying a preview video of the HDR video in the user interface of the video editing application from the perspective of the electronic device 100 and its functional modules. The user interface may also display a video type control, and when a user operation acting on the control is detected, the electronic device 100 may execute S105.

[0165] S105, when the electronic device 100 detects a user operation on a video type control in a user interface of a video editing application, in response to the operation, a video type adjustment window is output in the user interface of the video editing application.

[0166] The video type adjustment window may provide the user with options for export format, video type, resolution, and frame rate. Figure 1E Window 149 is shown.

[0167] The export formats can be divided into video format and GIF animation format.

[0168] Video types can be divided into HDR video type and ordinary video type, ordinary video type is SDR video type. When the video type is HDR video type, it means that the preview video is HDR video; when the video type is ordinary video type, it means that the preview video is SDR video. Optionally, when the video type is HDR video type, the video type adjustment window can also display the text "High Dynamic Range"; when the video type is ordinary video type, the video type adjustment window can also display the text "Standard Dynamic Range" so that users can select the video type according to their needs.

[0169] The resolution can be divided into 1080P and 2K / 4K. Optionally, when the resolution is 1080P, the video type adjustment window can also display the text "Poor picture restoration effect"; when the resolution is 2K / 4K, the video type adjustment window can also display the text "Extreme picture restoration, large space occupancy", so that users can select the resolution according to their needs.

[0170] The frame rate can be divided into 24, 25, 30, 50 and 60. FPS is a definition in the field of images, which refers to the number of frames transmitted per second. Generally speaking, it refers to the number of frames in an animation or video. The higher the frame rate, the smoother the playback. Optionally, the video type adjustment window can also display the text "The higher the frame rate, the smoother the playback" so that users can select the frame rate according to their needs.

[0171] The video type adjustment window can also display the size of the preview video, which is related to the selected export format, video type, resolution and frame rate.

[0172] S106: When the electronic device 100 detects a switching operation of the video type in the video type adjustment window, in response to the operation, the preview video is converted from the HDR video to the SDR video.

[0173] The switching operation of the video type in the video type adjustment window is used to switch the video type from HDR video to SDR video. When the electronic device 100 detects the switching operation of the video type in the video type adjustment window, in response to the operation, the preview video is converted from HDR video to SDR video. For example, see Figure 1E In the user interface shown, when the electronic device 100 detects a click operation on the ordinary video in the window 149, in response to the click operation, the video type of the preview video is adjusted to SDR video, and the preview video is converted from HDR video to SDR video.

[0174] The electronic device 100 converts the preview video from the HDR video to the SDR video to display the SDR video of the HDR video to be edited in the preview area of ​​the video editing application.

[0175] For an HDR video frame, the electronic device 100 converts the HDR video frame into an SDR video frame. Figure 6 Steps (1) to (4) show the specific process of the electronic device 100 converting the HDR video frame into the SDR video frame.

[0176] Step (1) converts the HDR nonlinear electrical signal of the HDR video frame into an HDR linear optical signal through an electro-optical transfer function (EOTF).

[0177] In the embodiment of the present application, EOTF can be expressed as F D =EOTF[E',L]orF D =EOTF[E']+L,F D Represents nonlinear color value (i.e. HDR linear light signal), E' represents linear color value (i.e. HDR nonlinear electrical signal), and L represents brightness parameter. The above formula can be understood as taking brightness parameter as a calculation parameter in the process of converting nonlinear electrical signal into linear light signal. Brightness parameter represents the picture brightness of the current HDR video frame. Different HDR video frames in the same HDR video may have different brightness, for example, dark HDR video frame has 300nit and bright HDR video frame has 4000nit, so different HDR video frames in the same HDR video can have different brightness parameters.

[0178] In one implementation, for an HDR video frame, its brightness parameter is the maximum brightness of the HDR video frame. In order to convert the HDR video frame into an SDR video frame with obvious contrast (for example, brighter parts and darker parts), the maximum brightness is used as the brightness parameter. Exemplarily, the electronic device 100 divides the HDR video frame into 50*50 grouping areas horizontally and vertically, and calculates the maximum brightness of each grouping area to obtain 2500 brightness maximum values. The first 20 brightness maximum values ​​are selected from the 2500 brightness maximum values ​​in descending order, and the average value of these 20 brightness maximum values ​​is calculated, which is the maximum brightness of the HDR video frame. Step (2) performs color space conversion (Color Space Converting) on ​​the HDR linear light signal, that is, converting the color gamut of the HDR linear light signal from BT2020 to BT709.

[0179] Step (3) performs tone mapping on the HDR linear light signal after the color space conversion to map it into an SDR linear light signal.

[0180] Step (4) converts the SDR linear optical signal into an SDR nonlinear electrical signal through an optical-electro transfer function (OETF), thereby converting the HDR video frame into an SDR video frame.

[0181] In one implementation, the electronic device 100 may use the FFmpeg command line to implement Figure 6 Steps (1) to (4) in .

[0182] In another implementation, OpenGL controls the GPU to execute Figure 6 For example, for step (1), OpenGL sends an instruction to the GPU, which instructs the GPU to convert the HDR nonlinear electrical signal into an HDR linear optical signal through the EOTF.

[0183] In another implementation, the electronic device 100 decodes the HDR video into a Texture texture, OpenGL reads the YUV 10-bit data from the Texture texture, and then executes Figure 6 Steps (1) to (4) in .

[0184] S107, the electronic device 100 displays the SDR video of the HDR video to be edited on the user interface of the video editing application.

[0185] After the electronic device 100 converts the preview video from the HDR video to the SDR video, the electronic device 100 may display the SDR video of the HDR video to be edited in the user interface of the video editing application. Specifically, the electronic device 100 may display the SDR video in the preview area in the user interface of the video editing application, for example, Figure 1F , the cover video frame of the SDR video is displayed in window 141. When a user operation on the play button 145 is detected, window 141 can display the video frame stream of the SDR video in sequence.

[0186] The electronic device 100 also displays the video type as SDR video in the user interface of the video editing application. Specifically, the electronic device 100 may indicate that the video type is SDR video in the video type control in the user interface of the video editing application, for example, Figure 1F , “SDR” displayed in the video type control 148 indicates that the video type is SDR video.

[0187] The above S105 to S107 briefly introduce the operation of the electronic device 100 converting the preview video from the HDR video to the SDR video and displaying the SDR video in the preview area from the perspective of the electronic device 100 and its functional modules.

[0188] For displaying SDR video in the preview area, if the electronic device 100 detects a user operation on a video type control in the user interface of a video editing application, in response to the operation, a video type adjustment window is output in the user interface of the video editing application, and the video type in the video type adjustment window is a normal video type. When the electronic device 100 detects a switching operation of the video type in the video type adjustment window (i.e., the switching operation is used to indicate switching the normal video type to the HDR video type), in response to the operation, the video editing environment is initialized, a preview video of the video type being HDR video is generated, and the HDR video is displayed in the preview area of ​​the user interface of the video editing application. Among them, initializing the video editing environment can refer to Figure 3 For a detailed description of S102, please refer to the following for generating a preview video of HDR video type. Figure 3 The specific description of S103 is omitted here.

[0189] Figure 7 The schematic diagram of the hardware structure of the electronic device 100 is exemplarily shown.

[0190] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0191] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0192] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0193] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0194] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0195] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0196] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0197] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0198] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0199] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0200] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0201] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0202] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0203] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0204] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0205] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0206] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0207] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0208] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0209] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0210] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0211] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0212] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0213] In the embodiment of the present application, the electronic device 100 displays Figure 1A-1F The user interface shown can be completed through GPU, encoder, decoder, OpenGL, and display screen 194.

[0214] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0215] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0216] In an embodiment of the present application, the HDR video to be edited can be obtained by the electronic device 100 from other electronic devices through a wireless communication function, or it can be captured by the electronic device 100 through an ISP, a camera 193, a video codec, a GPU, and a display screen 194.

[0217] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0218] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0219] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0220] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0221] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0222] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0223] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc. Non-volatile memory may include disk storage devices and flash memory.

[0224] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0225] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0226] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0227] In an embodiment of the present application, the internal memory 121 can support the electronic device 100 to apply for Surface, BufferQueue, etc. from the memory.

[0228] The external memory interface 120 can be used to connect an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video and other files are stored in an external non-volatile memory. In an embodiment of the present application, the electronic device 100 can collect sound through the microphone 170C when shooting HDR video. During the video playback process, the speaker 170A or the speaker connected to the headphone interface 170D can support the audio in the video.

[0229] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0230] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0231] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0232] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0233] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0234] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0235] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0236] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0237] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0238] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0239] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0240] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0241] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0242] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0243] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0244] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0245] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0246] In the embodiment of the present application, the electronic device 100 can detect whether there is a user operation acting on the display screen 194 of the electronic device 100 through the touch sensor 180K. After the touch sensor 180K detects the above user operation, the electronic device 100 can perform the image processing indicated by the above user operation, such as displaying HDR video in the preview area of ​​the user interface of the video editing application.

[0247] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0248] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0249] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0250] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0251] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0252] In implementing the embodiment of the present application, the electronic device can use SurfaceView as a display control of the preview area of ​​the video editing application, set the property of the layer to which SurfaceView belongs to BT2020, and bind Surface C in SurfaceView to Surface B for receiving OpenGL rendering output, so that the video frames in Surface B can be transmitted to Surface C, so that HDR video can be displayed in the preview area of ​​the user interface of the video editing application. Furthermore, the electronic device can also switch the HDR video displayed in the preview area to SDR video according to the video type switching operation.

[0253] In the embodiments of the present application:

[0254] 1. In a user interface for displaying a video, an operation in which a user clicks an editing control for the user interface may be referred to as a first operation, for example Figure 1CThe operation of clicking the control 133 in the middle. The user interface showing the video may be referred to as the first user interface, for example Figure 1C The window for displaying a video in the user interface for displaying a video may be referred to as a first window, for example Figure 1C When the first user operation is detected, the video currently displayed by the electronic device, that is, the video selected by the user to be edited, can be called the first video, for example Figure 1C The video M is displayed in the middle window 131. A series of video frames obtained by decoding the first video by the decoder can be called first video frames. There are N first video frames. The specific value of N is determined by the duration of the first video.

[0255] 2. The Surface in SurfaceView can be called the first Surface, for example Figure 4 The identification information of the first Surface is the ID and / or address of the first Surface. The Surface requested by the video editing application through the instruction calling OpenGL to the memory can be called the second Surface, for example Figure 4 In steps (25)-(28), the video editing application calls OpenGL through instructions to apply for Surface B from the memory. The Surface used to store the video frames output by the decoder can be called the third Surface, for example Figure 4 Surface A in.

[0256] 3. The video frame obtained by OpenGL converting the format of the video frame can be called the second video frame, for example Figure 5 The HDR video frame to be edited in the (RGB, FLOAT type, BT2020) format in step (6) is obtained by converting the format of the video frame by the GPU and is referred to as the third video frame. Figure 5 The HDR video frame to be edited in the (YUV, INT type, BT2020) format in step (8).

[0257] 4. The user interface of the video editing application displaying the HDR video may be referred to as the second user interface, e.g. Figure 1D The video preview window in the user interface can be called a second display window, for example Figure 1D Window 141 is shown.

[0258] 5. The operation of the user clicking on the video type control can be called the second operation, for example Figure 1D In response to the operation of clicking the control 148, the output video type adjustment window can refer to Figure 1F Window 149 is shown.

[0259] 6. When the video type adjustment window is output, if the HDR video type is selected in the window (i.e., the video currently displayed in the preview area is an HDR video), the operation of the user clicking on the normal video type can be called a switching operation, which is used to instruct the electronic device to switch the video type from the HDR video type to the normal video type. If the normal video type is selected in the window (i.e., the video currently displayed in the preview area is an SDR video), the operation of the user clicking on the HDR video type can be called a switching operation, which is used to instruct the electronic device to switch the video type from the normal video type to the HDR video type.

[0260] 7. Figure 4 In the embodiment of the present invention, the request in step (2) may be referred to as the first request; the request in step (4) may be referred to as the second request; step (6) may be described as the memory sending a second response to the application framework layer, the second response including the identification information of the first Surface; step (7) may be described as the application framework layer sending a first response to the video editing application, the first response including the identification information of the first Surface; the instruction in step (25) may be referred to as the first instruction; the request in step (26) may be referred to as the third request; step (28) may be described as the memory sending a third response to OpenGL, the third response including the identification information of the second Surface; the request in step (14) may be referred to as the fourth request; the request in step (15) may be referred to as the fifth request; step (17) may be described as the memory sending a fifth response to MediaCodec, the fifth response including the identification information of the third Surface; step (18) may be described as MediaCodec sending a fifth response to the video editing application, the fifth response including the identification information of the third Surface.

[0261] 8. Figure 5 In the embodiment of the present invention, instruction 1 in step (5) may be referred to as the second instruction; instruction 2 in step (7) may be referred to as the third instruction; instruction 3 in step (9) may be referred to as the fourth instruction.

[0262] The term "user interface (UI)" in the specification, claims and drawings of this application refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、

[0263] <videoview>The controls contained in the interface are specified by nodes such as <head> and . A node corresponds to a control or attribute in the interface. After parsing and rendering, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (GTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, GTML is defined by . 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0264] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0265] As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items. As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrase "when determining..." or "if (stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "when (stated condition or event) is detected" or "in response to detecting (stated condition or event)", depending on the context.

[0266] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0267] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A video display method, applied to an electronic device, characterized in that: The method comprises: Displaying a first user interface, wherein the first user interface includes a first display window and an editing control for displaying a first video; the video type of the first video is a high dynamic range HDR video; A first operation acting on the edit control is detected, and in response to the first operation: Create a SurfaceView and apply for the first Surface for the SurfaceView; Create a decoder; Apply for a second Surface, and bind the first Surface to the second Surface; Decoding the first video into N first video frames by the decoder; Performing format conversion on the N first video frames through OpenGL to obtain N second video frames output by the OpenGL; Calling a graphics processor GPU of the electronic device to perform format conversion on the N second video frames to obtain N third video frames, and outputting the N third video frames to the second Surface; Based on the binding between the first Surface and the second Surface, output the N third video frames on the second Surface to the first Surface; The properties of the layer to which the SurfaceView belongs are set as BT2020 properties, the layers of the preview area of ​​the video editing application are integrated together, and the layer to which the SurfaceView belongs is placed on the top layer; the brightness displayed on the screen is calculated according to the properties of the layer to which the SurfaceView belongs, and tone mapping is performed based on the brightness; based on the N third video frames on the first Surface and the adjusted layer to which the SurfaceView belongs, a preview video of the first video is synthesized, and the video type of the preview video is HDR video; A second user interface is displayed, wherein the second user interface includes a second display window, and the second display window is used to display the preview video, and the display control of the second display window is the SurfaceView.

2. The method according to claim 1, characterized in that: The format of the first video frame includes: the color encoding format is YUV format, the data type of the color value is integer, and the color gamut is BT2020; The format of the second video frame includes: the color encoding format is RGB format, the data type of the color value is floating point type, and the color gamut is BT2020; The format of the third video frame includes: the color encoding format is YUV format, the data type of the color value is integer, and the color gamut is BT2020.

3. The method according to claim 1, characterized in that The second user interface also includes a video type control, and the video type control is used to indicate that the video type of the preview video is HDR video.

4. The method according to claim 3, characterized in that The method further comprises: detecting a second operation acting on the video type control, and outputting a video type adjustment window on the second user interface in response to the second operation; Among them, the video type adjustment window includes at least one of the following options: export format, video type, resolution, frame rate; the export format includes video format options and GIF animation format options, the video type includes HDR video type options and ordinary video type options, the resolution includes 1080P options and 2K / 4K options, and the frame rate includes 24 options, 25 options, 30 options, 50 options and 60 options.

5. The method according to claim 4, characterized in that The method further comprises: A switching operation of the video type in the video type adjustment window is detected, and in response to the switching operation: Convert the preview video from an HDR video to an SDR video; The SDR video of the first video is displayed in the second display window.

6. The method according to claim 5, characterized in that The step of converting the preview video from an HDR video to an SDR video comprises: Converting the HDR nonlinear electrical signal of the third video frame n into an HDR linear optical signal through an electro-optical transfer function EOTF; Performing color space conversion on the HDR linear light signal; Performing tone mapping on the HDR linear light signal after color space conversion to obtain an SDR linear light signal; Converting the SDR linear optical signal into an SDR nonlinear electrical signal through a photoelectric conversion function to obtain an SDR nonlinear electrical signal of the third video frame n; The third video frame n is any third video frame among the N third video frames.

7. The method according to claim 6, characterized in that The EOTF is related to a brightness parameter of the third video frame n, and the brightness parameter of the third video frame n is a maximum brightness of the third video frame n.

8. The method according to claim 7, characterized in that The method further comprises: The third video frame n is divided into L grouping areas, and the maximum brightness value of each grouping area in the L grouping areas is calculated to obtain L maximum brightness values; M is a positive integer; Selecting the first K maximum brightness values ​​from the L maximum brightness values ​​in order from largest to smallest, where K is an integer greater than 1 and less than M; An average value of the K maximum brightness values ​​is calculated, where the average value is the maximum brightness of the third video frame n.

9. The method according to any one of claims 1 to 8, characterized in that: The electronic device includes a video editing application, an application architecture layer, and a memory; The creating of SurfaceView and applying for the first Surface for the SurfaceView specifically includes: The video editing application sends a first request to the application architecture layer, where the first request is used to request the creation of the SurfaceView; The application architecture layer creates the SurfaceView in response to the first request, and sends a second request to the memory, where the second request is used to request the memory to allocate the first Surface; The memory allocates the first Surface in response to the second request, and sends a second response to the application architecture layer, where the second response includes identification information of the first Surface; The application architecture layer sends a first response to the video editing application, where the first response includes identification information of the first Surface.

10. The method according to claim 9, characterized in that The electronic device also includes OpenGL; The applying for the second Surface and binding the first Surface to the second Surface specifically includes: The video editing application sends a first instruction to the OpenGL, where the first instruction is used to instruct the OpenGL to apply for the second Surface from the memory; In response to the first instruction, the OpenGL sends a third request to the memory, where the third request is used to request the memory to allocate the second Surface; In response to the third request, the memory allocates the second Surface and sends a third response to the OpenGL, where the third response includes identification information of the second Surface; The OpenGL binds the first Surface and the second Surface.

11. The method according to claim 9, characterized in that The electronic device further includes a MediaCodec; the MediaCodec is used to create the decoder; The method further comprises: The video editing application sends a fourth request to the MediaCodec, where the fourth request is used to request a third Surface; the third Surface is used to store the N first video frames output by the decoder; In response to the fourth request, the MediaCodec sends a fifth request to the memory, where the fifth request is used to request the memory to allocate the third Surface; The memory allocates the third Surface in response to the fifth request, and sends a fifth response to the MediaCodec, where the fifth response includes identification information of the third Surface; The MediaCodec sends a fourth response to the video editing application, where the fourth response includes identification information of the third Surface.

12. The method according to claim 11, characterized in that The method further comprises: The decoder created by the MediaCodec sends the N first video frames to the memory; The memory writes the N first video frames into the third Surface; The converting the formats of the N first video frames by using OpenGL to obtain N second video frames output by OpenGL includes: The video editing application sends a second instruction to the OpenGL, where the second instruction is used to instruct the OpenGL to read the N first video frames from the third Surface; In response to the second instruction, the OpenGL reads the N first video frames from the third Surface, and performs format conversion on the N first video frames to obtain N second video frames output by the OpenGL; The OpenGL stores the N second video frames in the first video memory.

13. The method according to claim 12, characterized in that The calling of the GPU of the electronic device to convert the formats of the N second video frames to obtain N third video frames, and outputting the N third video frames to the second Surface specifically includes: The OpenGL sends a third instruction to the GPU, where the third instruction is used to instruct the GPU to perform format conversion on the N second video frames; In response to the third instruction, the GPU performs format conversion on the N second video frames stored in the first video memory to obtain N third video frames; The OpenGL sends a fourth instruction to the GPU, where the fourth instruction is used to instruct the GPU to output the N third video frames to the second Surface; the fourth instruction includes identification information of the second Surface; In response to the fourth instruction, the GPU outputs the N third video frames to the second Surface based on the identification information of the second Surface.

14. The method according to claim 13, characterized in that The third instruction or the fourth instruction includes identification information of the first Surface bound to the second Surface; The step of outputting the N third video frames on the second Surface to the first Surface based on the binding between the first Surface and the second Surface specifically includes: The GPU determines, based on the third instruction or the fourth instruction, that the first Surface is bound to the second Surface, and based on the binding of the first Surface to the second Surface, outputs the N third video frames on the second Surface to the first Surface.

15. An electronic device, characterized in that: The method comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the method according to any one of claims 1 to 14 is executed.

16. A computer program product comprising instructions, characterized in that When the computer program product runs on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 14.

17. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method according to any one of claims 1 to 14 is executed.