Image editing method and device based on Hongmeng operating system

Abstract

The present invention proposes a picture editing method and device based on the Hongmeng operating system, which relate to the field of picture processing technology. The method runs on an electronic device equipped with the Hongmeng operating system, and imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; based on the GPU acceleration technology of the Hongmeng system, real-time preview and text editing of the pictures are performed to obtain real-time preview pictures and text-edited pictures; based on the Hongmeng AI engine, the main body of the real-time preview picture is identified and cropped to obtain a cropped picture; the color tone is adjusted according to the color adjustment algorithm of the Hongmeng system to obtain a color-edited picture; graffiti is performed according to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture; according to the I / O operation of the Hongmeng system, the pictures in the editing process are saved in real time to the distributed storage of the Hongmeng system. Thus, based on the Hongmeng operating system, fast editing, cropping, color adjustment and graffiti functions of the pictures are realized, thereby improving the user's operating efficiency and experience.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for image editing based on the Hongmeng operating system. Background Art

[0002] With the increasing popularity of smartphones, users are increasingly demanding image editing capabilities on their mobile devices. Existing image editing apps often have complex functions and operations, lack cross-device collaborative editing capabilities, and fail to meet users' needs for fast editing or provide a smooth user experience. Therefore, a more reliable method for fast image editing is urgently needed. Summary of the Invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, the first purpose of the present invention is to propose a picture editing method based on the Harmony operating system, which realizes the fast editing, cropping, tone adjustment and graffiti functions of pictures based on the Harmony operating system, thereby improving the user's operating efficiency and experience.

[0005] The second purpose of the present invention is to provide a picture editing device based on the Hongmeng operating system.

[0006] A third object of the present invention is to provide an electronic device.

[0007] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions.

[0008] A fifth object of the present invention is to provide a computer program product.

[0009] To achieve the above objectives, the first embodiment of the present invention provides a picture editing method based on the Hongmeng operating system. The picture editing method based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the method includes:

[0010] Import images stored locally or in the cloud using the distributed file subsystem of Hongmeng OS;

[0011] Based on the GPU acceleration technology of Hongmeng system, real-time preview and text editing of pictures are performed to obtain real-time preview pictures and text editing pictures;

[0012] Based on the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image. The cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping.

[0013] Adjust the hue of the real-time preview image according to the hue adjustment algorithm of the Hongmeng system to obtain a hue-edited image, wherein the hue adjustment algorithm includes histogram analysis, image hue contrast adjustment, hue enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0014] Graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing;

[0015] According to the I / O operation of the Harmony system, the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image are saved in real time to the distributed storage of the Harmony system.

[0016] To achieve the above objectives, a second embodiment of the present invention provides a picture editing device based on the Hongmeng operating system. The picture editing device based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the device includes:

[0017] The image import module is used to import images stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system;

[0018] The editing module is used for real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text editing images;

[0019] A cropping module, configured to execute a cropping algorithm based on the HarmonyOS AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image. The cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping.

[0020] A tone adjustment module, configured to adjust the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone-edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0021] A graffiti module, configured to graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing;

[0022] To achieve the above-mentioned purpose, the third aspect embodiment of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0023] In order to achieve the above-mentioned objectives, an embodiment of the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0024] In order to achieve the above-mentioned objectives, a fifth embodiment of the present invention provides a computer program product, which implements the method described in the first aspect when executed by a processor.

[0025] The embodiments of the present invention provide a picture editing method, device, electronic device and storage medium based on the Hongmeng operating system. The method runs on an electronic device equipped with the Hongmeng operating system, and imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; performs real-time preview and text editing on the pictures based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview pictures and text-edited pictures; identifies the subject of the real-time preview picture and crops it according to the Hongmeng AI engine to obtain a cropped picture; adjusts the color tone according to the color adjustment algorithm of the Hongmeng system to obtain a color-edited picture; graffiti is performed according to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture; and saves the pictures of the editing process in real time to the distributed storage of the Hongmeng system according to the I / O operation of the Hongmeng system. Thus, based on the Hongmeng operating system, the functions of fast editing, cropping, color adjustment and graffiti of pictures are realized, thereby improving the user's operating efficiency and experience.

[0026] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A schematic diagram of a flow chart of a picture editing method based on the Hongmeng operating system provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of a flow chart of another image editing method based on the Hongmeng operating system provided by an embodiment of the present invention;

[0030] Figure 3A structural diagram of a picture editing device based on the Hongmeng operating system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0032] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of the present invention comply with the relevant provisions of relevant laws and regulations.

[0033] The following describes the picture editing method and device based on the Hongmeng operating system according to an embodiment of the present invention with reference to the accompanying drawings.

[0034] Figure 1 A flowchart of a picture editing method based on the Hongmeng operating system provided in an embodiment of the present invention.

[0035] like Figure 1 As shown, the image editing method based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the method includes the following steps:

[0036] Step 101: Import pictures stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system.

[0037] In some embodiments, images can be imported from local gallery, camera and cloud storage, and asynchronous loading can be used to ensure user interface responsiveness. In addition, for large-size images, thumbnails are loaded first, and then the full resolution is loaded step by step.

[0038] Step 102: Based on the GPU acceleration technology of the Hongmeng system, real-time preview and text editing of the image are performed to obtain real-time preview images and text editing images.

[0039] In some embodiments, the real-time preview image is displayed on the user interface of the HarmonyOS system. The user interface can be developed using the UI framework (HarmonyOS UI) to achieve responsive design, adapt to electronic device screens of different sizes, and support gesture operations and shortcut keys.

[0040] Optionally, graphics processing unit (GPU) acceleration technology can be developed based on the HarmonyOS Graphics Engine, and then GPU acceleration technology can be used to achieve real-time preview and text editing, support multi-threaded processing, and improve processing efficiency.

[0041] In step 103, according to the HarmonyOS AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image. The cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping.

[0042] Optionally, the HarmonyOS AI Engine can use an integrated pre-trained deep learning model for image subject recognition and cropping, and supports online updating of deep learning models to continuously improve recognition accuracy.

[0043] In some embodiments, the HarmonyOS AI engine executes a cropping algorithm to identify the subject of a live preview image and crops it to obtain a cropped image. In one implementation, the HarmonyOS AI engine uses bilinear interpolation to resize the input image into a first image of a fixed size; performs color space conversion on the RGB image of the first image to Lab color space to obtain a second image in Lab color space; performs subject object detection on the second image using a YOLO v5 model to detect a subject object bounding box and confidence score; performs semantic segmentation on the second image using a U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; and, based on the subject object bounding box and confidence score of the second image and the pixel-level segmentation mask, crops the second image using preset cropping rules to obtain a third image, wherein the cropping rules include maintaining the integrity of the subject object, adhering to the rule of thirds composition principle, and avoiding cropping key facial areas; and performs multi-size cropping on the third image using preset cropping ratios to obtain the cropped image. This enables intelligent subject recognition and automatic cropping, providing users with a better image editing experience.

[0044] Furthermore, when adjusting the image size, the fixed size to which the image is adjusted may be 224x224 pixels, but is not limited thereto.

[0045] Furthermore, when performing multi-size cropping, the preset size cropping ratio can be commonly used ratios such as 1:1, 4:3, 16:9, etc. In addition, the aesthetic score of each ratio cropped image can be calculated to select the cropped image with a higher aesthetic score, wherein the aesthetic score can be calculated using an aesthetic evaluation model based on deep learning.

[0046] In addition, according to the HarmonyOS AI engine, the cropping algorithm is executed asynchronously in the background thread, and a cache mechanism is used to store intermediate results to avoid repeated calculations. It also supports progressive processing, returning rough results first and then gradually optimizing.

[0047] Step 104: Adjust the hue of the real-time preview image according to the hue adjustment algorithm of the HarmonyOS system to obtain a hue-edited image. The hue adjustment algorithm includes histogram analysis, image hue contrast adjustment, hue enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing.

[0048] In some embodiments, according to the hue adjustment algorithm of the Hongmeng system, the hue of the real-time preview image is adjusted to obtain an implementation method of the hue edited image, which can be to calculate the histogram of the image of the real-time preview image in the RGB and Lab color spaces to perform histogram analysis and determine the brightness contrast and color saturation of the image; the adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content, adjusts the grid size according to the complexity of the image, and optimizes the brightness contrast based on the contrast limit threshold and the grid size to obtain the adjusted first image hue contrast; performs hue enhancement by performing nonlinear mapping on the two color channels in the Lab color space to obtain a first color saturation after hue enhancement, and uses the S curve to adjust the lightness channel in the Lab color space to obtain a second image hue contrast after hue enhancement of the first image hue contrast; uses the grayscale world hypothesis algorithm to estimate the light source color temperature corresponding to the real-time preview image, adjusts the RGB channel gain in the Lab color space according to the light source color temperature to complete white balance correction, and obtains the second image hue contrast and the first color saturation after RGB channel gain. A third image hue contrast and a second color saturation are obtained by using a guided filter (Guided Filter). A filter (likely a filter) is used to separate the base layer and detail layer of the first preview image, which are composed of the hue contrast of the third image and the color saturation of the second image. The detail layer is enhanced using an adaptive contrast enhancement algorithm. The enhanced detail layer is then recombined with the base layer to obtain a second preview image. A local tone mapping algorithm is then used to simulate the HDR effect on the second preview image to obtain a tone-edited image. This allows for precise color adjustment.

[0049] Furthermore, when adjusting the image hue contrast, the adaptive histogram equalization method may be a CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm.

[0050] Furthermore, when performing HDR effect simulation processing on an image, highlight and shadow details can be maintained while enhancing the overall dynamic range to achieve HDR effect simulation of the second preview image and obtain a reliable tone-edited image.

[0051] Step 105: Preview the picture in real time according to a variety of graffiti methods corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture, wherein the graffiti stroke method in the graffiti method includes picture data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing.

[0052] In some embodiments, one implementation method of graffiti real-time previewing an image based on multiple graffiti modes corresponding to the multi-modal input technology of the Hongmeng system to obtain a graffiti image can be to collect a sequence of input points of the user's touch screen or stylus and record the collected data of each input point, the collected data including coordinates, pressure and timestamp; use a Kalman filter to smooth the collected data of each input point sequence to remove noise input points generated by hand shaking to obtain a point sequence of noise-free input points; use a cubic Bezier curve to fit the smoothed point sequence to adjust the control points of the user's touch screen or stylus to adapt to different stroke shapes; calculate the stroke width of the user's touch screen or stylus based on the pressure value and speed of the input point determined by the coordinates, pressure and timestamp; use supersampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing to obtain the texture and anti-aliasing of the user's touch screen or stylus; use a vertex shader and a fragment shader to render the stroke shape, stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image. Therefore, through functions such as multi-mode input, the user operation process is simplified and the user experience is enhanced.

[0053] Furthermore, when calculating the stroke width, an exponential function can be used to map the pressure value to the width range, and the speed can be taken into account. The faster the speed, the thinner the stroke width.

[0054] Furthermore, when performing real-time rendering processing, a subset of the Open Graphics Library (OpenGL) 3D graphics API (OpenGL ES) may be used for GPU-accelerated real-time rendering.

[0055] In addition, you can use view hierarchy optimization to reduce overdrawing and optimize real-time rendering processing.

[0056] Step 106: According to the I / O operation of the Harmony system, the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image are saved in real time to the distributed storage of the Harmony system.

[0057] In some embodiments, the distributed storage can be a HarmonyOS distributed file system, which uses the HarmonyOS distributed file system to achieve seamless switching between local and cloud storage, and support incremental saving and version control.

[0058] Optionally, all image text editing, cropping, tone adjustment, and graffiti functions are edited based on a copy of the original image, and use command mode to record each editing step, support undo and redo, and use low-resolution preview during the editing process to improve response speed.

[0059] Furthermore, during the real-time saving process, an incremental saving strategy can be used to save only the changed parts, and Copy-on-Write technology can be adopted to optimize storage efficiency. The incremental updates can then be regularly merged into complete files to ensure data consistency.

[0060] In some embodiments, when managing distributed storage, memory pool technology can be used to reduce memory fragmentation, a block loading strategy can be adopted for large image processing, and weak references can be used to cache non-critical data to respond to memory pressure in a timely manner.

[0061] The embodiment of the present invention is a picture editing method based on the Hongmeng operating system. The method runs on an electronic device equipped with the Hongmeng operating system, and imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; performs real-time preview and text editing on the pictures based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview pictures and text-edited pictures; identifies the subject of the real-time preview picture and crops it according to the Hongmeng AI engine to obtain a cropped picture; adjusts the color tone according to the color adjustment algorithm of the Hongmeng system to obtain a color-edited picture; graffiti is performed according to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture; and saves the pictures of the editing process in real time to the distributed storage of the Hongmeng system according to the I / O operation of the Hongmeng system. Thus, based on the Hongmeng operating system, the functions of fast editing, cropping, color adjustment and graffiti of pictures are realized, thereby improving the user's operating efficiency and experience.

[0062] In order to clearly illustrate the previous embodiment, this embodiment also provides another image editing method based on the Hongmeng operating system, such as Figure 2 Specifically, you can select an image from images imported into local or cloud storage, and then edit the selected image by selecting an operation item. The selected operation items include cropping, text editing, color adjustment, and graffiti. According to the I / O operations of the Harmony system, the text-edited image, cropped image, color-edited image, and graffiti image after cropping, text editing, color adjustment, and graffiti are saved in real time, output, and saved to the distributed storage of the Harmony system. In addition, you can re-execute the selected operation item to process the text-edited image, cropped image, color-edited image, and graffiti image saved in real time. In this way, the characteristics of the Harmony system are fully utilized to achieve efficient and intelligent image editing functions, providing users with a better image editing experience.

[0063] In addition, when the real-time preview image is a sensitive image, the sensitive image is stored in a preset privacy area, and / or the image metadata of the sensitive image is cleaned to protect the privacy of the sensitive image and realize privacy management of sensitive images.

[0064] In some embodiments, the distributed operating subsystem of Hongmeng OS enables collaborative editing of text, cropping, color adjustment, and graffiti on the image across different devices. This enables a seamless editing experience across devices based on the distributed capabilities of Hongmeng OS.

[0065] Specifically, the distributed operation subsystem can be the HarmonyOS distributed soft bus, which can realize real-time data synchronization and collaborative editing between electronic devices, as well as seamless operation experience across devices.

[0066] Furthermore, when performing cross-device collaborative editing, a differential synchronization algorithm can be used to transmit only changed data, while prioritizing P2P direct connection to reduce network latency. In addition, when conflicts arise in cross-device collaborative editing, timestamps are prioritized, supplemented by user confirmation.

[0067] In some embodiments, the processing performance of the image editing method based on the HarmonyOS operating system can be dynamically adjusted according to the battery power of the electronic device. Specifically, task scheduling is used to schedule intensive calculations to be performed when the electronic device is charging; network requests are optimized to reduce unnecessary data transmission.

[0068] In some embodiments, delayed loading and on-demand loading strategies can also be used; commonly used resources can be preloaded; and compile-time annotation processing can be used to reduce runtime reflection overhead and achieve startup optimization of the image editing method based on the Hongmeng operating system.

[0069] In some embodiments, the AES-256 encryption algorithm can also be used to protect locally stored images and editing data during editing, cropping, tone adjustment, and doodling, while the TLS1.3 protocol is used for network transmission to ensure the transmission security of locally stored images and editing data.

[0070] In some embodiments, the principle of least privilege can also be adopted to request only necessary system permissions, provide fine-grained permission control, allow users to customize the permissions of each function, and implement detailed permission management of image editing methods based on the HarmonyOS operating system.

[0071] In some embodiments, digital signature technology can also be used to verify the integrity of the image editing method based on the HarmonyOS operating system, prevent tampering, implement runtime integrity checks, and detect potential injection attacks.

[0072] In some embodiments, a plug-in API can also be designed so that the image editing method based on the HarmonyOS operating system supports third-party developers to extend editing functions, and use a sandbox mechanism to isolate the plug-in running environment to ensure system security.

[0073] In some embodiments, a cloud API can also be designed so that the image editing method based on the HarmonyOS operating system supports future expansion of cloud AI processing capabilities, and a data synchronization interface is reserved to prepare for cross-platform (device) editing.

[0074] In some embodiments, the image editing method based on the Harmony operating system can also use resource files to separate UI text, facilitate localization, and support dynamic language package downloads without updating the entire execution application of the image editing method based on the Harmony operating system.

[0075] In order to implement the above embodiments, the present invention also proposes a picture editing device based on the Hongmeng operating system.

[0076] Figure 3 A structural diagram of a picture editing device based on the Hongmeng operating system provided in an embodiment of the present invention.

[0077] like Figure 3 As shown, the image editing device 30 based on the Hongmeng operating system includes: an image import module 31, an editing module 32, a cropping module 33, a tone adjustment module 34, a graffiti module 35, and a saving module 36.

[0078] The image import module 31 is used to import images stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system;

[0079] An editing module 32 is used for real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text-edited images;

[0080] a cropping module 33 for executing a cropping algorithm based on the Hongmeng AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping;

[0081] a tone adjustment module 34 for adjusting the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone-edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0082] a graffiti module 35 for graffiti on the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing;

[0083] The saving module 36 is used to save the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image in the distributed storage of the Harmony system in real time according to the I / O operation of the Harmony system.

[0084] Furthermore, in a possible implementation of the embodiment of the present invention, the cropping module 33 is specifically configured to:

[0085] Based on the Hongmeng AI engine, bilinear interpolation is used for image resizing to resize the input image to a first image of a fixed size;

[0086] Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space;

[0087] Performing subject object detection on the second image using the YOLO v5 model to detect a subject object bounding box and a confidence score for the second image;

[0088] Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image;

[0089] Combining the subject object bounding box and confidence score of the second image with the pixel-level segmentation mask, the second image is cropped using preset cropping rules to obtain a third image. The cropping rules include keeping the subject object intact, adhering to the rule of thirds composition principle, and avoiding cropping key facial areas.

[0090] The third image is cropped to multiple sizes using a preset size cropping ratio to obtain a cropped image.

[0091] Furthermore, in a possible implementation of the embodiment of the present invention, the hue adjustment module 34 is specifically configured to:

[0092] Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image;

[0093] The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size to obtain an adjusted first image tonal contrast;

[0094] The hue enhancement is completed by performing nonlinear mapping on the two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue-enhanced, and the lightness channel in the Lab color space is adjusted using an S-curve to obtain the second image hue contrast after the hue contrast of the first image is hue-enhanced;

[0095] Use the grayscale world hypothesis algorithm to estimate the color temperature of the light source corresponding to the real-time preview image, adjust the RGB channel gains in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast and second color saturation of the third image after applying RGB channel gains to the second image and the first color saturation;

[0096] Using guided filtering to separate a base layer and a detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, performing detail layer enhancement processing on the detail layer using an adaptive contrast enhancement algorithm, and recombining the enhanced detail layer with the base layer to obtain a second preview image;

[0097] A local tone mapping algorithm is used to simulate the HDR effect of the second preview image to obtain a tone-edited image.

[0098] Furthermore, in a possible implementation of the embodiment of the present invention, the graffiti module 35 is specifically configured to:

[0099] Collect the sequence of input points from the user's touch screen or stylus and record the collected data of each input point, including coordinates, pressure and timestamp;

[0100] The Kalman filter is used to smooth the collected data of each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points;

[0101] Use a cubic Bezier curve to fit the smoothed point sequence of the data to adjust the control points of the user's touch screen or stylus to adapt to different stroke shapes;

[0102] Calculate the stroke width of the user's touch screen or stylus based on the pressure value and velocity of the input point determined by the coordinates, pressure, and timestamp;

[0103] Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, thereby obtaining the texture and anti-aliasing of the user's touch screen or stylus;

[0104] Vertex shaders and fragment shaders are used to render the brush stroke shape, brush stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

[0105] Furthermore, in a possible implementation of the embodiment of the present invention, the device further includes:

[0106] The protection module is used to store the sensitive picture in a preset privacy zone and / or clean up the picture metadata of the sensitive picture when the real-time preview picture is a sensitive picture, so as to protect the privacy of the sensitive picture.

[0107] Furthermore, in a possible implementation of the embodiment of the present invention, the apparatus further includes:

[0108] The collaborative editing module is used for collaborative editing of text, cropping, tone adjustment and graffiti of the image on different devices based on the distributed operating subsystem of the Harmony system.

[0109] The image editing device based on the Hongmeng operating system of the embodiment of the present invention imports images stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; performs real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text-edited images; identifies the main body of the real-time preview image and crops it according to the Hongmeng AI engine to obtain a cropped image; adjusts the color tone according to the color tone adjustment algorithm of the Hongmeng system to obtain a color-edited image; graffiti is performed according to the multi-mode input technology of the Hongmeng system to obtain a graffiti image; and saves the images of the editing process in real time to the distributed storage of the Hongmeng system according to the I / O operation of the Hongmeng system. Thus, the Hongmeng operating system realizes the functions of fast editing, cropping, color tone adjustment and graffiti of images, thereby improving the user's operating efficiency and experience.

[0110] In order to implement the above embodiment, the present invention further provides an electronic device, including:

[0111] at least one processor; and

[0112] a memory communicatively connected to the at least one processor; wherein,

[0113] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the aforementioned method.

[0114] In order to implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the above method.

[0115] In order to implement the above embodiments, the present invention further provides a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0116] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0117] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0118] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0119] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0120] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0121] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0122] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0123] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limiting the present invention. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A picture editing method based on Hongmeng operating system, characterized in that: The image editing method based on the Hongmeng operating system is run on an electronic device equipped with the Hongmeng operating system, and the method includes: Import images stored locally or in the cloud using the distributed file subsystem of Hongmeng OS; Based on the GPU acceleration technology of Hongmeng system, real-time preview and text editing of pictures are performed to obtain real-time preview pictures and text editing pictures; Based on the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image. The cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping. Adjust the hue of the real-time preview image according to the hue adjustment algorithm of the Hongmeng system to obtain a hue-edited image, wherein the hue adjustment algorithm includes histogram analysis, image hue contrast adjustment, hue enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing; Graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing; According to the I / O operation of the Hongmeng system, the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image are saved in real time to the distributed storage of the Hongmeng system; The Hongmeng AI engine executes a cropping algorithm to identify the subject of the real-time preview image and crops it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image segmentation, cropping area optimization, and multi-size cropping, including: Based on the Hongmeng AI engine, bilinear interpolation is used for image resizing to resize the input image to a first image of a fixed size; Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space; Performing subject object detection on the second image using the YOLO v5 model to detect a subject object bounding box and a confidence score for the second image; Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; Combining the subject object bounding box and confidence score of the second image with the pixel-level segmentation mask, the second image is cropped using preset cropping rules to obtain a third image. The cropping rules include keeping the subject object intact, adhering to the rule of thirds composition principle, and avoiding cropping key facial areas. The third image is cropped to multiple sizes using a preset size cropping ratio to obtain a cropped image.

2. The method according to claim 1, wherein the tone of the real-time preview image is adjusted according to the tone adjustment algorithm of the Hongmeng system to obtain a tone edited image, wherein: The tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing, including: Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image; The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size to obtain an adjusted first image tonal contrast; The hue enhancement is completed by performing nonlinear mapping on the two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue-enhanced, and the lightness channel in the Lab color space is adjusted using an S-curve to obtain the second image hue contrast after the hue contrast of the first image is hue-enhanced; Use the grayscale world hypothesis algorithm to estimate the color temperature of the light source corresponding to the real-time preview image, adjust the RGB channel gains in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast and second color saturation of the third image after applying RGB channel gains to the second image and the first color saturation; Using guided filtering to separate a base layer and a detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, performing detail layer enhancement processing on the detail layer using an adaptive contrast enhancement algorithm, and recombining the enhanced detail layer with the base layer to obtain a second preview image; A local tone mapping algorithm is used to simulate the HDR effect of the second preview image to obtain a tone-edited image.

3. The method according to claim 1, characterized in that The graffiti stroke method includes a user touching a touch screen or a stylus, and when graffiti is performed on the real-time preview image according to the graffiti stroke method of the HarmonyOS system to obtain a graffiti image, the method includes: Collect the sequence of input points from the user's touch screen or stylus and record the collected data of each input point, including coordinates, pressure and timestamp; Use the Kalman filter to smooth the data collected from each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points; Use a cubic Bezier curve to fit the smoothed point sequence of the data to adjust the control points of the user's touch screen or stylus to adapt to different stroke shapes; Calculate the stroke width of the user's touch screen or stylus based on the pressure value and velocity of the input point determined by the coordinates, pressure, and timestamp; Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, thereby obtaining the texture and anti-aliasing of the user's touch screen or stylus; Vertex shaders and fragment shaders are used to render the brush stroke shape, brush stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

4. The method according to claim 1, wherein The method further comprises: In the case where the real-time preview image is a sensitive image, the sensitive image is stored in a preset privacy zone, and / or the image metadata of the sensitive image is cleaned to protect the privacy of the sensitive image.

5. The method according to claim 1, wherein The method further comprises: Based on the distributed operating subsystem of the Hongmeng system, text editing, cropping, tone adjustment and collaborative editing of graffiti of the above-mentioned pictures are performed on different devices.

6. A picture editing device based on Hongmeng operating system, characterized in that: The image editing device based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the device includes: The image import module is used to import images stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system; The editing module is used for real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text editing images; A cropping module, configured to execute a cropping algorithm based on the HarmonyOS AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image. The cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping. A tone adjustment module, configured to adjust the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone-edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing; A graffiti module, configured to graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing; A saving module is used to save the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image to the distributed storage of the Hongmeng system in real time according to the I / O operation of the Hongmeng system; The cutting module is specifically used for: Based on the Hongmeng AI engine, bilinear interpolation is used for image resizing to resize the input image to a first image of a fixed size; Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space; Performing subject object detection on the second image using the YOLO v5 model to detect a subject object bounding box and a confidence score for the second image; Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; Combining the subject object bounding box and confidence score of the second image with the pixel-level segmentation mask, the second image is cropped using preset cropping rules to obtain a third image. The cropping rules include keeping the subject object intact, adhering to the rule of thirds composition principle, and avoiding cropping key facial areas. The third image is cropped to multiple sizes using a preset size cropping ratio to obtain a cropped image.

7. The apparatus according to claim 6, wherein the hue adjustment module is specifically configured to: Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image; The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size to obtain an adjusted first image tonal contrast; The hue enhancement is completed by performing nonlinear mapping on the two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue-enhanced, and the lightness channel in the Lab color space is adjusted using an S-curve to obtain the second image hue contrast after the hue contrast of the first image is hue-enhanced; Use the grayscale world hypothesis algorithm to estimate the color temperature of the light source corresponding to the real-time preview image, adjust the RGB channel gains in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast and second color saturation of the third image after applying RGB channel gains to the second image and the first color saturation; Using guided filtering to separate a base layer and a detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, performing detail layer enhancement processing on the detail layer using an adaptive contrast enhancement algorithm, and recombining the enhanced detail layer with the base layer to obtain a second preview image; A local tone mapping algorithm is used to simulate the HDR effect of the second preview image to obtain a tone-edited image.

8. The device according to claim 6, characterized in that The graffiti module is specifically used to: Collect the sequence of input points from the user's touch screen or stylus and record the collected data of each input point, including coordinates, pressure and timestamp; Use the Kalman filter to smooth the data collected from each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points; Use a cubic Bezier curve to fit the smoothed point sequence of the data to adjust the control points of the user's touch screen or stylus to adapt to different stroke shapes; Calculate the stroke width of the user's touch screen or stylus based on the pressure value and velocity of the input point determined by the coordinates, pressure, and timestamp; Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, thereby obtaining the texture and anti-aliasing of the user's touch screen or stylus; Vertex shaders and fragment shaders are used to render the brush stroke shape, brush stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

9. The device according to claim 6, characterized in that The device further comprises: The protection module is used to store the sensitive picture in a preset privacy zone and / or clean up the picture metadata of the sensitive picture when the real-time preview picture is a sensitive picture, so as to protect the privacy of the sensitive picture.

10. The device according to claim 6, characterized in that The device further comprises: The collaborative editing module is used for collaborative editing of text, cropping, tone adjustment and graffiti of the image on different devices based on the distributed operating subsystem of the Harmony system.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

Images