Picture editing method and device based on swan operating system

Abstract

The invention provides a picture editing method and device based on a swan-gap operating system, and relates to the technical field of picture processing, the method is operated on electronic equipment carrying the swan-gap operating system, and pictures stored locally or in a cloud end are imported according to a distributed file subsystem of the swan-gap operating system; carrying out real-time preview and text editing on the picture based on a GPU acceleration technology of the swan Mongolian system so as to obtain a real-time preview picture and a text editing picture; identifying a main body of the real-time preview picture according to a swan AI engine, and cutting the main body to obtain a cut picture; adjusting the hue according to a hue adjustment algorithm of the swan-gap system to obtain a hue edited picture; graffiti is carried out according to a multi-mode input technology of a swan-mong system to obtain a graffiti picture; and according to the I / O operation of the swan-gap system, the picture in the editing process is stored in the distributed storage of the swan-gap system in real time, so that the rapid editing, cutting, hue adjusting and graffiti functions of the picture are realized based on the swan-gap operation system, and the operation efficiency and experience of a user are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image editing method and device based on Hongmeng operating system. Background Art

[0002] With the popularity of smart phones, users have a growing demand for image editing functions on mobile electronic devices. Existing image editing applications are often cumbersome and complex to operate. They cannot be edited collaboratively across devices, cannot meet users' needs for fast editing, and cannot provide a smooth user experience. Therefore, a more reliable method for fast image editing is urgently needed. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, the first purpose of the present invention is to propose a picture editing method based on the Hongmeng operating system, which realizes the functions of quick editing, cropping, tone adjustment and graffiti of pictures based on the Hongmeng operating system, thereby improving the user's operating efficiency and experience.

[0005] The second purpose of the present invention is to provide a picture editing device based on the Hongmeng operating system.

[0006] A third objective of the present invention is to provide an electronic device.

[0007] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions.

[0008] A fifth object of the present invention is to provide a computer program product.

[0009] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a picture editing method based on the Hongmeng operating system, and the picture editing method based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the method includes:

[0010] Import pictures stored locally or in the cloud using the distributed file subsystem of Hongmeng system;

[0011] Based on Hongmeng system's GPU acceleration technology, real-time preview and text editing of pictures are performed to obtain real-time preview pictures and text editing pictures;

[0012] According to the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping;

[0013] According to the tone adjustment algorithm of the Hongmeng system, adjust the tone of the real-time preview image to obtain a tone edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0014] Graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing;

[0015] According to the I / O operation of the Hongmeng system, the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image are saved in real time to the distributed storage of the Hongmeng system.

[0016] To achieve the above-mentioned purpose, the second aspect of the present invention proposes a picture editing device based on the Hongmeng operating system, and the picture editing device based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the device includes:

[0017] The image import module is used to import images stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system;

[0018] The editing module is used for real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text editing images;

[0019] A cropping module, which is used to execute a cropping algorithm according to the Hongmeng AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping;

[0020] A tone adjustment module, used to adjust the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0021] A graffiti module, used to graffiti the real-time preview image according to a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing and real-time rendering processing;

[0022] To achieve the above-mentioned purpose, the third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0023] In order to achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0024] In order to achieve the above-mentioned purpose, a fifth aspect of the present invention provides a computer program product, and when the computer program is executed by a processor, it implements the method described in the first aspect.

[0025] The embodiments of the present invention provide a picture editing method, device, electronic device and storage medium based on the Hongmeng operating system. The method runs on an electronic device equipped with the Hongmeng operating system, and imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; based on the GPU acceleration technology of the Hongmeng system, real-time preview and text editing of the pictures are performed to obtain real-time preview pictures and text-edited pictures; according to the Hongmeng AI engine, the subject of the real-time preview picture is identified and cropped to obtain a cropped picture; the color tone is adjusted according to the color tone adjustment algorithm of the Hongmeng system to obtain a color tone edited picture; graffiti is performed according to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture; according to the I / O operation of the Hongmeng system, the pictures in the editing process are saved in real time to the distributed storage of the Hongmeng system, thereby realizing the functions of fast editing, cropping, color tone adjustment and graffiti of pictures based on the Hongmeng operating system, thereby improving the user's operating efficiency and experience.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A schematic diagram of a flow chart of a picture editing method based on the Hongmeng operating system provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of a flow chart of another picture editing method based on the Hongmeng operating system provided by an embodiment of the present invention;

[0030] Figure 3A structural schematic diagram of a picture editing device based on the Hongmeng operating system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0032] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of relevant laws and regulations.

[0033] The following describes a picture editing method and device based on the Hongmeng operating system according to an embodiment of the present invention with reference to the accompanying drawings.

[0034] Figure 1 A flowchart of a picture editing method based on the Hongmeng operating system provided in an embodiment of the present invention.

[0035] like Figure 1 As shown, the picture editing method based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the method includes the following steps:

[0036] Step 101, import pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system.

[0037] In some embodiments, images are imported from local gallery, camera and cloud storage, and asynchronous loading can be used to ensure user interface responsiveness. In addition, for large-size images, thumbnails are loaded first, and then the full resolution is loaded step by step.

[0038] Step 102, based on the GPU acceleration technology of the Hongmeng system, real-time preview and text editing of the picture are performed to obtain real-time preview pictures and text editing pictures.

[0039] In some embodiments, the real-time preview image is displayed on the user interface of the HarmonyOS system. The user interface can be developed using a UI framework (HarmonyOS UI) to achieve responsive design, adapt to electronic device screens of different sizes, and support gesture operations and shortcut keys.

[0040] Optionally, graphics processing unit (GPU) acceleration technology can be developed based on the HarmonyOS Graphics Engine, and then GPU acceleration technology can be used to achieve real-time preview and text editing, support multi-threaded processing, and improve processing efficiency.

[0041] Step 103: According to the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping.

[0042] Optionally, the HarmonyOS AI Engine can use an integrated pre-trained deep learning model for image subject recognition and cropping, and supports online updating of deep learning models to continuously improve recognition accuracy.

[0043] In some embodiments, according to the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image. One implementation method can be: based on the Hongmeng AI engine, bilinear interpolation is used to resize the image to adjust the input image to a first image of a fixed size; the RGB image of the first image is converted into a Lab color space to obtain a second image in the Lab color space; the subject object of the second image is detected by the YOLO v5 model to detect the subject object bounding box and confidence score of the second image; the second image is semantically segmented using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; combined with the subject object bounding box and confidence score of the second image, as well as the pixel-level segmentation mask, the second image is cropped using a preset cropping rule to obtain a third image, wherein the cropping rule includes keeping the subject object of the image intact, following the rule of thirds composition principle, and avoiding cutting the key facial area of ​​the character; the third image is multi-size cropped according to a preset size cropping ratio to obtain a cropped image. In this way, intelligent subject recognition and automatic cropping are achieved, providing users with a better picture editing experience.

[0044] Furthermore, when adjusting the image size, the fixed size of the image may be 224×224 pixels, but is not limited thereto.

[0045] Furthermore, when performing multi-size cropping, the preset size cropping ratio can be commonly used ratios such as 1:1, 4:3, 16:9, etc. In addition, the aesthetic score of each ratio cropped image can be calculated to select the cropped image with a higher aesthetic score, wherein the aesthetic score can be calculated using an aesthetic evaluation model based on deep learning.

[0046] In addition, according to the Hongmeng AI engine, the cropping algorithm is executed asynchronously in the background thread, and a cache mechanism is used to store intermediate results to avoid repeated calculations. It also supports progressive processing, returning rough results first and then gradually optimizing.

[0047] Step 104, according to the tone adjustment algorithm of the Hongmeng system, adjust the tone of the real-time preview picture to obtain a tone edited picture, wherein the tone adjustment algorithm includes histogram analysis, picture tone contrast adjustment, tone enhancement, white balance correction, picture detail layer enhancement processing, and picture HDR effect simulation processing.

[0048] In some embodiments, according to the hue adjustment algorithm of the Hongmeng system, the hue of the real-time preview image is adjusted to obtain an implementation method of the hue edited image, which can be to calculate the histogram of the image of the real-time preview image in the RGB and Lab color spaces to perform histogram analysis and determine the light and dark contrast and color saturation of the image; the adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content, and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size to obtain the adjusted hue contrast of the first image; the hue enhancement is completed by performing nonlinear mapping on the two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue enhanced, and the lightness channel in the Lab color space is adjusted using the S curve to obtain the hue contrast of the second image after the hue contrast of the first image is hue enhanced; the gray world hypothesis algorithm is used to estimate the light source color temperature corresponding to the real-time preview image, and the RGB channel gain in the Lab color space is adjusted according to the light source color temperature to complete the white balance correction, and the hue contrast of the second image and the hue contrast of the third image after the RGB channel gain of the first color saturation are obtained; the guided filtering (Guided filtering) is used to obtain the hue contrast of the third image and the second color saturation after the RGB channel gain of the first color saturation is obtained. The first preview image is separated by a filter (filter) from the base layer and detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, and the detail layer is enhanced by an adaptive contrast enhancement algorithm, and the detail layer after the detail layer enhancement is recombined with the base layer to obtain a second preview image; the second preview image is simulated by a local tone mapping algorithm to obtain a tone edited image. Thus, accurate color adjustment is achieved.

[0049] Furthermore, when adjusting the image tone contrast, the adaptive histogram equalization method may be a CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm.

[0050] Furthermore, when performing HDR effect simulation processing on an image, highlight and shadow details can be maintained while enhancing the overall dynamic range to achieve HDR effect simulation of the second preview image and obtain a reliable tone-edited image.

[0051] Step 105, graffiti a real-time preview image according to a variety of graffiti methods corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke method in the graffiti method includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing.

[0052] In some embodiments, one implementation method of graffiti real-time previewing a picture according to multiple graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti picture can be to collect a sequence of input points of a user's touch screen or stylus, and record the collected data of each input point, the collected data including coordinates, pressure and timestamp; use a Kalman filter to perform data smoothing on the collected data of each input point sequence to remove the noise input points generated by hand jitter to obtain a point sequence of noise-free input points; use a cubic Bezier curve to fit the point sequence after data smoothing to adjust the control points of the user's touch screen or stylus to adapt to different stroke shapes; calculate the stroke width of the user's touch screen or stylus based on the pressure value and speed of the input point determined by the coordinates, pressure and timestamp; use supersampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing to obtain the texture and anti-aliasing of the user's touch screen or stylus; use a vertex shader and a fragment shader to render the stroke shape, stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti picture. Therefore, through functions such as multi-mode input, the user operation process is simplified and the user experience is enhanced.

[0053] Furthermore, when calculating the stroke width, an exponential function may be used to map the pressure value to the width range, and the speed may be taken into consideration. The faster the speed, the thinner the stroke width.

[0054] Furthermore, when performing real-time rendering processing, a subset (OpenGL ES) of the Open Graphics Library (OpenGL) 3D graphics API may be used to perform GPU accelerated real-time rendering.

[0055] In addition, you can use view hierarchy optimization to reduce overdrawing and optimize real-time rendering processing.

[0056] Step 106, according to the I / O operation of the Hongmeng system, save the real-time preview picture, text editing picture, cropped picture, tone editing picture, and graffiti picture in real time to the distributed storage of the Hongmeng system.

[0057] In some embodiments, the distributed storage can be a HarmonyOS distributed file system, which uses the HarmonyOS distributed file system to achieve seamless switching between local and cloud storage, and support incremental saving and version control.

[0058] Optionally, all image text editing, cropping, tone adjustment and graffiti functions are edited based on a copy of the original image, and use command mode to record each editing step, support undo and redo, and use low-resolution preview during editing to improve response speed.

[0059] Furthermore, during the real-time saving process, an incremental saving strategy can be used to save only the changed parts, and the Copy-on-Write technology can be used to optimize storage efficiency, and then the incremental updates can be regularly merged into complete files to ensure data consistency.

[0060] In some embodiments, when managing distributed storage, memory pool technology can be used to reduce memory fragmentation, large image processing adopts a block loading strategy, and weak references are used to cache non-critical data to respond to memory pressure in a timely manner.

[0061] The picture editing method based on the Hongmeng operating system of the embodiment of the present invention runs on an electronic device equipped with the Hongmeng operating system, and imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; performs real-time preview and text editing on the pictures based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview pictures and text-edited pictures; identifies the subject of the real-time preview picture and crops it according to the Hongmeng AI engine to obtain a cropped picture; adjusts the color tone according to the color tone adjustment algorithm of the Hongmeng system to obtain a color tone edited picture; doodles according to the multi-mode input technology of the Hongmeng system to obtain a doodled picture; and saves the pictures of the editing process in real time to the distributed storage of the Hongmeng system according to the I / O operation of the Hongmeng system, thereby realizing the functions of fast editing, cropping, color tone adjustment and doodles of pictures based on the Hongmeng operating system, thereby improving the user's operating efficiency and experience.

[0062] In order to clearly illustrate the previous embodiment, this embodiment also provides another image editing method based on the Hongmeng operating system, such as Figure 2 As shown, specifically, you can select a picture from the pictures imported into the local or cloud storage, and then edit the selected picture by selecting an operation item. The selected operation items include cropping, text editing, tone adjustment and graffiti. According to the I / O operation of the Hongmeng system, the text-edited picture, cropped picture, tone-edited picture and graffiti picture after cropping, text editing, tone adjustment and graffiti are saved in real time, and the text-edited picture, cropped picture, tone-edited picture and graffiti picture are output and saved to the distributed storage of the Hongmeng system. In addition, you can re-execute the selection operation item to process the text-edited picture, cropped picture, tone-edited picture and graffiti picture saved in real time. In this way, the characteristics of the Hongmeng system are fully utilized, and efficient and intelligent picture editing functions are realized, providing users with a better picture editing experience.

[0063] In addition, when the real-time preview image is a sensitive image, the sensitive image is stored in a preset privacy area, and / or the image metadata of the sensitive image is cleaned to protect the privacy of the sensitive image and implement privacy management of sensitive images.

[0064] In some embodiments, based on the distributed operating subsystem of the Hongmeng system, text editing, cropping, color adjustment and collaborative editing of the image are performed on different devices. Thus, based on the distributed capabilities of the Hongmeng system, a seamless editing experience across devices is achieved.

[0065] Specifically, the distributed operation subsystem can be the HarmonyOS distributed soft bus, which can realize real-time data synchronization and collaborative editing among various electronic devices, as well as seamless operation experience across devices.

[0066] Furthermore, when performing cross-device collaborative editing, a differential synchronization algorithm can be used to transmit only changed data, while giving priority to P2P direct connection to reduce network latency. In addition, when conflicts arise in cross-device collaborative editing, timestamps are given priority, supplemented by user confirmation.

[0067] In some embodiments, the processing performance of the image editing method based on the Hongmeng operating system can be dynamically adjusted according to the battery power of the electronic device. Specifically, task scheduling can be used to schedule intensive calculations to be performed when the electronic device is charging; network requests can be optimized to reduce unnecessary data transmission.

[0068] In some embodiments, delayed loading and on-demand loading strategies can also be used; commonly used resources can be preloaded; and compile-time annotation processing can be used to reduce runtime reflection overhead and achieve startup optimization of the image editing method based on the Hongmeng operating system.

[0069] In some embodiments, the AES-256 encryption algorithm can also be used to protect locally stored images and editing data during editing, cropping, tone adjustment, and doodling, while the TLS1.3 protocol is used for network transmission to ensure the security of transmission of locally stored images and editing data.

[0070] In some embodiments, the principle of least privilege can also be adopted to request only necessary system permissions, provide fine-grained permission control, allow users to customize the permissions of each function, and implement detailed permission management of image editing methods based on the Hongmeng operating system.

[0071] In some embodiments, digital signature technology can also be used to verify the integrity of the image editing method based on the Hongmeng operating system, prevent tampering, implement runtime integrity checks, and detect potential injection attacks.

[0072] In some embodiments, a plug-in API can also be designed so that the image editing method based on the Hongmeng operating system supports third-party developers to extend editing functions, and use a sandbox mechanism to isolate the plug-in running environment to ensure system security.

[0073] In some embodiments, a cloud API can also be designed so that the image editing method based on the Hongmeng operating system supports future expansion of cloud AI processing capabilities, and reserves a data synchronization interface to prepare for cross-platform (device) editing.

[0074] In some embodiments, resource files can also be used in the picture editing method based on the Hongmeng operating system to separate UI texts for easy localization and support dynamic language package downloads without having to update the entire execution application of the picture editing method based on the Hongmeng operating system.

[0075] In order to implement the above embodiments, the present invention also proposes a picture editing device based on the Hongmeng operating system.

[0076] Figure 3 A structural schematic diagram of a picture editing device based on the Hongmeng operating system provided in an embodiment of the present invention.

[0077] like Figure 3 As shown, the picture editing device 30 based on the Hongmeng operating system includes: a picture import module 31, an editing module 32, a cropping module 33, a tone adjustment module 34, a graffiti module 35, and a saving module 36.

[0078] The picture import module 31 is used to import pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system;

[0079] The editing module 32 is used for real-time preview and text editing of pictures based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview pictures and text editing pictures;

[0080] A cropping module 33 is used to execute a cropping algorithm according to the Hongmeng AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization and multi-size cropping;

[0081] A tone adjustment module 34 is used to adjust the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing;

[0082] The graffiti module 35 is used to graffiti the real-time preview image according to a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing;

[0083] The saving module 36 is used to save the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image in the distributed storage of the Hongmeng system in real time according to the I / O operation of the Hongmeng system.

[0084] Furthermore, in a possible implementation of the embodiment of the present invention, the cropping module 33 is specifically configured to:

[0085] Based on the Hongmeng AI engine, bilinear interpolation is used to resize the image to adjust the input image to a first image of a fixed size;

[0086] Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space;

[0087] Performing subject object detection on the second image through the YOLO v5 model to detect a subject object bounding box and a confidence score of the second image;

[0088] Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image;

[0089] Combine the subject object bounding box and confidence score of the second image, and the pixel-level segmentation mask, and use a preset cropping rule to crop the second image to obtain a third image, wherein the cropping rule includes keeping the subject object of the image intact, following the rule of thirds composition principle, and avoiding cutting key facial areas of the person;

[0090] The third image is cropped into multiple sizes according to a preset size cropping ratio to obtain a cropped image.

[0091] Furthermore, in a possible implementation of the embodiment of the present invention, the hue adjustment module 34 is specifically configured to:

[0092] Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image;

[0093] The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content, and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size, and obtain the adjusted first image hue contrast;

[0094] The hue enhancement is completed by performing nonlinear mapping on two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue enhanced, and the lightness channel in the Lab color space is adjusted using an S curve to obtain the hue contrast of the second image after the hue contrast of the first image is hue enhanced;

[0095] Use the grayscale world hypothesis algorithm to estimate the light source color temperature corresponding to the real-time preview image, adjust the RGB channel gain in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast of the second image and the hue saturation of the first color after performing RGB channel gain on the hue contrast of the second image and the hue saturation of the first color;

[0096] Use guided filtering to separate the base layer and detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, perform image detail layer enhancement processing on the detail layer through an adaptive contrast enhancement algorithm, and recombine the enhanced detail layer with the base layer to obtain a second preview image;

[0097] The second preview image is processed by simulating the HDR effect of the image using a local tone mapping algorithm to obtain a tone-edited image.

[0098] Furthermore, in a possible implementation of the embodiment of the present invention, the graffiti module 35 is specifically used to:

[0099] Collect the input point sequence of the user's touch screen or stylus, and record the collected data of each input point, including coordinates, pressure and timestamp;

[0100] The Kalman filter is used to perform data smoothing on the collected data of each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points;

[0101] Use a cubic Bezier curve to fit the point sequence after data smoothing to adjust the control points of the user's touch screen or stylus to adapt to different pen stroke shapes;

[0102] Calculate the stroke width of the user's touch screen or stylus based on the pressure value and speed of the input point determined by the coordinates, pressure and timestamp;

[0103] Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, and obtain the texture and anti-aliasing of the user's touch screen or stylus;

[0104] Vertex shaders and fragment shaders are used to render the stroke shape, stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

[0105] Furthermore, in a possible implementation of the embodiment of the present invention, the device further includes:

[0106] The protection module is used to store the sensitive picture in a preset privacy zone and / or clean up the picture metadata of the sensitive picture when the real-time preview picture is a sensitive picture, so as to protect the privacy of the sensitive picture.

[0107] Furthermore, in a possible implementation of the embodiment of the present invention, the device further includes:

[0108] The collaborative editing module is used for collaborative editing of text editing, cropping, tone adjustment and graffiti of the image on different devices based on the distributed operating subsystem of the Hongmeng system.

[0109] The picture editing device based on the Hongmeng operating system of the embodiment of the present invention imports pictures stored locally or in the cloud according to the distributed file subsystem of the Hongmeng system; performs real-time preview and text editing on the pictures based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview pictures and text-edited pictures; identifies the main body of the real-time preview picture and crops it according to the Hongmeng AI engine to obtain a cropped picture; adjusts the color tone according to the color tone adjustment algorithm of the Hongmeng system to obtain a color tone edited picture; doodles according to the multi-mode input technology of the Hongmeng system to obtain a doodled picture; and saves the pictures in the editing process in real time to the distributed storage of the Hongmeng system according to the I / O operation of the Hongmeng system, thereby realizing the functions of fast editing, cropping, color tone adjustment and doodles of pictures based on the Hongmeng operating system, thereby improving the user's operating efficiency and experience.

[0110] In order to implement the above embodiment, the present invention further provides an electronic device, including:

[0111] at least one processor; and

[0112] a memory communicatively connected to the at least one processor; wherein,

[0113] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the aforementioned method.

[0114] In order to implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the above method.

[0115] In order to implement the above embodiments, the present invention further provides a computer program product, including a computer program, wherein the computer program implements the above method when executed by a processor.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0117] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0118] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0120] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0121] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0122] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules. If the integrated module is implemented in the form of software functional modules and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0123] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A picture editing method based on Hongmeng operating system, characterized in that: The picture editing method based on the Hongmeng operating system is run on an electronic device equipped with the Hongmeng operating system, and the method includes: Import pictures stored locally or in the cloud using the distributed file subsystem of Hongmeng system; Based on Hongmeng system's GPU acceleration technology, real-time preview and text editing of pictures are performed to obtain real-time preview pictures and text editing pictures; According to the Hongmeng AI engine, a cropping algorithm is executed to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping; According to the tone adjustment algorithm of the Hongmeng system, adjust the tone of the real-time preview image to obtain a tone edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing; Graffiti the real-time preview image in a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing, and real-time rendering processing; According to the I / O operation of the Hongmeng system, the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image are saved in real time to the distributed storage of the Hongmeng system.

2. According to the method of claim 1, the Hongmeng AI engine executes a cropping algorithm to identify the subject of the real-time preview image and crops it to obtain a cropped image, wherein: Cropping algorithms include image resizing, color space conversion, image segmentation, cropping region optimization, and multi-size cropping, including: Based on the Hongmeng AI engine, bilinear interpolation is used to resize the image to adjust the input image to a first image of a fixed size; Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space; Performing subject object detection on the second image through the YOLO v5 model to detect a subject object bounding box and a confidence score of the second image; Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; Combine the subject object bounding box and confidence score of the second image, and the pixel-level segmentation mask, and use a preset cropping rule to crop the second image to obtain a third image, wherein the cropping rule includes keeping the subject object of the image intact, following the rule of thirds composition principle, and avoiding cutting key facial areas of the person; The third image is cropped into multiple sizes according to a preset size cropping ratio to obtain a cropped image.

3. The method according to claim 1, wherein the tone of the real-time preview image is adjusted according to the tone adjustment algorithm of the Hongmeng system to obtain a tone edited image, wherein: The tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing, including: Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image; The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content, and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size, and obtain the adjusted first image hue contrast; The hue enhancement is completed by performing nonlinear mapping on two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue enhanced, and the lightness channel in the Lab color space is adjusted using an S curve to obtain the hue contrast of the second image after the hue contrast of the first image is hue enhanced; Use the grayscale world hypothesis algorithm to estimate the light source color temperature corresponding to the real-time preview image, adjust the RGB channel gain in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast of the second image and the hue saturation of the first color after performing RGB channel gain on the hue contrast of the second image and the hue saturation of the first color; Use guided filtering to separate the base layer and detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, perform image detail layer enhancement processing on the detail layer through an adaptive contrast enhancement algorithm, and recombine the enhanced detail layer with the base layer to obtain a second preview image; The second preview image is processed by simulating the HDR effect of the image using a local tone mapping algorithm to obtain a tone-edited image.

4. The method according to claim 1, characterized in that: The graffiti stroke method includes a user touch screen or a stylus, and when graffitiing the real-time preview image according to the graffiti stroke method of the Hongmeng system to obtain a graffiti image, it includes: Collect the input point sequence of the user's touch screen or stylus, and record the collected data of each input point, including coordinates, pressure and timestamp; The Kalman filter is used to perform data smoothing on the collected data of each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points; Use a cubic Bezier curve to fit the point sequence after data smoothing to adjust the control points of the user's touch screen or stylus to adapt to different pen stroke shapes; Calculate the stroke width of the user's touch screen or stylus based on the pressure value and speed of the input point determined by the coordinates, pressure and timestamp; Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, and obtain the texture and anti-aliasing of the user's touch screen or stylus; Vertex shaders and fragment shaders are used to render the stroke shape, stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

5. The method according to claim 1, characterized in that The method further comprises: When the real-time preview image is a sensitive image, the sensitive image is stored in a preset privacy zone, and / or the image metadata of the sensitive image is cleaned to protect the privacy of the sensitive image.

6. The method according to claim 1, characterized in that The method further comprises: Based on the distributed operating subsystem of the Hongmeng system, collaborative editing of text editing, cropping, tone adjustment and graffiti of the above pictures can be performed on different devices.

7. A picture editing device based on Hongmeng operating system, characterized in that: The picture editing device based on the Hongmeng operating system runs on an electronic device equipped with the Hongmeng operating system, and the device includes: The image import module is used to import images stored locally or in the cloud based on the distributed file subsystem of the Hongmeng system; The editing module is used for real-time preview and text editing of images based on the GPU acceleration technology of the Hongmeng system to obtain real-time preview images and text editing images; A cropping module, which is used to execute a cropping algorithm according to the Hongmeng AI engine to identify the subject of the real-time preview image and crop it to obtain a cropped image, wherein the cropping algorithm includes image resizing, color space conversion, image subject object detection, semantic segmentation, cropping area optimization, and multi-size cropping; A tone adjustment module, used to adjust the tone of the real-time preview image according to the tone adjustment algorithm of the Hongmeng system to obtain a tone edited image, wherein the tone adjustment algorithm includes histogram analysis, image tone contrast adjustment, tone enhancement, white balance correction, image detail layer enhancement processing, and image HDR effect simulation processing; A graffiti module, used to graffiti the real-time preview image according to a plurality of graffiti modes corresponding to the multi-mode input technology of the Hongmeng system to obtain a graffiti image, wherein the graffiti stroke mode in the graffiti mode includes image data collection, data smoothing, Bezier curve fitting, stroke width calculation, texture and anti-aliasing processing and real-time rendering processing; The saving module is used to save the real-time preview image, text editing image, cropped image, tone editing image, and graffiti image in the distributed storage of the Hongmeng system in real time according to the I / O operation of the Hongmeng system.

8. The device according to claim 7, wherein the cropping module is specifically configured to: Based on the Hongmeng AI engine, bilinear interpolation is used to resize the image to adjust the input image to a first image of a fixed size; Performing a color space conversion on the RGB image of the first picture to convert it into a Lab color space, thereby obtaining a second picture in the Lab color space; Performing subject object detection on the second image through the YOLO v5 model to detect a subject object bounding box and a confidence score of the second image; Perform semantic segmentation on the second image using the U-Net model to obtain a pixel-level segmentation mask corresponding to the second image; Combine the subject object bounding box and confidence score of the second image, and the pixel-level segmentation mask, and use a preset cropping rule to crop the second image to obtain a third image, wherein the cropping rule includes keeping the subject object of the image intact, following the rule of thirds composition principle, and avoiding cutting key facial areas of the person; The third image is cropped into multiple sizes according to a preset size cropping ratio to obtain a cropped image.

9. The device according to claim 7, wherein the hue adjustment module is specifically configured to: Calculate the histogram of the real-time preview image in RGB and Lab color space to perform histogram analysis and determine the light and dark contrast and color saturation of the image; The adaptive histogram equalization method dynamically adjusts the contrast limit threshold according to the image content, and adjusts the grid size according to the complexity of the image, so as to optimize the light and dark contrast based on the contrast limit threshold and the grid size, and obtain the adjusted first image hue contrast; The hue enhancement is completed by performing nonlinear mapping on two color channels in the Lab color space to obtain the first color saturation after the color saturation is hue enhanced, and the lightness channel in the Lab color space is adjusted using an S curve to obtain the hue contrast of the second image after the hue contrast of the first image is hue enhanced; Use the grayscale world hypothesis algorithm to estimate the light source color temperature corresponding to the real-time preview image, adjust the RGB channel gain in the Lab color space according to the light source color temperature to complete white balance correction, and obtain the hue contrast of the second image and the hue saturation of the first color after performing RGB channel gain on the hue contrast of the second image and the hue saturation of the first color; Use guided filtering to separate the base layer and detail layer of the first preview image formed by the hue contrast of the third image and the second color saturation, perform image detail layer enhancement processing on the detail layer through an adaptive contrast enhancement algorithm, and recombine the enhanced detail layer with the base layer to obtain a second preview image; The second preview image is processed by simulating the HDR effect of the image using a local tone mapping algorithm to obtain a tone-edited image.

10. The device according to claim 7, characterized in that The graffiti module is specifically used for: Collect the input point sequence of the user's touch screen or stylus, and record the collected data of each input point, including coordinates, pressure and timestamp; The Kalman filter is used to perform data smoothing on the collected data of each input point sequence to remove the noise input points generated by hand shaking and obtain a point sequence of noise-free input points; Use a cubic Bezier curve to fit the point sequence after data smoothing to adjust the control points of the user's touch screen or stylus to adapt to different pen stroke shapes; Calculate the stroke width of the user's touch screen or stylus based on the pressure value and speed of the input point determined by the coordinates, pressure and timestamp; Using super sampling anti-aliasing technology to smooth the edge of the user's touch screen or stylus to complete texture and anti-aliasing processing, and obtain the texture and anti-aliasing of the user's touch screen or stylus; Vertex shaders and fragment shaders are used to render the stroke shape, stroke width, and texture and anti-aliasing of the user's touch screen or stylus in real time to obtain a graffiti image.

11. The device according to claim 7, characterized in that The device also includes: The protection module is used to store the sensitive picture in a preset privacy zone and / or clean up the picture metadata of the sensitive picture when the real-time preview picture is a sensitive picture, so as to protect the privacy of the sensitive picture.

12. The device according to claim 7, characterized in that The device also includes: The collaborative editing module is used for collaborative editing of text editing, cropping, tone adjustment and graffiti of the image on different devices based on the distributed operating subsystem of the Hongmeng system.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

Images