Image processing method, electronic device, chip system and storage medium

Abstract

The embodiment of the present application provides an image processing method, which relates to the field of terminal technology. The method includes: when a user takes a photo, the camera application converts the data format of the thumbnail image into a target format based on the GPU acceleration capability of the open node in the image engine, and then a preset watermark can be added to the thumbnail image. In this way, when the user clicks on the thumbnail image to view the target image after taking the photo, the target image will also have a watermark, avoiding the sudden change of the watermark of the target image from nothing to something, maintaining the consistency of the image content, and improving the user experience of adding watermarks when taking photos.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method, electronic equipment, a chip system and a storage medium. Background Art

[0002] With the development of smartphone technology, more and more users will add watermarks when taking photos. Watermarks can not only enhance the visual effects of photos, but also protect the copyright of works. Therefore, it is becoming more and more important to optimize the technology of adding watermarks when taking photos.

[0003] One of the current ways to add watermarks when taking photos is that after the camera captures the image data, the preset watermark image is merged with the original image in the HAL, and then the photo with the watermark is saved. At the same time, the photo application often generates a thumbnail image for users to quickly browse and confirm the content of the photo.

[0004] However, generally no watermark is added to the thumbnail image. That is, the existing technology of adding watermarks to photos has such a problem: the image content of the thumbnail image is inconsistent with the image content of the final output photo, which affects the user experience. Summary of the invention

[0005] The embodiment of the present application provides an image processing method, which is applied to the field of terminal technology. The image processing method can quickly add watermarks to thumbnail images, so that the image content seen by users can remain the same, improving the user experience of adding watermarks when taking photos.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] In response to a first operation for triggering a photo-taking operation in the first interface, a thumbnail image is displayed in the first interface, the thumbnail image includes first watermark information, and the size of the thumbnail image in the first interface is a first size;

[0008] In response to a second operation on the thumbnail image, a target image is displayed in a second interface, the target image includes the first watermark information, and the size of the target image in the second interface is a second size, which is larger than the first size;

[0009] The target image is a thumbnail image, or the target image is an output image corresponding to the thumbnail image.

[0010] In this implementation, after taking a photo, the user clicks the control corresponding to the thumbnail image in the first interface, jumps to the second interface to view the target image, and the first watermark information is included in the target image. There are two specific situations for the target image. One situation is that when the user clicks the control corresponding to the thumbnail image, the output image has been generated. At this time, the target image displayed in the second interface is the output image, which carries the first watermark information. Another situation may be more common, that is, when the user clicks the control corresponding to the thumbnail image, the output image needs to go through more processing processes and has not been generated. At this time, the target image displayed in the second interface is actually a thumbnail image. In the prior art, since it is difficult to add watermarks to the closed source format of thumbnails, and in order to ensure the rapid generation of thumbnail images, the target image in the second interface does not include the first watermark information when it is a thumbnail image. In contrast, in this implementation, a watermark is also added to the thumbnail image through an image processing method, so that the target image seen in the second interface, whether it is a thumbnail image or an output image, will carry the first watermark information, and there will be no inconsistency in image content, which will improve the user experience.

[0011] In a possible implementation, the image processing method is applied to a terminal device, and the terminal device includes an application layer and a hardware abstraction layer HAL;

[0012] The application layer includes a first application, and the first interface and the second interface are interfaces in the first application;

[0013] The HAL includes an open node, which is used to provide an interface for applications in the application layer to access the chip.

[0014] In this implementation, by using the chip capability exposure node set in the HAL, the thumbnail image can be quickly converted into a format, and then a watermark can be added, which overcomes the problem of difficulty in adding watermarks to thumbnail images in the existing technical solutions. When the user views the target image in the second interface of the first application after taking a photo, the image can always be watermarked, ensuring the consistency of the screen content in the second interface. In this way, while maintaining the rapid generation of thumbnail images so that the user can view the target image immediately after taking a photo, a watermark is also added to the thumbnail image, so that the user will not see a thumbnail image that is significantly different from the output image, which can enhance the user's experience of taking photos and adding watermarks using the terminal device.

[0015] In a possible implementation, the image processing method further includes:

[0016] In response to a third operation for selecting a watermark template in the first interface, a target watermark template is determined, and the first watermark information is generated according to the target watermark template.

[0017] In this implementation, users can obtain watermark resources from cloud servers. Compared with the existing technology of obtaining watermark resources locally, this can greatly enrich the watermark template resources available for users to choose from and save local storage space. It can be understood that through the rich watermark template resources, users can find their favorite watermark templates, and then they may be more likely to use the function of adding watermarks when taking photos. In this way, the series of optimizations and improvements related to adding watermarks provided by this method are more meaningful.

[0018] In a possible implementation manner, after determining the target watermark template, the image processing method further includes:

[0019] Determine a preset image size according to the image resolution selected by the user in the first interface;

[0020] Determine a watermark size according to a preset image size, wherein a length of the watermark size in a first direction is less than or equal to a length of the image size in the first direction;

[0021] The target size is determined according to the preset image size and the watermark size, wherein the target size is greater than or equal to the preset image size, and the target size is greater than or equal to the watermark size; the sizes of the thumbnail image and the output image are both the target size.

[0022] In this implementation, the method for determining the preset image size, watermark size and target size is explained. Among them, the preset image size can also be understood as the original image size, which is the size corresponding to the image resolution selected by the user. The watermark size refers to the size corresponding to the watermark template when it is added to the original image, and the size cannot be larger than the size of the corresponding original image in the width or height direction. The target size refers to the size of the image obtained after the watermark is added to the original image. Since some watermarks will extend in height or width of the original image, the target size is generally greater than or equal to the preset image size. The target size calculation method proposed in this implementation can maintain the overall beauty and coordination when different types of watermark templates are applied to original images of different sizes. At the same time, the calculated target size can also provide a reference for the cache size required for subsequent image processing.

[0023] In a possible implementation, in response to a first operation for triggering photographing in the first interface, displaying a thumbnail image in the first interface includes:

[0024] In response to the first operation, the first application sends first information to the HAL, where the first information includes a first enabling message and a target size, and the first enabling message is used to indicate the use of an open node;

[0025] HAL applies for the first cache according to the target size, and the size of the first cache is the target size;

[0026] In response to the first enable message, the HAL sends the first cache, the first original image, and the watermark filling information to the first application through the open node, where the first original image is an initially generated thumbnail image;

[0027] The first application generates a thumbnail image according to the first cache, the first original image, the watermark filling information and the target watermark template;

[0028] The first application displays thumbnail images in the first interface.

[0029] In this implementation, the process of the image processing method is described as a whole from the perspective of system layering. After the user triggers taking a photo, the first application in the application layer sends the first information to the HAL, and then after receiving the first information, the HAL returns the information and data required for adding watermark processing to the first application through the open node, and then the first application generates a thumbnail image with a watermark according to the information and data returned by the HAL, and displays the thumbnail image in the first interface.

[0030] From the above process, it can be seen that in this implementation, the process of adding watermarks is placed in the application layer, and HAL is only responsible for receiving instructions and transmitting corresponding data. Compared with the prior art of adding watermarks in HAL and obtaining watermark resources locally, this implementation can achieve the decoupling of the application layer and HAL in the process of adding watermarks when taking pictures. That is, the first application of the application layer can update and iterate the specific logic of adding watermarks according to needs without the need for synchronous adjustment of HAL. In addition, the watermark resources are obtained from the cloud server and then directly processed by the application layer, which can also make the processing process more efficient.

[0031] In a possible implementation, the first application generates a thumbnail image according to the first cache, the first original image, the watermark filling information, and the target watermark template, including:

[0032] The first application generates a first watermark image corresponding to the first watermark information according to the target watermark template and the watermark filling information;

[0033] The first application adds the first original image and the first watermark image in the first buffer to generate a thumbnail image.

[0034] In a possible implementation, the image processing method further includes:

[0035] The first application calls the open node to convert the format of the first original image into the target format; wherein the first original image converted into the target format is added to the first cache.

[0036] In this implementation, it is explained how to use the open nodes in HAL to help complete the addition of watermarks on the first original image corresponding to the thumbnail image. It is understandable that the reason why the prior art does not add watermarks to thumbnail images is, on the one hand, to ensure the rapid generation of thumbnail images, and on the other hand, because thumbnail images are originally a closed-source format and are difficult to edit directly. If a processor is used to convert the original format of the thumbnail image into a target format that is easy to add a watermark, then the format conversion requires a lot of computing power, resulting in the actual processing being prolonged, and thus it is difficult to ensure the rapid generation of thumbnail images. In this implementation, since the open node can call the corresponding chip's capabilities, such as GPU acceleration, then based on such efficient parallel processing capabilities, it is possible to quickly convert the original format of the thumbnail image into the target format, thus overcoming the above-mentioned problems.

[0037] In a possible implementation, in response to the first operation, after the first application sends the first information to the HAL, the image processing method further includes:

[0038] HAL applies for a second cache based on the target size;

[0039] In response to the first enable message, the HAL sends the second cache, the second original image, and the watermark filling information to the first application through the open node, where the second original image is the initially generated output image;

[0040] The first application generates an intermediate image according to the second cache, the second original image, the watermark filling information and the target watermark template;

[0041] The first application sends the intermediate image and image information of the intermediate image to the HAL;

[0042] HAL encapsulates the intermediate image and image information to obtain the output image.

[0043] In this implementation, the overall image processing flow of the output image is described. The image processing flow of the output image can be parallel to the image processing flow of the thumbnail image, and unlike the image processing flow of the thumbnail image, the output image does not need to undergo format conversion processing of the open node, and needs to generate an intermediate image and corresponding image information first, and then be compressed and encapsulated. In terms of beneficial effects, similar to thumbnail images, adding watermarks to output images and some more image processing processes are transferred from HAL to the application layer, so that HAL only needs to maintain and provide a relatively basic set of interfaces, while the complex processing logic is implemented and adjusted by the first application in the application layer, which significantly reduces the coupling between the application layer and HAL, making the development and maintenance of terminal devices and applications simpler.

[0044] In a possible implementation, the first application generates an intermediate image according to the second cache, the second original image, the watermark filling information, and the target watermark template, including:

[0045] The first application generates a first watermark image corresponding to the first watermark information according to the target watermark template and the watermark filling information;

[0046] The first application adds the second original image and the first watermark image in the second buffer to generate an intermediate image.

[0047] In a possible implementation, the image processing method further includes:

[0048] The first application crops the second original image to obtain a cropped second original image and a cropped hidden image, and the image information of the intermediate image includes the hidden image.

[0049] In this implementation, the method of obtaining the intermediate image and the hidden image in the image information is explained. Among them, the first application crops the second original image, which means that the added first watermark image will overlap with a part of the second original image, and the information of the original image corresponding to this overlapping area will be lost. If you want to edit the watermark again or restore the original image later, you need to complete the information lost when adding the watermark. Therefore, the hidden image cropped and included in the image information here refers to the part of the image in which the information in the original image is lost due to the addition of the watermark. Through such a processing process, sufficient preparation is made for the user to be able to edit the watermark again after the watermark is added, which is also a prerequisite for restoring the original image and modifying the watermark.

[0050] In a possible implementation, the image processing method further includes:

[0051] In response to a fourth operation for adjusting the first watermark information, an adjusted target image is displayed in the third interface, the adjusted target image does not include the watermark information, or the adjusted target image includes the second watermark information.

[0052] In a possible implementation, in response to the fourth operation for adjusting the first watermark information, displaying the adjusted target image in the third interface includes:

[0053] In response to the fourth operation, the first application decapsulates the target image to obtain an intermediate image and image information of the intermediate image;

[0054] The first application obtains a cropped second original image according to the intermediate image;

[0055] The first application restores the second original image according to the hidden image included in the image information and the cropped second original image;

[0056] The first application obtains an adjusted target image according to the second original image;

[0057] The adjusted target image is displayed in the third interface.

[0058] In this implementation, it is explained how to edit the photo watermark again after taking a photo and adding a watermark. The user adjusts the target image again, and there are many ways to adjust it, for example, it can include removing the watermark information in the target image, or modifying the watermark information in the target image. It can be understood that no matter what adjustment method is used, it is generally necessary to first complete the image information that was originally lost due to the addition of the watermark. Combined with the implementation in the previous article, the hidden image included in the image information contains the part of the image information lost due to the watermark, so the hidden image can be used to restore the original image first, and then adjust the new watermark to obtain the adjusted target image. Through this implementation, not only the watermark of the photo is edited again, but it is also well combined with the method of adding watermarks when taking photos. It can be used as a complete set of functions of the first application to give users the best experience of taking photos and adding watermarks using terminal devices.

[0059] In a possible implementation manner, the first application obtains an adjusted target image according to the second original image, including:

[0060] The first application applies for a third cache;

[0061] The first application adds the second original image to the third buffer to obtain an adjusted target image, wherein the adjusted target image does not include watermark information; or,

[0062] The first application adds the second original image and the second watermark image to the third buffer to obtain an adjusted target image, wherein the adjusted target image includes second watermark information corresponding to the second watermark image.

[0063] In a possible implementation, when the target image is a thumbnail image, after the target image is displayed in the second interface, the image processing method further includes:

[0064] An output image corresponding to the thumbnail image is displayed in the second interface, the output image includes the first watermark information, and the size of the output image in the second interface is a third size, which is larger than the first size.

[0065] In this implementation, it is explained that when the target image is displayed in the second interface of the first application, if the thumbnail image is currently displayed, then after the output image is generated, the thumbnail image will be replaced with the output image to continue to be displayed in the second interface. The reason for doing this is that although a watermark is also added to the thumbnail image in this implementation, there is no obvious difference in image content from the output image, but because the output image has undergone more complete image processing processes, there may still be slight differences in some details. Therefore, after allowing the user to quickly view the target image through the thumbnail image, it is replaced with the generated output image. On the one hand, it ensures the consistency of the image content seen by the user before and after, and on the other hand, it allows the user to see the target image with the best image quality in a timely manner as much as possible without additional operations or switching interfaces.

[0066] In a second aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0067] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0068] In a fourth aspect, an embodiment of the present application provides a computer program product comprising a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0069] In a fifth aspect, the present application provides a chip or a chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0070] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0071] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of a photo with a watermark provided in an embodiment of the present application;

[0073] Figure 2 Schematic diagram of adding a watermark when taking a photo provided by an embodiment of the present application Figure 1 ;

[0074] Figure 3 Schematic diagram of the interface for adding a watermark when taking a photo provided by an embodiment of the present application Figure 1 ;

[0075] Figure 4 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application;

[0076] Figure 5 A schematic diagram of the software structure of a terminal device provided in an embodiment of the present application;

[0077] Figure 6 Schematic diagram of the interface for adding a watermark when taking a photo provided by an embodiment of the present application Figure 2 ;

[0078] Figure 7 Schematic diagram of the process of adding a watermark when taking a photo provided by the embodiment of the present application Figure 1 ;

[0079] Figure 8 Schematic diagram of the process of adding a watermark when taking a photo provided by the embodiment of the present application Figure 2 ;

[0080] Fig. 9 Schematic diagram of adding a watermark when taking a photo provided by an embodiment of the present application Figure 2 ;

[0081] Fig.10 A schematic diagram of the implementation of cloud-based watermark selection and loading provided in an embodiment of the present application;

[0082] Fig.11 Schematic diagram of the implementation of determining the target size provided in the embodiment of the present application Figure 1 ;

[0083] Fig.12 Schematic diagram of the implementation of determining the target size provided in the embodiment of the present application Figure 2 ;

[0084] Fig.13 A schematic diagram of an implementation of generating a watermark image provided in an embodiment of the present application;

[0085] Fig.14 A schematic diagram of implementing cropping of an original image provided in an embodiment of the present application;

[0086] Fig.15 A schematic diagram of an implementation of generating a thumbnail image provided in an embodiment of the present application;

[0087] Fig.16 Schematic diagram of the implementation of watermark re-editing provided in the embodiment of the present application Figure 1 ;

[0088] Fig.17 Schematic diagram of the implementation of watermark re-editing provided in the embodiment of the present application Figure 2 ;

[0089] Fig.18 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0090] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0091] 1. Watermark

[0092] The watermark referred to in the embodiments of the present application is different from the traditional watermark on paper, but refers to a digital watermark in the modern sense, which can be embedded in a digital image, video or audio. Digital watermarks can also be divided into visible watermarks and invisible watermarks.

[0093] In particular, watermarks are added to photos, which can usually be in the form of text, patterns, logos or a combination, and are directly displayed on the surface of the original image. The content of the watermark can include date, location and custom information. Adding watermarks to photos can bring a variety of beneficial effects, including special visual effects, increasing the interest of photos and reflecting commemorative significance, and in some cases can also protect copyright.

[0094] 2. UBWC

[0095] Unified bit-wise compression (UBWC) is an algorithm designed for compressing image data, especially image data in memory. UBWC compresses image data bit by bit, which can usually be used to reduce the storage space required for image data in memory and reduce bandwidth usage.

[0096] Moreover, UBWC is a private compression format, and its specific implementation details are not disclosed. UBWC is usually used in mobile devices, such as smartphones and tablets, to run some graphics tasks under limited power consumption and memory bandwidth conditions.

[0097] 3. YUV

[0098] YUV is a common color encoding format, mainly used in video and image processing. The three components of YUV correspond to different color information. Among them, Y represents luminance or grayscale value, which determines the brightness of the image. U and V represent two chrominance difference signals, which carry information related to color. U usually represents blue-difference and V represents red-difference.

[0099] An important feature of the YUV format is that it can be easily converted from the RGB color space, and because the human eye is more sensitive to brightness than chrominance, the chrominance information can be downsampled during data compression, thereby saving storage space without significantly affecting visual quality.

[0100] 4. JPEG

[0101] The Joint Photographic Experts Group (JPEG) is a widely used image file format. The JPEG format usually uses lossy compression technology to reduce file size. It can significantly reduce file size while maintaining good image quality and is suitable as a data format for saving images.

[0102] Users can adjust the compression level as needed. A lower compression level means higher image quality, but a larger file size; a higher compression level means the opposite. In addition, JPEG can also support lossless compression to meet the needs of some specific scenarios.

[0103] 5. Thumbnail images

[0104] The thumbnail image in the embodiment of the present application refers to an image in the process of taking photos at the terminal. For example, in the preview interface of the camera application of the mobile phone, after the user clicks the camera button, a small-sized image will appear in a place in the preview interface. Clicking the small-sized image can also browse the corresponding large-sized image. Such small-sized images can be called thumbnail images. And it can be understood that in some scenarios, small-sized thumbnail images can also be displayed in large size. The "small size" introduced here is not because the image size of the thumbnail image itself is small, but that the thumbnail image will be displayed in a small size in the preview interface of the camera. In fact, the image size of the thumbnail image is consistent with or similar to the image size of the final output image.

[0105] Thumbnail images can be generated in real time very soon after taking a photo, usually without complex image processing, and the size of the thumbnail image can be consistent with the final photo. Thumbnail images can facilitate users to quickly browse, or to confirm the content of the photo when the final photo has not yet been generated and saved.

[0106] 6. Chip Capability Open Node

[0107] Different chips can have different capabilities. However, as a kind of hardware, it is generally difficult for upper-layer applications to directly interact with the chip and call the chip's functions. Therefore, in order to solve this problem, a corresponding chip capability exposure node can be set up. This node can be regarded as a functional module, serving as an intermediate bridge connecting upper-layer applications and hardware.

[0108] For example, in the Android system, by adding a chip capability exposure node at the bottom of the system, the node interacts with the corresponding chip on the one hand, and provides its own functions or interfaces to upper-layer applications on the other hand. In this way, upper-layer applications can use the relevant functions of the chip relatively conveniently through the chip capability exposure node.

[0109] 7. APK

[0110] In the Android ecosystem, Android application package (APK) is a file format that is an installation package for Android applications, or the standard way to install and distribute Android applications. An APK file contains all the necessary components of an application, such as code, resource files, and native library files, which are required for the application to be installed and run. In some cases, people in the relevant technical field will use APK to refer to an application.

[0111] 8. Camera Service

[0112] The Camera Service is the core part of the Android framework layer related to camera functions. The Camera Service is a system service that manages all camera hardware instances and provides a unified interface for applications to access these hardware. The Camera Service is responsible for coordinating multiple applications' access requests to the camera to ensure that only one application can use the camera at any time. In addition, it also handles the allocation and release of camera resources.

[0113] 9. AIDL

[0114] Android interface definition language (AIDL) is a mechanism for implementing inter-process communication (IPC) in the Android system. It allows data exchange and method calls between different Android applications or different components of the same application, even if these components run in different processes. AIDL is an important tool for implementing inter-process communication in the Android system, especially when services or data need to be shared across applications.

[0115] 10. HAL

[0116] In the Android system, the hardware abstraction layer (HAL) is a software interface between the operating system and the hardware device. The main purpose of HAL is to separate the hardware driver from the core code of the operating system. The benefits of doing so include improving the portability of the system, simplifying the complexity of the kernel, and enhancing security. HAL usually exists in the form of modules, each module corresponds to a type of hardware device or function. HAL provides a unified interface so that the upper-level software does not have to worry about the specific hardware implementation details.

[0117] 11. Image processing engine

[0118] The image processing engine refers to a system used to perform various image processing on image data, and its functions may include: image format conversion, filter processing, resolution adjustment, resolution adjustment, face positioning, etc. The image processing system can be integrated into various electronic devices, such as mobile phones, and in order to improve the efficiency of image processing, the image processing engine can be set in the HAL.

[0119] 12. Pipline

[0120] In the embodiments of the present application, the pipeline (Pipline) can also be referred to as a data pipeline, pipeline processing flow, etc., which refers to a process of task execution, which is generally divided into multiple processing nodes in the process. After one node is executed, the next node is executed. The Pipline can have branches or be parallel. For example, the process of image processing a photo can be called a Pipline, which can include loading image data, format conversion, adding watermarks, and saving data. In addition, the Pipline is not fixed in some scenarios and can be adjusted according to specific circumstances.

[0121] 13. ODM partition

[0122] In Android devices, the original design manufacturer (ODM) partition is a special partition that is mainly used to store manufacturer-related HAL and other device-specific software components. By separating the device-specific HAL from other system components, system updates are simpler and faster, and compatibility issues are better avoided.

[0123] 14. Buffer

[0124] In the computer field, buffer usually refers to a section of memory used to temporarily store data. When data is moved from one location to another, the cache can be used to temporarily hold the data. One purpose of the cache is to coordinate data transfer between devices or systems with different processing speeds to improve overall performance.

[0125] In addition, cache is often used to refer to data stored in a piece of memory. In different scenarios, it is necessary to understand whether cache refers to the memory space or the data in it based on the context.

[0126] 15. Other terms

[0127] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0128] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0129] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0130] 16. Electronic equipment

[0131] The electronic devices of the embodiments of the present application may include handheld devices with camera functions, vehicle-mounted devices, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, personal digital assistants (PDA), handheld devices with camera functions, computing devices, terminal devices in 5G networks, or terminal devices in future evolved public land mobile networks (PLMN), etc., and the embodiments of the present application are not limited to this.

[0132] In addition, in the embodiments of the present application, the electronic device may also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0133] The electronic device in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0134] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an IOS operating system, or a Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0135] Based on the above introduction, the relevant technologies involved in this application are further introduced in detail below.

[0136] Nowadays, more and more users choose to add watermarks to photos when taking photos with their mobile phones. Figure 1 To illustrate with an example, Figure 1 is a schematic diagram of a photo with a watermark added.

[0137] exist Figure 1 (a) in the figure is the original photo 100, and its content is: in the middle of the picture is a simple standing figure, and there is a sun in the upper right corner. It can be understood that watermarks can be of various types. Figure 1 (b) and (c) in the figure show two different types of watermarks, respectively.

[0138] First Look Figure 1In (b), in the upper left corner of the photo is a watermark 101, specifically a date watermark, which can be used to record the time when the photo was taken, and the watermark 101 can be regarded as directly covering the photo 100, that is, the size of the original photo 100 is the same as the image size after the watermark 101 is added.

[0139] Let's look at it again Figure 1 In (c), there is another watermark 102 on the bottom border of the photo. The watermark 102 is a pattern watermark composed of a plurality of flower pattern sub-patterns arranged repeatedly. Regarding the size of the watermark 102, the length of the watermark 102 along the short side of the original photo 100 is less than or equal to the length of the short side of the original photo 100, and the length of the watermark 102 along the long side of the original photo 100 is less than the length of the long side of the original photo 100. Figure 1 (b), Figure 1 The watermark 102 in (c) expands the size of the original photo 100, making the original photo 100 longer in the direction of the original long side or height. At the same time, the watermark 102 also blocks part of the image content in the original photo 100, for example, Figure 1 In (c), the watermark 102 blocks part of the lower limbs of the person. Figure 1 The pattern border watermark in (c) can bring a unique visual effect to the original photo 100, and can also express the user's mood and state when taking the photo.

[0140] In the actual implementation process, the specific type of watermark added to the photo can be arbitrarily selected according to actual needs. For example, the watermark type can also include device information watermark, location watermark, and photo frame watermark, etc. This embodiment does not limit this.

[0141] Considering why users are happy to add watermarks, it can include the following aspects:

[0142] 1. Enhance aesthetics. Creatively designed watermarks can be used as decorative elements to increase the artistry and appreciation of photos. 2. Personalized expression. Users can express their emotions or ideas by adding personalized text or patterns. 3. Record information. Watermarks can also contain information such as the shooting date, location, and weather, which helps users recall the background of the shooting. 4. Copyright protection. Watermarks can serve as a copyright statement to indicate the ownership of photos and prevent others from using pictures without authorization. 5. Brand promotion. Adding brand logos or names to photos can increase brand awareness, especially when shared on social media.

[0143] Moreover, when users add watermarks to photos, they usually prefer to have the watermark added directly to the image after taking the photo, rather than post-editing the image to add the watermark. Adding watermarks simultaneously when taking photos can bring many benefits, including: instant feedback, users can see the watermark effect while taking photos, which is convenient for adjusting the composition to obtain the best visual effect; improved efficiency, reducing the time of post-editing, and getting the results immediately after taking the photo, which is very practical for a fast-paced lifestyle; convenient to try, real-time preview allows users to try different watermark styles during the shooting process, inspiring more creative inspiration; consistency guarantee, if the photos are taken for a specific event or project, real-time uniform watermarking can ensure the consistency of all pictures.

[0144] For the function of adding watermarks when taking photos with a mobile phone, combined with Figure 2 To explain, Figure 2 Implementation of adding watermarks when taking photos Figure 1 , which is implemented as follows:

[0145] Step 1: The camera management module in the camera application sends a photo request to the HAL.

[0146] like Figure 2 As shown, step 1 includes step 1-a and step 1-b. First, for step 1-a, after the user adjusts the picture to be photographed in the preview interface of the mobile phone, he presses the photo button. At this time, the photo management module in the camera application sends a photo request to the camera service in the framework layer. The photo request may carry a tag containing information enabling the watermark algorithm.

[0147] The enable information can also be called an enable signal. Generally speaking, the enable information is a "permit" signal. The enable information is generally used to convey information such as whether to enable a certain function or whether to allow a certain task to be executed. The enable information usually appears in the scene of interaction with hardware to ensure that the hardware will enable the corresponding function only after receiving the enable message, so as to avoid unnecessary accidents.

[0148] The tag here refers to a special parameter type in Android. Specifically, a tag can be a string or an object, which can contain multiple different types of data. When passing parameters in the application method, part of the data or information can be encapsulated in the tag, so that data or information can be passed without adding additional method parameters, which is convenient for opening and maintenance.

[0149] In this embodiment, the Tag of the watermark algorithm information can be used to instruct the HAL to enable the watermarking function, where the watermarking function is to add a watermark to the image captured by the photo. That is, in this embodiment, the Tag of the watermark algorithm information is used to inform the HAL that a watermark needs to be added to the image captured by the current photo request, so that the HAL layer can add a watermark based on the watermark algorithm in the subsequent image processing process.

[0150] Then for step 1-b, after the camera service in the framework layer receives the photo request in step 1-a, it further transmits the photo request to HAL through AIDL.

[0151] Step 2: After receiving the photo request from the camera application, the HAL returns the response information of the photo request to the camera application.

[0152] Step 2 specifically includes step 2-a and step 2-b. First, for step 2-a, HAL transmits the response information of the photo request to the photo module of the framework layer through AIDL. Among them, the response information of the photo request includes, for example, lens information and image name. The lens information may include, for example, the focal length, depth of field and other information corresponding to the lens currently performing image acquisition. This embodiment does not limit the specific content contained in the lens information. And in the process of performing image acquisition by the terminal device, for example, a name corresponding to the image can be generated for each captured image to distinguish multiple images according to the image name, and the image name is included in the response information, which can be used to associate and determine the image in the subsequent image processing process.

[0153] Then for step 2-b, after the camera module of the framework layer receives the response information of the camera request from the HAL, it further transmits the response information of the camera request to the camera management module of the application layer.

[0154] Step 3: The camera application packages the relevant information required for taking photos and adding watermarks according to the response information of the received photo request, sends a request for adding watermarks to the HAL again, and carries the packaged information in the request.

[0155] Step 3 also includes step 3-a and step 3-b. For step 3-a, the camera application of the application layer sends the above-mentioned request for adding a watermark to the camera module of the framework layer. The packaged information carried in the request may include, for example, the content required to fill or complete the watermark.

[0156] For step 3-b, the request for adding watermark is also transmitted to HAL through the framework layer and AIDL.

[0157] Step 4: After completing the interaction and preparation in the previous stage, the image processing engine in HAL will perform a series of image processing including adding watermarks on the image data captured by the camera according to the constructed pipeline.

[0158] exist Figure 2 The image processing pipeline is shown in Figure 2 It can be determined that the Pipline may include nodes such as algorithm 1, algorithm 2, stylization, filter, and watermarking. In one implementation, the image processing engine may process the image in sequence according to each node included in the Pipline, and finally output an image that can be displayed in a terminal device. In the actual implementation process, the nodes specifically included in the image processing Pipline may be arbitrarily set according to actual needs, and this embodiment does not limit this.

[0159] exist Figure 2 In the example, the Pipline includes a node for adding watermarks. When executing the algorithm for adding watermarks, the watermark material resources required for adding watermarks can be parsed and obtained from the ODM partition. Then, the watermarking process can be completed by combining the photographed image data, watermark material resources, and related information issued by the camera application.

[0160] During this process, thumbnail images are generated and displayed in parallel. For example, in the preview interface, the thumbnail image can generally be set at the lower left corner.

[0161] Step 5: Convert the format of the watermarked image data, for example, convert the image data from the original YUV format to the JPEG format.

[0162] Step 6: Return the image after JPEG encoding conversion to the camera application and save it to the album.

[0163] However, there are some problems with the above-mentioned implementation method of adding watermarks when taking photos with a mobile phone, which may lead to a poor user experience. The following is an explanation based on the corresponding scenarios.

[0164] Scenario 1: The content of the thumbnail image is inconsistent with the watermarked output image.

[0165] The output image here refers to the image that has completed the image processing process, been encoded, compressed and saved. For example, an image that has been watermarked and converted to JPEG format.

[0166] When taking photos, today's mobile phones, in addition to capturing and generating the final photo, generally generate thumbnail images in parallel and quickly. Because thumbnail images simplify the image processing process and use some special data formats, they can be completed faster than the final photo. Thumbnail images allow users to quickly view the general content of the photo without having to wait for the photo to be processed and saved before viewing it, allowing users to quickly determine whether the current photo meets expectations. It can be said that thumbnail images have become an indispensable part of the mobile phone application photography process.

[0167] refer to Figure 3 The content shown in the example is used to illustrate the scenario 1. Figure 3 This is the interface for adding watermarks when taking photos Figure 1 .like Figure 3 As shown in (a) of FIG. 3 , when a user uses a mobile phone 300 to take a photo, the user can adjust the photo image in the preview interface 301 of the camera application. At the same time, the user can also select the watermark he wants to add, and the corresponding terminal device can also display the watermark 304 pre-selected by the user in the preview interface 301, so that the user can determine the watermark effect. Figure 3 In the example of , the watermark 304 is located in the upper left corner of the screen and is a date type watermark. After the user presses the photo button 302, the camera application will start to perform a series of photo process related processing and generate a thumbnail image at almost the same time.

[0168] After generating the thumbnail image, you can refer to Figure 3 As shown in (b) of FIG. 301 , the thumbnail image is displayed in a smaller size at the position indicated by 303 in the lower left corner of the preview interface 301, corresponding to the thumbnail image control 303 in the lower left corner of the interface. The small-sized image you see at this time is obtained by reducing the thumbnail image, and it is not easy to see the details. Therefore, you can further click on the thumbnail image control 303 to jump to Figure 3 (c) shows the interface for viewing large-size thumbnail images.

[0169] Figure 3 (c) in the figure shows the situation that after clicking the thumbnail image control 303 in the preview interface 301, the corresponding large-size thumbnail image is jumped to the image viewing interface 305. Obviously, there is no watermark 304 mentioned above on the image displayed in the image viewing interface 305. This is actually because the thumbnail image is not watermarked during the rapid generation process, and the output image that needs to undergo a series of image processing and watermarking has not been generated. At this time, the image seen is a thumbnail rather than the real output image with the watermark added, so the above situation occurs.

[0170] In appeared Figure 3After the situation shown in (c), if the user stays on the image viewing interface 305 for a period of time, the following will appear: Figure 3 The picture shown in (d) in the figure. Figure 3 In (d), the watermark 304 originally present in the preview interface 301 is Figure 3 The watermark 304 that disappeared in the album interface 305 in (c) reappears in the upper left corner of the image. It is understandable that this happens because after the user waits for a period of time, the output image with the watermark added has been processed and saved. At this time, if the system detects that the user is viewing a thumbnail image, the output image will be used to replace the thumbnail image, so the watermark 304 reappears.

[0171] Understandably, when a user presets a watermark and clicks to take a photo, and then wants to quickly view the photo by clicking on the thumbnail, they may find that the photo does not contain the watermark they expected. At this time, the user may be confused or have questions, wondering if there is something wrong with their operation or with the camera application. The user may then immediately exit the album interface for viewing large photos, delete the photos they have taken, and take photos again. Even if the user chooses to stay for a while after viewing the thumbnail image without the watermark, and then sees the output image with the watermark added and saved, this inconsistent and sudden change, as well as the waiting process, will obviously seriously affect the user experience.

[0172] Combined with the content described in Scenario 1, let's look at the implementation method of adding watermarks when taking photos introduced in the previous article. This implementation method does not add watermarks to the thumbnail images, which makes the thumbnail images and the final photos inconsistent in content, and there will be similar Figure 3 The watermark that is displayed appears suddenly from nothing. The reason for this problem is that the data format of the thumbnail image itself is UBWC. Referring to the introduction of UBWC in the previous article, it can be seen that UBWC is a private, or closed-source data format, which is difficult to edit or modify in a complex way, and it is also impossible to add a watermark directly to the thumbnail image. If the thumbnail image format is converted first, for example, the thumbnail image is converted from UBWC to YUV format, and then the watermark is added by editing the YUV format data, the watermark can indeed be added. However, this kind of overall conversion and processing of image data generally requires a lot of computing power support, which will not only increase the load of the processor or SOC, but also significantly increase the time it takes to complete the output of the thumbnail image, which in turn reduces the user experience.

[0173] Scenario 2: When setting a watermark, the user hopes to find a suitable watermark template.

[0174] With the enrichment of users' daily lives, changes and increases in work needs, and the increasing power of smartphones, more and more people are starting to use their phones to record the beautiful moments in their lives, and users have more ideas about what kind of watermarks to add. In order to make photos more personalized and interesting, users hope to be able to add unique watermarks to their photos. From the beginning, users only added some simple text and graphics, to now users may want to add watermarks with specific styles and content, and users also have a greater demand for the diversification of watermarks. Therefore, providing a variety of watermark templates has become an important part of improving user experience.

[0175] In digital watermarking technology, a watermark template usually refers to the original design or pattern used to generate a watermark. This template can be an image, text, logo or other form of identification information. The watermark template can also be understood as a sample that restricts and regulates the format and content of the watermark. The watermark fill information is applied to the watermark template to supplement and complete the information required in the watermark template. For example, for a watermark template involving a date, the date can be dynamically determined based on the time when the watermark is added, and the date information here can be called watermark fill information. It can be understood that watermark information refers to the information reflected by the watermark template or the complete watermark to which the watermark fill information has been added. The information can include the watermark pattern, texture, color, text and other related content.

[0176] Here are some different types of watermarks or watermark templates: date and location watermarks, which automatically or manually input the time and location of the photo for easy recall in the future; emotional expression watermarks, which contain various emoticons, mood tags, etc., which can intuitively express the photographer's current emotional state or feelings about the scene; holiday theme watermarks, which are watermarks designed for specific festivals, such as the Spring Festival, Dragon Boat Festival or Mid-Autumn Festival, which increase the festive atmosphere and make the photos more appropriate and commemorative; personalized signature watermarks, which allow users to customize text content, such as personal signatures, famous quotes, etc., which can reflect personal style and protect the copyright of works.

[0177] By providing a more diverse selection of watermark templates, it can not only meet the personalized needs of users, but also stimulate their creativity and make the shooting process more interesting. In addition, this also provides a differentiated competitive point for mobile phone manufacturers, helping to enhance brand image and user satisfaction.

[0178] On the contrary, when a user imagines an expected watermark in a certain situation, but when browsing the watermark template resources of the camera application, he finds that there are only a few fixed templates to choose from and they cannot be customized. This will bring a bad user experience to the user.

[0179] In the above-mentioned implementation method of adding watermarks when taking pictures, the specific process of adding watermarks is performed by HAL, and the watermark material resources are also stored in the ODM partition, that is, the fixed type of watermark material resources are stored in the designated partition of the terminal device, and their style and quantity are very limited, which is difficult to meet the needs of users who pursue diversified watermarks. In addition, there will be a relatively high degree of coupling between the application layer and HAL of the camera application in this implementation method, which increases the difficulty of maintenance. For example, each time a watermark is added, what kind of watermark needs to be used, or what kind of adjustment needs to be made, the application layer needs to be synchronized with HAL. For another example, when adding a new watermark template, the application layer also needs to be synchronized with HAL and stored in ODM.

[0180] Scenario 3: After adding a watermark when taking a photo and saving the photo, the user wants to further modify the watermark.

[0181] In some cases, after taking a photo and adding a watermark, the user may think that the watermark is inappropriate and want to remove the watermark and restore the photo, or modify the watermark. It is understandable that the function of re-editing a photo after adding a watermark is a function that can effectively improve the user experience.

[0182] For example, a user uses a preset watermark when taking a photo, and after taking the photo, a photo with the watermark is obtained. However, the user may think that the watermark in the photo blocks some important information, such as blocking the face of a person, and the user hopes to remove the watermark and restore the original photo.

[0183] For example, when viewing a photo taken with a watermark, the user believes that the watermark is inappropriate. For example, the original watermark contains the user's full name, and the user does not want such photos to be shared later, so the user wants to change the full name in the original watermark to a pseudonym.

[0184] The examples listed above illustrate the importance and necessity of being able to edit watermarks again for photos with watermarks.

[0185] Similarly, let's look at the above-mentioned implementation method of adding watermarks. This implementation method does not take into account the function of re-editing the watermark of the watermarked photo. After the photographed image data is converted into JPEG format and saved, the added watermark cannot be removed to restore the original image, nor can the watermark or the information in the watermark be modified. Among them, the original image refers to the image without the watermark added after taking the photo, which does not contain any watermark information.

[0186] The above scenarios all involve a series of problems and corresponding needs in the process of taking photos and adding watermarks, as well as after taking photos. Before taking photos, users hope to be able to select a watermark template that they think is appropriate from a rich resource of watermark templates, and after taking photos and generating thumbnail images, they can quickly view images that are consistent with the final watermarked photo content, and after saving photos, they can also edit the watermarks in photos. The existing technical solutions for adding watermarks when taking photos cannot solve the problems in these scenarios, and it is difficult to meet the corresponding needs of users and further improve the user experience. Therefore, a new technical solution is needed to meet the needs of users in some of the above scenarios.

[0187] Based on this, the embodiment of the present application proposes the following technical concept: by setting a chip capability opening node in the HAL, the application layer can indirectly interact with the corresponding chip through the chip capability opening node. Since the chip has some capabilities including GPU acceleration, the original UBWC format of the thumbnail image can be quickly converted into a format that can add watermarks, such as the YUV format, so that a watermark can be added to the thumbnail image, which ensures the speed of thumbnail generation and keeps the content of the thumbnail consistent with the final output image.

[0188] Furthermore, the specific process of adding watermarks is transferred from the original HAL to the application layer, and the watermark material resources are no longer stored in the ODM partition, but can be obtained from the cloud through the network. In this way, on the one hand, the application layer and HAL of the original watermarking function when taking pictures are decoupled, and the subsequent application layer adjusts the watermarking algorithm or watermark template, and HAL can remain unchanged. On the other hand, because watermark templates can be obtained from the cloud, there are a large number of rich watermark templates to choose from, and it is no longer limited to a limited number of watermark templates.

[0189] In addition, the embodiment of the present application also encodes some intermediate data in the process of adding watermarks into the image data in some way. That is to say, the watermark of the photo can be edited again or the watermark can be removed and the original image can be restored by parsing these intermediate data in the image data.

[0190] The technical solution provided in this application can be applied to terminal devices. The following first briefly introduces the terminal devices.

[0191] For example, Figure 4 A schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application is shown.

[0192] Figure 4A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. The terminal device 400 may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc. The sensor module 480 may include a pressure sensor 480A, a gyroscope sensor 480B, an air pressure sensor 480C, a magnetic sensor 480D, an acceleration sensor 480E, a distance sensor 480F, a proximity light sensor 480G, a fingerprint sensor 480H, a temperature sensor 480J, a touch sensor 480K, an ambient light sensor 480L and a bone conduction sensor 480M, etc.

[0193] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 400. In other embodiments of the present application, the terminal device 400 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0194] The processor 410 may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0195] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0196] The processor 410 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. The memory may store instructions or data that the processor 410 has just used or cyclically used. If the processor 410 needs to use the instruction or data again, it may be called from the memory. This avoids repeated access, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0197] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is a schematic illustration and does not constitute a structural limitation on the terminal device 400. In other embodiments of the present application, the terminal device 400 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0198] The terminal device 400 implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0199] The display screen 494 is used to display images, videos, etc. The display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed or a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 400 may include 1 or N display screens 494, where N is a positive integer greater than 1.

[0200] The terminal device 400 can realize the shooting function through ISP, camera 493, video codec, GPU, display screen 494 and application processor.

[0201] ISP is used to process the data fed back by camera 493. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 493.

[0202] The camera 493 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 400 may include 1 or N cameras 493, where N is a positive integer greater than 1.

[0203] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 400 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0204] The internal memory 421 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 421 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 400 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 410 executes various functional applications and data processing of the terminal device 400 by running instructions stored in the internal memory 421 and / or instructions stored in a memory provided in the processor.

[0205] Chip 492 is used to implement or optimize one or more tasks in terminal device 400 according to requirements. Chip 492 itself can have multiple features and functions. For example, chip 492 can have the computing and control functions of processor 410, the storage function of internal memory 421, or the accelerated image processing and parallel computing functions of GPU. Based on the characteristics and capabilities of chip 492, other applications with related requirements can be optimized and upgraded with the help of chip 492. For example, camera applications that require image processing speed can accelerate the processing of image data through chip 492 to optimize the user experience.

[0206] The software system of the terminal device 400 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 400.

[0207] For example, Figure 5 A schematic diagram of the software structure of a terminal device provided in an embodiment of the present application. Figure 5 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate through interfaces. In some embodiments, the system may include an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, a hardware abstraction layer (HAL), and a kernel layer. It should be noted that the embodiment of the present application is illustrated by taking the Android system as an example. In other operating systems (such as Hongmeng system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those of the embodiment of the present application, the solution of the present application can also be implemented.

[0208] Among them, the application layer can include a series of application packages.

[0209] like Figure 5 As shown, the application layer can also be referred to as the application layer, which can include applications such as camera, gallery, calendar, call, map, navigation, wireless local area network (WLAN), Bluetooth, music, video, short message, lock screen application, setting application, etc. Of course, the application layer can also include other application packages, such as payment application, shopping application, banking application, social application and other third-party applications, which are not limited in this application.

[0210] The application framework layer may also be referred to as the framework layer for short. The framework layer provides an application programming interface (API) and a programming framework for the application programs in the application layer. The application framework layer includes some predefined functions or functions, such as a camera service, an activity manager, a window manager, a content provider, a view system, a resource manager, and a notification manager, etc., and the embodiments of the present application do not impose any restrictions on this.

[0211] The HAL layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware.

[0212] The HAL layer may include an image processing engine, Wi-Fi HAL, audio HAL, the camera HAL Server unit of the HAL layer, and a software code library, etc. Among them, the image processing engine can be used to build a pipeline for the image data captured by the camera, so as to perform a series of image processing on the image data through the pipeline, such as format conversion, adding watermarks, color adjustment, etc.

[0213] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver and charging driver. Among them, the camera driver is the driver layer of the camera device, which is mainly responsible for the interaction with the hardware module. In addition, the kernel layer can also include a newly added chip driver to realize the control and interaction of a specific chip.

[0214] The following, in conjunction with the accompanying drawings, describes in detail the technical solutions of the embodiments of the present application and how the technical solutions of the embodiments of the present application solve the above-mentioned technical problems with specific embodiments. The following specific embodiments can be implemented independently or in combination with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0215] First combine Figure 6 and Figure 7 An image processing method for adding a watermark when taking a photo provided by an embodiment of the present application is described. Figure 6 Schematic diagram of the interface for adding a watermark when taking a photo provided by an embodiment of the present application Figure 2 , Figure 7 Schematic diagram of the process of adding a watermark when taking a photo provided by the embodiment of the present application Figure 1 .

[0216] In order to facilitate the understanding of the relevant methods in the following text, some of the terms involved are defined and explained here:

[0217] Watermark image. The watermark image in the embodiment of the present application can be an image obtained by combining a watermark template and watermark filling information, which contains corresponding watermark information. The watermark image can be a simple pattern or text, etc., or a combination of the above.

[0218] The intermediate image in the embodiment of the present application refers to an image that has not yet completed the image processing process and has not yet been encoded and compressed in the image data format. For example, a YUV format image data that has not yet been watermarked, or has been watermarked but has not yet undergone subsequent image processing can be called an intermediate image.

[0219] The following are Figure 7 The image processing method flow shown in is explained.

[0220] S701. In response to a first operation for triggering a photo-taking operation in a first interface, a thumbnail image is displayed in the first interface, the thumbnail image includes first watermark information, and the size of the thumbnail image in the first interface is a first size.

[0221] In the embodiment of the present application, the first interface can be understood as a preview interface of the camera application when taking pictures, and the first watermark information can be a watermark image in the thumbnail image.

[0222] You can refer to Figure 6 In (a), the user opens the first application on the terminal device 300 and is in the first interface 301 of the first application, wherein the first application may be a camera application, and the first interface 301 may be used as a preview interface for taking photos. The user may adjust the picture to be taken in the first interface 301.

[0223] There is a photo control 302 at the bottom of the first interface. The first operation for triggering photo taking may be, for example, an operation in which the user clicks the photo control 302 after previewing the desired picture in the first interface 301. In the actual implementation process, the first operation may also be, for example, a click operation on the lower volume key, or a voice input operation, etc. This embodiment does not limit the specific implementation of the first operation, as long as the first operation can be output to the terminal device for taking a photo.

[0224] When the user makes the first operation, the first application will perform a series of hardware and software related processing, for example, the camera or camera of the terminal device 300 captures the shot picture, converts the optical signal into image data of the electrical signal, and the relevant software system then establishes a data processing pipeline for the image data, including format conversion, adding watermarks and some image algorithms, etc. In the embodiment of the present application, the thumbnail image data will be quickly format converted in the HAL by using an open node with chip capabilities, and then the first watermark 304 will be added. The added first watermark 304 can be understood as the first watermark information in this embodiment.

[0225] like Figure 6 As shown in (b), in the first interface 301, the thumbnail image obtained after taking a photo will first be displayed in the thumbnail image control 303 in the lower left corner. At this time, the size of the thumbnail image is the first size, and the size of the first size can be equal to the size of the thumbnail image control 303. Since the thumbnail image control 303 is relatively small on the first interface, the thumbnail image is also relatively small and it is not easy to see the details, which is one of the reasons why it is called a thumbnail image.

[0226] S702: In response to a second operation on the thumbnail image, display a target image in a second interface, the target image includes first watermark information, and a size of the target image in the second interface is a second size, which is larger than the first size.

[0227] In the embodiment of the present application, the second interface can be understood as an interface for viewing large-size images to which the user jumps when clicking on a thumbnail image control in the first interface.

[0228] refer to Figure 6 As shown in (b) of FIG. 1 , the second operation mentioned above may, for example, refer to the user clicking on the thumbnail image control 303 at the lower left corner of the first interface. After receiving the instruction corresponding to the second operation, the first application switches from the first interface 301 to the second interface 302. Figure 6 The second interface 305 shown in (c) in FIG. 3 is displayed, and the thumbnail images are displayed in their original sizes in the second interface 305 .

[0229] Combination Figure 6 The content shown in (c) in the figure is displayed in the second interface 305, and the size of the target image in the second interface 305 is a second size, which generally corresponds to the size of the second interface 305 and can occupy most of the screen display area of ​​the terminal device 300. It can be understood that the second size is larger than the first size.

[0230] It should be noted that the target image mentioned in the embodiment of the present application may include two situations. One is that the user immediately clicks the thumbnail image control 303 after taking a photo and then switches to the second interface 305. At this time, since the output image has not been generated, the target image displayed is actually a thumbnail image. The other situation is that the user waits for a certain period of time after taking a photo and then clicks the thumbnail image control 303. At this time, if the output image has been generated, the target image seen in the second interface 305 is actually the output image.

[0231] The target image displayed in the second interface 305 includes the first watermark 304 added when taking the photo, corresponding to the first watermark information. Figure 2 and Figure 3 The image processing method provided by the embodiment of the present application can achieve consistency of the image content displayed to the user in the whole process, that is, the first watermark 304 is also added to the thumbnail image, and is displayed in the second interface 305 of the first application. In this way, consistency in two aspects can be achieved. On the one hand, because watermarks are added to both the thumbnail image and the final output image, the target image seen in the second interface 305 after the user clicks the thumbnail image control 303 always has the corresponding added watermark. On the other hand, in the first interface 301, after the user triggers the photo shooting, for example, a small-sized thumbnail image can be seen at the thumbnail image control 303. Although some image details cannot be seen clearly on the small-sized thumbnail image, it is generally possible to clearly identify whether it has the desired added watermark. That is to say, after adding a watermark to the thumbnail image, the expected added watermark can also be seen on the corresponding small-sized image at the thumbnail image control 303, ensuring the consistency of the image content and avoiding unnecessary confusion for the user.

[0232] S703: When the target image is a thumbnail image, display an output image corresponding to the thumbnail image in the second interface, the output image includes the first watermark information, and the size of the output image in the second interface is a third size, which is larger than the first size.

[0233] It is understandable that, compared with the output image, the thumbnail image needs to go through relatively fewer processing steps during the generation process, so that it can appear in the thumbnail image control 303 almost immediately after the user clicks the photo control 302, while the output image at this time may still be undergoing corresponding image data processing in the pipeline and has not yet been completely generated. Therefore, if the user clicks the thumbnail image control 303 immediately after clicking the photo control 302, then since only the thumbnail image has been generated at this time, what is actually displayed in the second interface 305 is the thumbnail image.

[0234] If the user clicks the photo control 302 and waits for a certain period of time before clicking the thumbnail image control 303, the output image may have been generated at this time. In this way, what is displayed in the second interface 305 is actually the output image, that is, the final saved photo.

[0235] Although the image content of the thumbnail image and the output image is consistent and both contain the same first watermark 304, the output image has undergone more complete image processing processes than the thumbnail image, and there may still be differences in some image details. Therefore, for a better user experience, the application should use the output image for display as much as possible. Therefore, if the target image in the second interface 305 is a thumbnail image, then wait until the output image is generated, and then switch the thumbnail image to the corresponding output image in the second interface 305. Similarly, the first watermark information is also included in the output image.

[0236] Based on the above steps, the image processing method is further summarized below. In the embodiment of the present application, by using the chip capability exposure node set in the HAL, the thumbnail image is quickly converted into a format, and then a watermark can be added, which overcomes the problem of Figure 2 and Figure 3 The corresponding technical solution cannot add watermarks to thumbnail images. After the user triggers the photo shooting in the first interface 301, he will first see that the watermark has been successfully added in the small-sized image corresponding to the thumbnail image control 303, and then click the thumbnail image control 303 to jump to the second interface 305 to see the large-sized target image. Regardless of whether the target image actually corresponds to a thumbnail image or an output image, the image content is always consistent and carries the watermark that the user expects to add. In contrast, the entire process of taking photos and viewing the target image of the user can be made more natural and smooth, avoiding confusion or doubts caused by the user not seeing the added watermark in the small-sized image corresponding to the thumbnail image control 303 or in the target image of the second interface 305. It also avoids the user being surprised when seeing the watermark suddenly appear from nothing in the target image of the second interface 305, and the user does not have to waste time waiting for the watermark to appear in the target image.

[0237] In this way, while maintaining the fast generation of thumbnail images so that users can view the target image immediately after taking a photo, a watermark is also added to the thumbnail image. Therefore, the user will not see a thumbnail image that is significantly different from the output image, thereby enhancing the user's experience of taking photos and adding watermarks using a terminal device.

[0238] Based on the above embodiments, Figures 8 to 15The image processing method for adding a watermark when taking a photo provided by the embodiment of the present application is further described in detail. For example, Figure 8 The device shown in can be understood as Figure 7 A further form of the method flow shown in .

[0239] Here we first briefly describe the various figures involved. Figure 8 Illustration of the process of adding watermarks to photos Figure 2 , Fig. 9 Implementation of adding watermarks when taking photos Figure 2 , Fig.10 Schematic diagram of the implementation of selecting and loading watermarks for the cloud. Fig.11 Implementation diagram for determining target size Figure 1 , Fig.12 Implementation diagram for determining target size Figure 2 , Fig.13 The schematic diagram of the implementation of generating a watermark image is shown in Figure 2. Fig.14 This is a schematic diagram of cropping the original image. Fig.15 Schematic diagram of the implementation of generating thumbnail images.

[0240] in, Fig. 9 Can be seen as Figure 8 A form of corresponding system layering of the steps in can be used to better understand Figure 8 The method flow in Fig. 9 A brief description of the contents shown in the.

[0241] like Fig. 9 As shown, the first application is located in the application layer, including a first interface, a second interface, and a module for adding a watermark. The first application and HAL can interact through the framework layer, wherein the first application can be a camera application or an application with a photo-taking function. In HAL, an image processing engine is included. When taking photos, two parallel pipelines are constructed, namely a foreground processing pipeline and a background processing pipeline. The foreground processing pipeline is used to perform relevant processing on the thumbnail image, and the background processing pipeline is used to perform relevant processing on the output image. In each of the two parallel pipelines, an open node is set, and the open node can interact with the chip capabilities.

[0242] Next, combined with other figures Figure 8 The image processing method flow shown in is explained.

[0243] S801. The first application determines a target watermark template in response to a third operation for selecting a watermark template in the first interface.

[0244] This step corresponds to Fig. 9The interaction process indicated by the label 1 in the application layer is that the first application in the application layer interacts with the cloud server and downloads the watermark template selected by the user. Fig.10 to explain in detail.

[0245] like Fig.10 As shown, in Fig.10 In the first interface 301, the user can select a watermark template. The third operation here refers to the operation in the process of the user selecting and determining the target watermark template. For example, the user can first click the watermark control 1006, and a drop-down box control 1007 will appear. Various watermark templates will be listed in the drop-down box control 1007.

[0246] In this embodiment, after the user clicks a watermark template in the drop-down box control 1007, the terminal device can, for example, download the corresponding watermark resource data from the corresponding watermark resource server 1000, and can also display the corresponding watermark template on the preview interface, that is, the first interface 301. Fig.10 If the user clicks on a watermark template named date, a corresponding date watermark 304 is displayed in the upper left corner of the first interface 301 .

[0247] In some implementations, some watermark templates can be used in a superimposed manner. For example, a user can first click on a watermark template named date, and then click on a watermark template named address, so that a watermark with both the date and the address will appear on the preview interface.

[0248] Different from Figure 2 and Figure 3 In the method of obtaining watermark material resources from the ODM partition when taking photos, the embodiment of the present application sets the main processing process of adding watermarks in the application layer. Therefore, in this embodiment, a variety of watermark templates can be downloaded from the cloud server through the application layer, and then the watermark effect can be directly processed in the application layer, so that the watermark template downloaded from the cloud server can be put into use in real time and quickly. This effectively avoids the problem of not being able to download watermark templates from the cloud in real time in the implementation method of adding watermarks by coupling the HAL layer and the application layer, where the application layer and the HAL layer need to pre-configure a series of collaborative signaling to add a new watermark template, and the related resources of the watermark template need to be added in the ODM partition.

[0249] It is understandable that rich watermark templates are the basis for users to like to use terminal devices to add watermarks when taking photos. Different users have different needs for watermark templates in different situations. By downloading watermark templates from the cloud server, on the one hand, it fully meets the user's demand for watermark template styles, and on the other hand, it can save valuable storage space on the user's terminal device.

[0250] In some implementations, in this embodiment, the HAL can also report a watermark capability enabling message to the first application. The watermark capability enabling message here can be, for example, a message about whether the image processing engine in the HAL supports adding a chip capability open node, and the chip capability open node can be used to add a watermark function. For example, after determining that the HAL can activate the chip capability open node, the first application will subsequently perform the image processing steps in the embodiment of the present application when taking pictures and adding them for use. Exemplarily, the above-mentioned operation of reporting the watermark capability enabling message can be performed before the user selects a watermark template in the first application, or after the terminal device is turned on.

[0251] S802: The first application determines a preset image size, a watermark size, and a target size according to an image resolution selected by a user.

[0252] The image resolution mentioned here can be understood as the aspect ratio of the image, such as 4:3, 16:9 or 1:1, and can also be set to horizontal or vertical. It can also be the number of pixels in the width and height of the image, such as 1920×1080. Generally, the former, that is, the aspect ratio of the image, is more likely to be used in terminal devices, and then the number of pixels in the width and height of the image is obtained by combining the aspect ratio of the image and the hardware configuration of the terminal device.

[0253] In the embodiment of the present application, there is no specific restriction on the timing when the user selects the image resolution.

[0254] In some implementations, the user may select the image resolution before selecting the watermark template, that is, before S801. Similarly, S802 will participate in the subsequent calculation process with the image resolution selected by the user most recently.

[0255] In some implementations, the user may not specifically select an image resolution, in which case S802 will read a default image resolution and execute subsequent method flows according to the image resolution.

[0256] For example, reference may be made to Fig.11 The following content will explain the detailed process of this step. Fig.11 In (a), for example, the user can click on the control named "More" at the bottom right of the first interface 301, and after clicking, a column of vertical options will be expanded. Among the multiple options in the column, there is a control 1108 displayed as 4:3, which can be, for example, an image resolution control 1108, indicating that the current image resolution is 4:3.

[0257] Further, the user can click on the image resolution control 1108 to change to Fig.11 The situation shown in (b) in the figure. Fig.11 In (b), different image resolutions appear below the preview screen for the user to choose, such as 16:9, 4:3, and 1:1. After the user selects a certain image resolution, the preview screen in the first interface 301 will change accordingly. For example, Fig.11 In (b), if the user selects a 1:1 image resolution, black rectangles will appear on the upper and lower sides of the preview image to block it, making the middle image a square with a width and height ratio of 1:1.

[0258] It is understandable that the various interfaces provided in the embodiments of the present application only represent one design method among many different interface designs, and can be changed into other designs and layouts according to the appearance and requirements of the terminal device, and no specific restrictions are made here.

[0259] After the user selects the image resolution in the first interface 301, the preset image size can be determined according to the image resolution, wherein the preset image size can also be understood as the original image size. In this embodiment, the preset image size refers to the image size directly mapped by the image resolution.

[0260] After obtaining the preset image size, the watermark size of the watermark image can be further determined according to the preset image size, wherein the watermark size is less than or equal to the preset image size in a first direction, and the first direction here may include the width direction and the height direction of the image. In this embodiment, the watermark size refers to the size corresponding to the watermark information (or also referred to as the watermark image) added to the image.

[0261] After determining the preset image size and watermark size, for example, the target size can also be determined in combination with the preset image size and the watermark size. In this embodiment, the target size is actually the size of the target image, wherein the target image has been introduced in the above embodiment and will not be repeated here. In one implementation, the target size can be greater than or equal to the preset image size, and the target size is usually greater than the watermark size. It can be understood that the size of the thumbnail image and the output image are both the target size.

[0262] The implementation and relationship of the several sizes introduced here are exemplarily introduced below in conjunction with specific drawings.

[0263] like Fig.12 The figure shows the process of determining the target size in two cases, where different variables are used to represent different lengths. The length unit can be centimeters, millimeters, or pixels, etc. The length unit will be omitted below for the sake of simplicity, which will not affect the understanding of the solution. Fig.12(a) shows a situation where the original size is not changed, or the target size is consistent with the original size, while Fig.12 (b) shows a situation where adding a watermark makes the target size larger than the original size.

[0264] For example, Fig.12 In (a), it is assumed that the preset image size is x in width and y in height according to the image resolution selected by the user. And it is assumed that the watermark template is a date watermark template, and its original size is m in width and n in height. The date watermark template can be set in the upper left corner of the original image, and the size of the date watermark template does not change. In this case, the watermark template can be directly overlaid on the original image without expanding the size of the original image, that is, the target size is still x×y.

[0265] exist Fig.12 (b) shows another situation. The size of the original image is also x×y, and the watermark template is an image border watermark with a pattern, whose original size is p×q. The watermark is a watermark that is added to the original image in the length or width direction. Fig.12 As shown in (b) in the figure, the watermark template with the pattern is added to the bottom border of the original image, and it needs to be adapted in the width direction, so the size of the watermark template needs to be adjusted. The width of the adjusted watermark size is x, which is consistent with the width of the original image, and the height of the adjusted watermark size is z, where the specific size of z can be calculated by enlarging q according to the magnification of x relative to p, thus obtaining the watermark size x×z. Finally, because the center position of the watermark with the pattern in the height direction is aligned with the bottom border of the original image, the target size needs to be expanded based on the original image size, and the height is changed from the original y to y+z / 2, thus determining the target size to be x×(y / 2).

[0266] pass Fig.12The two examples (a) and (b) in the figure show how to gradually calculate the watermark size and the target size based on the preset image size. One reason why the watermark size and the target size need to be calculated is that different watermark templates are added to the original image in different ways. For example, some watermark templates are added to the inside of the original image without affecting the target size, or some watermark templates will be spliced ​​with the original image, and the sizes of the two need to be added together to calculate the target size, or some watermark templates will partially overlap with the original image, then the calculation process will be relatively complicated. Different image resolutions and watermark templates may correspond to different calculation methods, which need to be inferred according to the specific situation. The embodiments of the present application do not impose further restrictions on this. It is only necessary to satisfy the principle that the watermark size is less than or equal to the size of the preset image in the first direction, and the target size is greater than or equal to the size of the preset image.

[0267] S803: The first application responds to the first interface to trigger a first operation of taking a photo, and the first application sends first information to the HAL.

[0268] This step corresponds to Fig. 9 The interaction process indicated by numbers 2 to 5 in the figure is described in sequence below:

[0269] Corresponds to Fig. 9 In the label 2, after the user adjusts the picture to be photographed in the first interface 301, the user performs a first operation, for example, the first operation may be clicking the photo control 302, and the first application will first send a first message to the camera service of the framework layer. The first message may include a first enabling message and a target size, etc., wherein the first enabling message is used to inform the HAL that the current first application needs to use the chip capability exposure node.

[0270] Next reference Fig. 9 In the interactive process indicated by the numbers 3, 4 and 5 in FIG, the camera service of the framework layer transmits the first information to the HAL. For example, the first information can be transmitted to the HAL through the AIDL for cross-process communication. After receiving the first information transmitted by the framework layer, the HAL further transmits it to the image processing engine through the camera hardware interface development kit (CHI-CDK) and the adapter module.

[0271] S804: HAL applies for a first cache according to the target size in the first information.

[0272] After receiving the first information sent by the first application, the HAL calculates the cache space required to generate the thumbnail image with the watermark according to the target size contained in the first information. In this embodiment, the cache used to carry the thumbnail image is called the first cache. Then the HAL can apply for the first cache from the system. For example, for image data in RGB format, each pixel needs to be represented by three independent color channels, and each color channel of each pixel can be represented by one byte, such as 0~255. Then for a picture with a target size of 10×10 pixels, a cache size of 300 bytes is required.

[0273] S805 . The HAL sends the first cache, the first original image, and the watermark filling information to the first application through the open node in response to the first enabling message in the first information.

[0274] In addition to applying for the first cache, the HAL can also register and add a chip capability exposure node in the image data processing pipeline according to the first enabling message in the first information. Fig. 9 In the interactive process indicated by numbers 6 and 7, the chip capability opening node can send the first cache, the first original image and the watermark filling information to the first application through the service module in the image engine. Among them, the first cache can be used for the processing process of adding watermarks to thumbnail images, and to provide a carrying space for thumbnail images with completed watermarks. The first original image and the watermark filling information can be used as input parameters of the watermarking processing method. The first original image here corresponds to the original image of the thumbnail image, and the watermark filling information contains the data information required by the target watermark template, such as the current time, address and related lens information. The reason why the HAL or the chip capability opening node sends the watermark filling information to the first application is that some of the relevant information can be more conveniently obtained by the HAL, and some of the relevant information can only be obtained by the HAL.

[0275] In some implementations, the open node of chip capabilities may also include relevant interfaces for image processing in the first information, for example, an interface for image data format conversion based on GPU acceleration capabilities.

[0276] S806: The first application generates a first watermark image according to the target watermark template and the watermark filling information.

[0277] It is understandable that the watermark template generally needs to be combined with watermark filling information to generate the watermark image to be displayed, which is referred to as the first watermark image in the embodiment of the present application. Fig.13As shown in FIG. 1 , a method for generating a first watermark image is shown. For example, the target watermark template is a watermark template 1302 containing a date and an address, and the watermark filling information 1301 contains the relevant information of the current time and the current address. Then, the corresponding date and address fields in the watermark filling information 3101 can be filled into the corresponding positions of the target watermark template 3102, and the watermark of August 2, 2024 and the village C of county B of city A is obtained, thus generating Fig.13 The first watermark image 1303 is shown.

[0278] S807: The first application calls the open node to convert the format of the first original image into a target format.

[0279] In the embodiment of the present application, the open node or the chip capability open node is an important module. In order to better understand the role of the chip capability open node in the image processing method of adding a watermark when taking a photo, you can refer to Fig. 9 Related content in .

[0280] like Fig. 9 As shown, Fig. 9 The interaction process indicated by the labels a and b in the figure indicates that the open node can interact with the chip capability. The chip capability here can be regarded as a driver that can control the chip hardware. The chip capability can include multiple functions, such as heterogeneous acceleration, GPU acceleration, and memory pool.

[0281] The chip capability open node itself can have multiple chip capabilities, and can encapsulate the functions of various chips into functions or APIs for use by other modules, while ignoring the details of interaction with hardware. At the same time, the chip capability open node can also be flexibly added to the pipeline, and use the open node to interact with the first application, including sending messages and data, and providing a calling interface.

[0282] Considering that the original UBWC format of the thumbnail image is a closed-source private format, it is difficult to add watermarks directly to it, and a large amount of parallel computing power is required to ensure the conversion speed. Then, after having an open node with chip capabilities, the first application can quickly convert the format of the thumbnail image to the target format by calling the corresponding GPU acceleration interface of the chip capability open node. For example, it can be converted from UBWC format to YUV format, which is easy to add watermarks.

[0283] S808: The first application generates a thumbnail image based on the first original image and the first watermark image, and displays the thumbnail image in the first interface.

[0284] In the previous steps, the first application obtains the first original image and the first watermark image, and uses the open node of the chip capability to convert the first original image from its original format into a template format that is easy to add a watermark, and also obtains the first cache or the address of the first cache. Fig. 9 In the interactive process indicated by the label 8, the relevant data is passed from the thumbnail image acquisition node in the watermark adding module to the watermark algorithm node, so that the first application can complete the watermarking process on the first cache. Fig.14 A process for generating a thumbnail image with first watermark information is described in detail.

[0285] like Fig.14 As shown, in Fig.14 In the illustrated scenario, it is assumed that the watermark is set in the extension direction of the long side or height of the original image, and a part of the watermark image overlaps with the original image. Fig.14 (a) in the figure is the first original image corresponding to the thumbnail image. Fig.14 (b) is the first buffer, which can be regarded as a blank area when no image data is added to it. First, the first original image is written into the first buffer, and Fig.14 Here, since the target size is larger than the original image size and the original image is aligned with the upper border of the first buffer, there is still an area below the first buffer that is not filled with any image data. Then, Fig.14 The image data of the first watermark image in (d) is directly written and overwritten to the lower area of ​​the first buffer, and the result is Fig.14 (e), here the efficiency of adding watermarks can be improved by directly covering the overlapping part of the first original image with the image data of the watermark image.

[0286] In some implementations, the first watermark image is a semi-transparent watermark image. In this case, the overlapping portion of the first watermark image and the first original image should not be directly overwritten, but the image data of the superimposed image of the two should be calculated first and then written into the corresponding position of the first cache.

[0287] S809: The first application displays the thumbnail image including the first watermark information in the second interface in response to the second operation on the thumbnail image.

[0288] Corresponds to Fig. 9 The target image display module of the second interface in the second operation is assumed here that the output image has not been generated when the user performs the second operation, so what is displayed in the second interface is actually a thumbnail image. For more information about this step, please refer to Figure 7 S702 in the description will not be described in detail here.

[0289] S810. The HAL applies for a second cache according to the target size in the first information.

[0290] Corresponding to the HAL applying for the first cache in S804, the second cache is applied here for the subsequent processing of adding watermarks to the output image. It can be understood that, as mentioned above, two pipelines can be constructed in the HAL image processing engine, where the foreground processing pipeline corresponds to generating thumbnail images, and the background processing pipeline corresponds to generating output images, and the two pipelines are processed in parallel.

[0291] That is to say, S810 can be executed synchronously with S804 in the execution order, while S805 to S809 need to be executed after S804, and the corresponding S811 to S817 need to be executed after S810.

[0292] S811. The HAL sends a second cache, a second original image, and watermark filling information to the first application through an open node in response to a first enabling message in the first information.

[0293] This step corresponds to Fig. 9 In the interactive process indicated by numbers 9, 10 and 11, the open nodes in the background processing pipeline successively pass through the service module of the image processing engine, and the camera post-processing service of the framework layer passes the above-mentioned relevant data to the watermark algorithm node of the watermark module added in the first application.

[0294] Compared with the relevant content in S805, the main difference between this step and S805 is that the first original image in S805 corresponds to the thumbnail image, and the second original image in this step corresponds to the output image, and there are differences in data format and image algorithm processing. In addition, the first buffer in S805 is used to add a watermark to the thumbnail image, and the second buffer in this step is used to add a watermark to the output image.

[0295] In some implementations, since the watermark filling information in this step and the watermark filling information in S805 may be consistent, and since the processing of thumbnail images is faster than the processing of output images, the first application may temporarily save the watermark filling information after obtaining it in S805. In this way, the watermark filling information may be omitted in this step, and the temporarily stored watermark filling information may be used to avoid repeated transmission of information.

[0296] S812: The first application generates a first watermark image according to the target watermark template and the watermark filling information, and adds the second original image and the first watermark image to the second cache to generate an intermediate image.

[0297] Similar to S806 and S808, in this step, the first watermark image is generated first, and then the intermediate image is generated. Among them, the format of the second original image itself already supports the processing of adding watermarks, so there is no need to convert the data format with the help of open nodes like S807. The above-mentioned intermediate image refers to the image data obtained after adding the watermark, which will also undergo subsequent image processing and format conversion, and is not the final output image. For example, the intermediate image here can be image data in YUV format.

[0298] In some implementations, considering that the first watermark image used to generate the thumbnail image and the intermediate image of this step contains the same first watermark information and the watermark size is also the same, the first watermark image can be temporarily saved after it is generated in S806. In this way, in this step, it is not necessary to repeatedly generate the first watermark image, but directly obtain the already generated first watermark image and write it into the second cache, which speeds up the execution speed of the method flow.

[0299] S813: The first application crops the second original image, and the image information of the intermediate image includes the cropped hidden image.

[0300] In the embodiment of the present application, the first application crops the second original image and saves the cropped hidden image in the image information of the intermediate image so that the watermarked photo can be edited again later. Fig.15 The contents in are explained.

[0301] like Fig.15 As shown in (a) of the figure, the target size of the watermarked intermediate image is x×(y+z / 2). The meaning of the size variable can be referred to in the previous article. Fig.11 Where x represents the width of the target size, y represents the height of the second original image, z represents the height of the first watermark image, and y+z / 2 represents the height of the target size. That is, after adding the first watermark image to the second original image, Fig.15 In (b), the area at the bottom of the second original image divided by the dotted line with a height of z / 2 will be blocked to a certain extent. In order to be able to edit or restore the original image later, it is necessary to save the image data of this blocked image area. Fig.15 As shown in (c), the second original image is cropped, and the area below it that is blocked by the first watermark image can be called a hidden image, and the image information corresponding to the intermediate image includes the hidden image.

[0302] S814. The first application sends the intermediate image and image information of the intermediate image to the HAL.

[0303] This step corresponds to Fig. 9 In the interaction process indicated by numbers 12, 13, and 14, the first application passes the above data information to the open node through the camera post-processing service of the framework layer and the service module of the image processing engine of HAL. Among them, sending the intermediate image can be understood as sending the address of the second cache corresponding to the intermediate image, or a copy of the second cache. It can also be seen from here that in the process of interaction between the application layer and HAL, especially in the process of performing image processing to add watermarks, it is mainly the first application and the chip capability exposure node that interact, which further enhances the decoupling between the application layer and HAL.

[0304] S815. HAL performs encapsulation according to the intermediate image and the image information to obtain an output image.

[0305] This step corresponds to Fig. 9 In some implementations, after receiving the intermediate image and the corresponding image information sent by the first application, the HAL sends them to the image data format conversion node in the pipeline, for example, Fig. 9 . In this node, the intermediate image is converted and compressed, and the image information is encapsulated into the converted image data. For example, the intermediate image is converted from YUV format to JPEG format, and the image information is attached to the JPEG comment in a specific form.

[0306] S816. The HAL sends the output image to the first application.

[0307] This step corresponds to Fig. 9 In the interactive process indicated by reference numerals 16 and 17, after HAL completes the format conversion and compression of the intermediate image and the corresponding image information, HAL sends the output image to the first application.

[0308] S817: When the target image is a thumbnail image, the first application displays an output image corresponding to the thumbnail image in the second interface.

[0309] This step and Figure 7 The content of S703 in is consistent with that in the previous section and will not be repeated here.

[0310] After describing in detail each step in the image processing method flow in the embodiment of the present application, a corresponding summary description is then given.

[0311] Here we first summarize the method flow of the embodiment of the present application. The user selects the image resolution and the watermark template in the first interface, and then the target size can be calculated. After the user triggers the photo, the first application sends the first information to the HAL. According to the first information, the HAL constructs two pipelines in parallel, which are used to generate thumbnail images and output images respectively. In both pipelines, a chip capability open node is registered, through which the data and information required for adding watermarks can be transmitted to the first application, and the necessary format conversion of the thumbnail images can be performed. In the first application, a watermark can be added to the thumbnail image after the format conversion, and a watermark is also added to the intermediate image corresponding to the output image, and the original image corresponding to the output image is cropped and the cropped hidden image is saved, and then the intermediate image and the hidden image are encapsulated to obtain the output image, which is the final photo with a watermark. The method of the embodiment of the present application has many beneficial effects, which are explained one by one below:

[0312] First, the embodiment of the present application migrates the main processing logic of adding watermarks from HAL to the application layer, interacts more with the chip capability exposure node during the processing, and the watermark material resources required for adding watermarks no longer need to be obtained locally by HAL, but are instead obtained from the server in the cloud, thus achieving the decoupling of the application layer and HAL. In other words, HAL no longer needs to worry about the specific processing logic of adding watermarks when taking photos, nor does it need to synchronously update the corresponding data and functions when the function of the first application is updated. It only needs to provide relevant general functional interfaces and transmit relevant data. This can reduce the maintenance difficulty of HAL on the one hand, and make the development and iteration of the application layer more flexible on the other hand.

[0313] As mentioned above, since watermark resources are obtained from the cloud server, this can greatly enrich the watermark template resources available for users to choose from and save local storage space. It is understandable that through the rich watermark template resources, users can find suitable or favorite watermark templates, improve the user experience, and then more likely to use the function of adding watermarks when taking photos, so the series of related optimizations and improvements made in the embodiments of the present application will be more meaningful.

[0314] In addition, the method of adding a chip capability exposure node to add a watermark to a thumbnail image so that the image content seen by the user remains consistent, thereby enhancing the user experience, is the same as the previous embodiment and will not be repeated here.

[0315] Considering that users not only have a better terminal experience of adding watermarks when taking photos, they also hope to be able to edit the watermarks on the photos again after taking the photos. Fig.16 and Fig.17The method of re-editing the watermark is described in detail, and some steps in the above embodiments are also involved as the pre-steps of re-editing the watermark. Fig.16 Schematic diagram of the implementation of watermark re-editing provided in the embodiment of the present application Figure 1 , Fig.17 Schematic diagram of the implementation of watermark re-editing provided in the embodiment of the present application Figure 2 .

[0316] The method for re-editing the watermark of a photo provided in the embodiment of the present application is based on the embodiment in the previous text. In a broad sense, re-editing the watermark of a photo can include removing the watermark and restoring the original image, or modifying the originally added watermark into a new watermark. The latter mentioned above can be divided into modifications on the original watermark template, such as modifying the watermark fill information therein, or replacing it with a completely new watermark corresponding to another watermark template.

[0317] In order to edit the watermark of a photo again, it is generally necessary to first complete or restore the image information that was lost due to the watermark. As long as the original image can be restored, a series of subsequent processing can be done smoothly. Figure 8 S814 and Fig.15 In the example shown, the portion of the original image that is blocked due to the addition of the watermark is cropped and encapsulated into the output image in S816. The original image can then be restored using the encapsulated cropped image data.

[0318] The user can perform an operation of re-editing the watermark of the photo in the third interface, which can be referred to as the fourth operation in the embodiment of the present application. In response to the fourth operation, the first application applies for the third cache, and decapsulates the output image to obtain the corresponding intermediate image and the image information therein. In the third cache, the intermediate image is cropped to obtain a cropped second original image, and then the hidden image in the image information is used to splice the cropped second original image to restore the second original image. Furthermore, the second watermark image corresponding to the second watermark information can be added to the restored second original image, thereby completing the addition of the new watermark. After restoring the second original image or adding a new watermark, the adjusted target image will be displayed in the third interface.

[0319] For specific examples, please refer to Fig.16 as well as Fig.17 The content displayed in.

[0320] exist Fig.16 In Fig.16As shown in (a), the watermark in the output image is a watermark with a pattern added to the bottom border of the original image. Therefore, the output image is first cropped, and the two parts obtained by cropping are the watermark image part 1603 and the original image part 1601 that does not contain the watermark image. Then Fig.16 In (b), the portion 1601 of the original image that does not contain the watermark image obtained by cropping is spliced ​​with the portion 1602 originally blocked by the watermark image obtained from the image information, thereby obtaining the following: Fig.16 The original image 1600 shown in (c) thus realizes the restoration of the target image.

[0321] exist Fig.17 In , another situation is shown, such as Fig.17 As shown in (a), the watermark in the output image is a date watermark and is located above the output image. If the user thinks that the date in the date watermark is inappropriate and wants to adjust the value, for example, from August 2 to August 5, then the watermark needs to be edited again. Fig.16 Similar to Fig.17 In (a), the output image is first cropped into the watermark image portion 1703 and the original image portion 1701 that does not contain the watermark image. Fig.17 In (b), the portion 1701 of the original image that does not contain the watermark image and the portion 1702 that was originally blocked by the watermark image are spliced ​​to restore Fig.17 The original image 1700 in (c) is shown in FIG. 17. Finally, new watermark information 1704 is added to the original image 1700 to obtain a new image with a different date watermark, thus completing the re-editing of the photo watermark.

[0322] In the embodiment of the present application, the original image is restored by encapsulating a hidden image in the output image, thereby realizing the re-editing of the watermark. It is understandable that when taking pictures, the user may not select the most appropriate watermark for some reasons, but is eager to capture the rare moment with the camera first, or in some cases, when viewing the photos later, the user thinks that the watermark is inappropriate and hopes to remove or modify it. These are relatively common needs. The embodiment of the present application not only realizes the re-editing of the watermark of the photo, but also combines it well with the method of adding watermarks when taking pictures. It can be used as a set of functions of the first application to give users the best user experience.

[0323] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0324] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0325] The method for adding a watermark when taking a photo in the embodiment of the present application has been described above, and the device for executing the above method is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device can execute the steps in the above method for adding a watermark when taking a photo.

[0326] The method of adding a watermark when taking a photo can be applied to an electronic device with a photo taking function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be repeated here.

[0327] In one implementation, an embodiment of the present application provides an electronic device, Fig.18 Schematic diagram of the hardware structure of the electronic device.

[0328] like Fig.18 As shown, the electronic device 180 includes: a processor 1801 and a memory 1802; the memory 1802 stores computer-executable instructions; the processor 1801 executes the computer-executable instructions stored in the memory 1802, so that the electronic device 180 executes the above method.

[0329] When the memory 1802 is independently provided, the electronic device further includes a bus 1803 for connecting the memory 1802 and the processor 1801 .

[0330] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0331] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0332] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage device, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile discs (DVD), floppy disks and Blu-ray discs, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0333] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: Applied to a terminal device, the method comprises: In response to a first operation for triggering a photo-taking operation in a first interface, a thumbnail image is displayed in the first interface, the thumbnail image includes first watermark information, and the size of the thumbnail image in the first interface is a first size; the thumbnail image displayed in the first interface is obtained by the application layer by calling an interface of a chip with GPU acceleration capability corresponding to an open node in a hardware abstraction layer HAL, performing format conversion on an initially generated thumbnail image and adding the first watermark information; In response to a second operation on the thumbnail image displayed in the first interface, displaying a target image in a second interface, the target image including the first watermark information, and the size of the target image in the second interface being a second size, the second size being larger than the first size; The target image is a thumbnail image displayed in the first interface, or the target image is an output image corresponding to the thumbnail image displayed in the first interface.

2. The method according to claim 1, characterized in that The application layer includes a first application, and the first interface and the second interface are interfaces in the first application.

3. The method according to claim 2, characterized in that The method further comprises: In response to a third operation for selecting a watermark template in the first interface, a target watermark template is determined, and the first watermark information is generated according to the target watermark template.

4. The method according to claim 3, characterized in that After determining the target watermark template, the method further includes: Determine a preset image size according to an image resolution selected by the user in the first interface; Determine a watermark size according to the preset image size, wherein a length of the watermark size in a first direction is less than or equal to a length of the preset image size in the first direction; A target size is determined according to the preset image size and the watermark size, wherein the target size is greater than or equal to the preset image size, and the target size is greater than or equal to the watermark size; and the sizes of the thumbnail image displayed in the first interface and the output image are both the target size.

5. The method according to claim 4, characterized in that The response acts on a first operation for triggering a photo-taking operation in a first interface, and a thumbnail image is displayed in the first interface, including: In response to the first operation, the first application sends first information to the HAL, where the first information includes a first enable message and the target size, and the first enable message is used to indicate the use of the open node; The HAL applies for a first cache according to the target size, and the size of the first cache is the target size; In response to the first enable message, the HAL sends the address of the first cache, the first original image, and watermark filling information to the first application through the open node, where the first original image is an initially generated thumbnail image; The first application generates a thumbnail image displayed in the first interface according to the first cache, the first original image, the watermark filling information, and the target watermark template; The first application displays the thumbnail image in the first interface.

6. The method according to claim 5, characterized in that The first application generates a thumbnail image displayed in the first interface according to the first cache, the first original image, the watermark filling information, and the target watermark template, including: The first application generates a first watermark image corresponding to the first watermark information according to the target watermark template and the watermark filling information; The first application adds the first original image and the first watermark image in the first cache to generate a thumbnail image displayed in the first interface.

7. The method according to claim 6, characterized in that The method further comprises: The first application calls the open node to convert the format of the first original image into a target format; wherein the first original image converted into the target format is added to the first cache.

8. The method according to any one of claims 5 to 7, characterized in that: In response to the first operation, after the first application sends first information to the HAL, the method further includes: The HAL applies for a second cache according to the target size; In response to the first enable message, the HAL sends the address of the second cache, the second original image, and the watermark filling information to the first application through the open node, where the second original image is an initially generated output image; The first application generates an intermediate image according to the second cache, the second original image, the watermark filling information and the target watermark template; The first application sends the intermediate image and image information of the intermediate image to the HAL; The HAL performs encapsulation according to the intermediate image and the image information to obtain the output image.

9. The method according to claim 8, characterized in that The first application generates an intermediate image according to the second cache, the second original image, the watermark filling information, and the target watermark template, including: The first application generates a first watermark image corresponding to the first watermark information according to the target watermark template and the watermark filling information; The first application adds the second original image and the first watermark image in the second cache to generate the intermediate image.

10. The method according to claim 9, characterized in that The method further comprises: The first application crops the second original image to obtain a cropped second original image and a cropped hidden image, and the image information of the intermediate image includes the hidden image.

11. The method according to claim 10, characterized in that The method further comprises: In response to a fourth operation for adjusting the first watermark information, an adjusted target image is displayed in a third interface, wherein the adjusted target image does not include the watermark information, or the adjusted target image includes the second watermark information.

12. The method according to claim 11, characterized in that The step of responding to the fourth operation for adjusting the first watermark information and displaying the adjusted target image in the third interface includes: In response to the fourth operation, the first application decapsulates the target image to obtain the intermediate image and image information of the intermediate image; The first application obtains the cropped second original image according to the intermediate image; The first application restores the second original image according to the cropped second original image and the hidden image included in the image information; The first application obtains the adjusted target image according to the second original image; The adjusted target image is displayed in the third interface.

13. The method according to claim 12, characterized in that The first application obtains the adjusted target image according to the second original image, including: The first application applies for a third cache; The first application adds the second original image to the third buffer to obtain an adjusted target image, wherein the adjusted target image does not include watermark information; or The first application adds the second original image and the second watermark image to the third cache to obtain an adjusted target image, wherein the adjusted target image includes second watermark information corresponding to the second watermark image.

14. The method according to claim 1, characterized in that In a case where the target image is a thumbnail image displayed in the first interface, after the target image is displayed in the second interface, the method further includes: An output image corresponding to the thumbnail image displayed in the first interface is displayed in the second interface, the output image includes the first watermark information, and the size of the output image in the second interface is a third size, which is larger than the first size.

15. An electronic device, characterized in that: The electronic device comprises: a memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the method according to any one of claims 1 to 14.

16. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 14.

Images